wz337 8140494afd [3/N][2D] Enable training with new 2D flow (#110034) ... Replacing https://github.com/pytorch/pytorch/pull/109553 as it gets reverted. This PR enables training with new 2D flow and adds associated test. In addition, this PR moves the tensor/parallel/_data_parallel_utils.py that are fsdp specific back to tensor/parallel/fsdp.py to avoid circular dependency for ddp.py and test/distributed/tensor/parallel/test_ddp_2d_parallel.py. state_dict related changes would be in later PRs. cc. @fegin, @fduwjj, @wanchaol, @awgu Pull Request resolved: https://github.com/pytorch/pytorch/pull/110034 Approved by: https://github.com/fduwjj		2023-09-26 09:14:15 +00:00
..
__init__.py
test_ddp_2d_parallel.py	[2D][TP] Enable DDP TP integration with unit test (#106583)	2023-08-17 02:54:17 +00:00
test_fsdp_2d_parallel.py	[3/N][2D] Enable training with new 2D flow (#110034)	2023-09-26 09:14:15 +00:00
test_parallelize_api.py
test_tp_examples.py
test_tp_random_state.py
test_tp_style.py
test_view_sharding_dim_change.py