Will Feng bc9597b7d8 [Traceable FSDP2] Minor refactor to traceable FSDP2 unit tests (#136219) ... Changes in this PR: - Monkey-patching `F.scaled_dot_product_attention` with a lambda seems to not work in some cases. This PR avoids using a lambda. - Running `fullgraph=True` and `fullgraph=False` in the same unit test seems to cause the two cases to interfere with each other and causes error. This PR splits them into two separate unit tests. - The checks in the unit tests might not work with compile cache. This PR turns off the cache in order to have a more predictable compile behavior to do unit test on. Test commands: - `pytest -rA test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_inductor_fullgraph_True` - `pytest -rA test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_inductor_fullgraph_False` - `pytest -rA test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_inductor_fullgraph_True` - `pytest -rA test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_inductor_fullgraph_False` Pull Request resolved: https://github.com/pytorch/pytorch/pull/136219 Approved by: https://github.com/yifuwang		2024-09-18 22:30:23 +00:00
..
_composable	[Traceable FSDP2] Minor refactor to traceable FSDP2 unit tests (#136219)	2024-09-18 22:30:23 +00:00
_shard
_tensor	Revert "Add decomposition for permute_copy (#130944)"	2024-09-17 13:42:55 +00:00
_tools	Runtime Estimator for estimating GPU compute time (#134243)	2024-08-28 20:06:54 +00:00
algorithms	Fix ROCm skip decorator for test_ddp_tp and multiprocess UTs (#136161)	2024-09-18 11:01:23 +00:00
bin
checkpoint	[DCP] Fixes the stateless optimizer issue of distributed state_dict (#135535)	2024-09-10 03:10:00 +00:00
elastic	Adding entry-point based support for out-of-tree rendezvous plugins (#132633)	2024-09-11 03:35:02 +00:00
flight_recorder	[FR] Make pg_name unique, show P2P collective status and fix bugs when running the script as command (#134780)	2024-08-30 18:03:17 +00:00
fsdp	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
launcher
nn/jit
optim	Fix ROCm skip decorator for test_ddp_tp and multiprocess UTs (#136161)	2024-09-18 11:01:23 +00:00
pipelining	[PP] Fix zero bubble composability with DP (#134052)	2024-09-04 23:46:29 +00:00
rpc
tensor/parallel	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
argparse_util_test.py
test_c10d_common.py	[c10d][Reland] Remove Option for ProcessGroup and Expose backend Options to reflect the correct code structure (#132931) (#135653)	2024-09-16 19:56:42 +00:00
test_c10d_functional_native.py	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
test_c10d_gloo.py
test_c10d_logger.py
test_c10d_nccl.py	Fix ROCm skip decorator for test_ddp_tp and multiprocess UTs (#136161)	2024-09-18 11:01:23 +00:00
test_c10d_object_collectives.py
test_c10d_ops_nccl.py
test_c10d_pypg.py
test_c10d_spawn.py
test_c10d_spawn_gloo.py
test_c10d_spawn_nccl.py
test_c10d_spawn_ucc.py
test_c10d_ucc.py
test_collective_utils.py
test_compute_comm_reordering.py	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
test_control_collectives.py
test_data_parallel.py
test_device_mesh.py	[c10d][Reland] Remove Option for ProcessGroup and Expose backend Options to reflect the correct code structure (#132931) (#135653)	2024-09-16 19:56:42 +00:00
test_distributed_spawn.py
test_dynamo_distributed.py	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
test_fake_pg.py
test_functional_api.py	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
test_inductor_collectives.py	Revert "Add Triton CPU as an Inductor backend (#133408)"	2024-09-16 18:33:33 +00:00
test_launcher.py
test_multi_threaded_pg.py
test_nccl.py
test_pg_wrapper.py
test_store.py
test_symmetric_memory.py	[CUDA][P2P] Check device capability in requires_cuda_p2p_access (#134523)	2024-08-30 14:08:55 +00:00