Andrew Gu 9117779b0a [FSDP2] Added test for N-way TP and 1-way FSDP with CPU offloading (#127024) ... This PR shows that we can use FSDP solely for CPU offloading when composing with N-way TP. Each FSDP mesh is just 1 rank. This was motivated from an ask on Slack :) Pull Request resolved: https://github.com/pytorch/pytorch/pull/127024 Approved by: https://github.com/weifengpy, https://github.com/wanchaol ghstack dependencies: #127004		2024-05-24 17:09:12 +00:00
..
fsdp	[FSDP2] Added test for N-way TP and 1-way FSDP with CPU offloading (#127024)	2024-05-24 17:09:12 +00:00
fully_shard	[BE]: FURB142 - Remove set mutations. Use set update (#124551)	2024-04-21 14:12:33 +00:00
test_checkpoint.py
test_compose.py
test_contract.py
test_replicate.py
test_replicate_with_compiler.py	Prevent partitioner from ever saving views (#126446)	2024-05-22 17:28:46 +00:00