saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/docs/source/fsdp.rst

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

47 lines
1.1 KiB
ReStructuredText
Raw Normal View History

make fsdp folder to be public (#72084) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/72084 make fsdp folder to be public ghstack-source-id: 148173447 Test Plan: unit tests Reviewed By: mrshenli Differential Revision: D33903417 fbshipit-source-id: 7852a2adc4af09af48a5ffa52ebf210489f834d5 (cherry picked from commit bd06513cfe2f391941bb0afa611dd39994585513)
2022-02-02 15:18:49 +00:00
FullyShardedDataParallel
========================
.. automodule:: torch.distributed.fsdp
.. autoclass:: torch.distributed.fsdp.FullyShardedDataParallel
:members:
[FSDP()][3/N] Refactor public APIs (#87917) - This PR defines a new `api.py` meant to hold the public API for FSDP (minus `FullyShardedDataParallel` itself). This is needed because several of the `_<...>_utils.py` files rely on the public API, and we cannot import from `torch.distributed.fsdp.fully_sharded_data_parallel` without a circular import. Calling the file `api.py` follows the convention used by `ShardedTensor`. - This PR cleans up the wording in the `BackwardPrefetch`, `ShardingStrategy`, `MixedPrecision`, and `CPUOffload` docstrings. - This PR adds the aforementioned classes to `fsdp.rst` to have them rendered in public docs. - To abide by the public bindings contract (`test_public_bindings.py`), the aforementioned classes are removed from `fully_sharded_data_parallel.py`'s `__all__`. This is technically BC breaking if someone uses `from torch.distributed.fsdp.fully_sharded_data_parallel import *`; however, that does not happen in any of our own external or internal code. Pull Request resolved: https://github.com/pytorch/pytorch/pull/87917 Approved by: https://github.com/mrshenli
2022-10-30 15:26:12 +00:00
.. autoclass:: torch.distributed.fsdp.BackwardPrefetch
:members:
.. autoclass:: torch.distributed.fsdp.ShardingStrategy
:members:
.. autoclass:: torch.distributed.fsdp.MixedPrecision
:members:
.. autoclass:: torch.distributed.fsdp.CPUOffload
:members:
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.StateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.FullStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.ShardedStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.LocalStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.OptimStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.FullOptimStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.ShardedOptimStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.LocalOptimStateDictConfig
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
[FSDP][state_dict] Expose optimizer state_dict config (#105949) Optimizer state_dict config are not exposed. This PR exposes the 2 dataclass. Differential Revision: [D47766024](https://our.internmc.facebook.com/intern/diff/D47766024/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105949 Approved by: https://github.com/rohan-varma
2023-08-18 16:45:12 +00:00
.. autoclass:: torch.distributed.fsdp.StateDictSettings
[FSDP][Docs] Make model/optim state dict configs visible in docs (#105848) This closes https://github.com/pytorch/pytorch/issues/104717. Rendered docs: ![Screenshot 2023-07-25 at 11 15 23 AM](https://github.com/pytorch/pytorch/assets/31054793/3c38166a-70c0-472c-805d-452d3bd9c700) ![Screenshot 2023-07-25 at 11 15 30 AM](https://github.com/pytorch/pytorch/assets/31054793/6d275d94-020a-44a2-a64c-0eeba083d47f) Pull Request resolved: https://github.com/pytorch/pytorch/pull/105848 Approved by: https://github.com/rohan-varma
2023-07-25 11:21:15 +00:00
:members:
Reference in a new issue Copy permalink