saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/test/distributed/_tensor
History
fduwjj aa84078c6c [PTD][TP] Add BWD support for colwise embedding sharding (#104820) 
Originally, we didn't enable BWD for colwise embedding because we thought it was just for inference, but it turns out that we do need it for training. So, let's enable it for now and unit test is also added.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/104820
Approved by: https://github.com/fegin
2023-07-10 22:33:20 +00:00
..
debug [dtensor] add debug tool to track op coverage (#100124) 2023-05-02 01:45:55 +00:00
__init__.py
README.md
test_api.py Allow ORT backend for DTensor (#101914) 2023-06-01 22:37:09 +00:00
test_basic_strategy.py
test_common_rules.py
test_device_mesh.py DeviceMesh use dispatchable PG to support custom backend (#102336) 2023-05-30 19:22:37 +00:00
test_dtensor.py [dtensor] support torch.save/load with DTensor (#103106) 2023-06-09 04:11:15 +00:00
test_dtensor_ops.py
test_embedding_ops.py [PTD][TP] Add BWD support for colwise embedding sharding (#104820) 2023-07-10 22:33:20 +00:00
test_init.py [reland][DTensor][3/N] add DTensor constructor function: full (#101436) (#103165) 2023-06-08 16:18:33 +00:00
test_math_ops.py
test_matrix_ops.py
test_pointwise_ops.py
test_random_ops.py [DTensor][TP][Random] Introduce TensorParallelRNGTracker to integrate parallel RNG state with Tensor Parallel (#103910) 2023-06-30 08:06:41 +00:00
test_redistribute.py [DTensor] Change Sharding algorithm to be in line with `torch.chunk()` (#98722) 2023-04-21 02:05:22 +00:00
test_tensor_ops.py
test_utils.py [DTensor] Change Sharding algorithm to be in line with `torch.chunk()` (#98722) 2023-04-21 02:05:22 +00:00
test_view_ops.py

README.md

Run distributed tensor tests:

from root, run (either CPU or GPU)

pytest test/spmd/tensor/test_tensor.py

pytest test/spmd/tensor/test_ddp.py

run specific test case and print stdout/stderr:

pytest test/spmd/tensor/test_tensor.py -s -k test_tensor_from_local