onnxruntime/docs
Adam Louly ee74fb6908
Introducing ORTPipelineModule - DeepSpeed Parallel Pipeline Support. (#20287)
### Description
Introducing a new class ORTPipelineModule to handle wrapping layers in
DeepSpeed pipeline parallel.


### Motivation and Context
To support pipeline parallelism on ORTModule.

This PR will include an initial support of deepspeed Pipeline
parallelism.

- [x] Support Pipeline parallel where layers are nn Modules in
Sequential.
- [ ] Support LayerSpec and TiedLayerSpec
- [ ] Enable partitioning to accept List
- [ ] Full-GPU Graph Consolidation
- [ ] Subgraph Merging for Inference
2024-04-18 11:30:15 -07:00
..
c_cxx
execution_providers/images
images
python
ABI_Dev_Notes.md
Android_testing.md
C_API_Guidelines.md
cmake_guideline.md
Coding_Conventions_and_Standards.md
ContribOperators.md
FAQ.md
How_To_Update_ONNX_Dev_Notes.md
Memory_Optimizer.md
Model_Test.md
NotesOnThreading.md
ONNX_Runtime_Server_Usage.md
onnxruntime_dependencies.dot
onnxruntime_dependencies.png
onnxruntime_extensions.md
OperatorKernels.md
ORT_Format_Update_in_1.13.md
ORT_Use_Trtion_Kernel.md
ORTMobilePackageOperatorTypeSupport.md
ORTModule_Convergence_Notes.md
ORTModule_ModuleWithLoss_Wrapper.md
ORTModule_PythonOp_Notes.md
ORTModule_Training_Guidelines.md
PR_Guidelines.md
Privacy.md
Python_Dev_Notes.md
Reduced_Operator_Kernel_build.md
ReleaseManagement.md
Roadmap.md
Server.md
TVM_EP.md
Versioning.md
WinML_principles.md