onnxruntime/orttraining
pengwa b457cfaa8f
Enable conditional optimization automatically (#15885)
### Enable conditional optimization on inputs

Label sparsity based optimization can be enabled depending on the input
inspection result.

So this PR introduce a conditional optimization path for ORTModule,
where we automatically detect data sparsity from label or embedding, and
enable the graph optimization accordingly without any user interaction.

This feature had a new requirement of delaying passing pre_grad graph
transformation config to OrtModuleGraphBuilder, from `Initialize` phase
to its `Build` phase. Because once after `_initialize_graph_builder` we
can detect the input sparsity, and make a decision to enable the
label/embed sparisty based graph optimizations.

Add UT cases for label/embed input runtime inspector.
2023-05-23 13:08:05 +08:00
..
orttraining Enable conditional optimization automatically (#15885) 2023-05-23 13:08:05 +08:00
pytorch_frontend_examples Enable pylint and numpy rules (#15218) 2023-03-27 20:37:53 -07:00
tools [ROCm] reduce batch size to fix CI error (#15714) 2023-05-16 13:10:02 +08:00