onnxruntime/orttraining
Sherlock 8e200e13fe
Rewrite ORTModule background task coordination (#6700)
* Introduce OrtTasks to replace EventPool

* return run_id to frontend

* pass run_id to backward

* OrtTasks support multiple bg_events

* make message_queue a member of orttask

* Replace MessageQueue with std::promise

* Move status_promise into Task

* Move terminate flag into Task

* Reenable previously disabled UTs

* Add unit tests

* Replace condition variables with std::promise

* Move to CreateBackgroundTask in the main thread

* return status and output in forward_future

* use throw for terminating background thread

* cleanup tasks at destructor

* reenable test_mixed_nnmodule_ortmodules_training

* add mutex for ORTTasks functions

* add mutex for bg_threads

* delay tests before start

* add ut for multi-task common backbone

Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2021-02-24 18:00:25 -08:00
..
orttraining Rewrite ORTModule background task coordination (#6700) 2021-02-24 18:00:25 -08:00
pytorch_frontend_examples Sync ORTModule branch with master and fix tests (#6526) 2021-02-02 08:59:56 -08:00
tools Add nightly pipeline for MI100 to run convergence and batch size test similar to V100. (#6611) 2021-02-12 13:22:06 -08:00