* update benchmark for transformers 4.* and ORT 1.7
* Fix gpt2 onnx conversion for transformers 4.3.*. Add a check of transformer version >= 3.1.
* remove code related to openmp
* update pretrain model list: keep representitive models only
* Add robust dependency check for Python package
* Add version_info.py to .gitignore
* Fix Linux build
* Fix Windows CPU build
* Fix Windows 32-bit build
* Minor tweak
* Generate version_info.py earlier in onnxruntime_python.cmake
* Print a user-friendly message if cuDNN is not found in
* Relax version requirements for CUDA 11 - only the major version has to match
* Fix PATH environment variable to include CUDA 11 in 'Python packaging pipeline' (Windows/GPU)
* Fix the build with cuDNN 7
* Add support for custom ops library to the ORT model conversion script
Simplify model conversion now that we read ops from the ORT format model.
Enable custom ops in the python bindings if custom ops are turned on in a minimal build.
* Add test of model conversion involving custom ops.
* Integrate memory improvements from NVidia
* compute max_global_num before buffer allocation
* update conversion script to support transformers 4.0
* update benchmark script for creating dummy inputs for different batch_size
* Use a wrapper of cuda event to avoid memory leak
* Support to allow user to specify compute stream per session
Create computation cuda stream explicitly rather than use default legacy stream or per-thread default stream.
remove some redudant cudaStreamSynchronize
fix gpt2 model test failures
don't use default stream in nccl either.
add stream schronization in OnRunEnd()
using cub::DeviceScan::InclusiveSum which can be called with stream specified.
fix topK failure due to latest rebase
fix tensorrt
support user specified stream
add user_stream support in tensorrt EP
use same stream for both tensort and CUDA EP.
fix ScatterND
specify stream for adasum and p2p kernels.
fix loop
fix CApiTest.custom_op_handler
fix CApiTest.varied_input_custom_op_handler
change for cudaMemcpyFromSymbol
improve provider options for user specified compute stream
* add changes for ROCM EP
* fix GatherGrad UT for ROCM EP
* clean code and fix NonMaxSuppression
* use default stream for ROCM now
* fix CApiTest.custom_op_handler:OrtFormatCustomOpTests.ConvertOnnxModelToOrt
* fix tensorrt ut: CApiTest.io_binding_cuda
Co-authored-by: Weixing Zhang <wezhan@microsoft.com>
* Refactor the code and move data reader, preprocessing, evaluation to
E2E_example_mode
* Refactor the code.
Move data reader, preprocessing, evaluation to model specific example
under E2E_example_mode
* refactor code
* Move yolov3 example to specific folder and add additional pre/post
processing
Support pad operator in quantization tool.
Support pad operator in quantized nhwc transformer.
Fix pad() operator bug when pad input's inner(right) most axis value is zero for Edge and Reflect mode, it copied wrong value to the cells to be padded. Note the Constant mode will not trigger this bug, as Edge/Reflect need copy value from the already copied array while Constant mode only fill specified value.
Add more test cases to cover pad() operator bug fixed here.
Fix quantization tools uint8/int8 value overflow issue when quantize weights in python.
* Make split working for quantization.
* NHWC transformer support for split operator
* Refactor some according to Feedback. Will add test cases soon.
* Fix build error on windows.
* Add test case for split op on uint8_t support
* Add nhwc_transformer_test for split uint8_t support
* Some change according to PR feedbacks.
* longformer fp16 e2e
* add fp16/fp32 parity check helper file
* excludes nodes with subgraph in profiling
* use onnxconverter_common to do fp32->fp16
* add version check for onnxconverter_common
* remove helper file
* add pkg installation on notebooks and script
Update Python API to allow more flexibility for setting providers and provider options.
The providers argument (InferenceSession/TrainingSession constructors, InferenceSession.set_providers()) now also accepts a tuple of (name, options dict).
Fix get_available_providers() API (and the corresponding function in the C API) to return the providers in default priority order. Now it can be used as a starting point for the providers argument and maintain the default priority order.
Convert some usages of the deprecated global configuration functions to use EP-specific options instead.
Update some EP-specific option parsing to fail on unknown options.
Other clean up.
Provide a tool to convert Loop to Scan for Nuphar performance
Fix Nuphar CI pipeline failures.
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* merge master, keep postprocess status commit
* download float16.py everytime
* using variables to reference eps
* adding ACL EP to ep perf tool
* accuracy with absolute tolerance configurable
* add acl to dict + remove commented line
* Initial implementation of generating calibration dynamic range table
* Initialize validation support for Quantization
* Initialize validation support for Quantization (cont.)
* Improve validation support for Quantization
* Improve validation support for Quantization
* Rewrite/Refine for calibration and validation
* Rewrite/Refine for calibration and validation (cont.)
* Refine code
* Refine code
* Add data reader for BERT
* Add flatbuffers to serialize calibration table
* Refine code and add BERT evaluation
* Refine the code
* minor modification
* Add preprocess/postprocess of vision team yolov3 and refine the code
* Update annotation
* Make bbox cooridates more accurate
* Fix bug
* Add support of batch processing
* Batch processing for model zoo yolov3
* Add batch inference for evaluation
* Refine the code
* Add README
* Add comments
* Refine the code for PR
* Remove batch support checking in data_reader and refine the code
* Refine the code for PR
* Refine the code for PR review
Co-authored-by: Olivia Jain <oljain@microsoft.com>
* Support to pass initial optimizer states to optimizer graph builder
* Changes for passing init optim state to training session config
* Pass optimizer state through cpp and python frontend
* Cleanup
* Review comments
* Fix windows and mac CI
* Review comments
* review comments
* Review comments
* Frontend review changes
* Fix CI
