* Support quantization linear binary element wise math ops, implement QLinearAdd.
Support tests for quantization linear binary element wise math ops, implement test for QLinearAdd.
Add QlinearAdd with SSE2 intrisinc implemntation, Avx2 assembly implemntation, Neon intrisinc support.
QLinearAdd support VectorOnVector, VectorOnScalar, ScalarOnVector.
Generalized QlinearBinaryOp parallel related with broadcasting.
* Modify according to PR feedbacks. Mainly:
* template helper for generalize the qladd logic on v2v, s2v, v2s
* remove GetKernel related.
* change mixed lagecy MM/SSE code in the AVX code
* formater, typos, convensions, etc.
* Utilize MlasSubtractInt32x4 in MlasDequantizeLinearVector().
* Some format fix.
* More nature parallel parameter type.
* Fix build break for x86.
* Comment goes to 80 before wrap.
* Many change on assembly on Marco related.
Using vminps than vpminsd to handle NaN.
tested on windows.
* Using CLang Format to format the file.
* Fix arm32 build error.
* Remove some duplicate in different #if defined
* working add.u8.vector to vector
* Fix runtime bus error on real arm32 linux.
* fix typo in store last one lane.
* arm32 qlinearadd handle scalar.
* Move qladd to seperate c++ file
* Add neon64 qladd.
* refactor some, enhance two instructions on arm64 only instructions
* Fix typo for arm64
* use strict op in pure c++ (min/max on float value)
* sse2 new version.
* mrege arm/sse2/avx2
* pass arm/sse/avx2 linux test
* remove non-used assembly file.
* Remove unused data definition and tailing spaces.
* Fix broadcasting parallel issue.
* Enhance broadcasting scenarios. Allow testing result diff due to round
on half.
* Add Mlas or MLAS_ prefix for namespace safety.
* Handle alignment issue for arm32 for GCC/MSVC. remove some unused
signed/unsigned int ops.
* Specify /arch:AVX2 for qladd_avx2.cpp
* Fix type during copy/paste when unrolling. Better one GreatEqual
condition. Better formater by splitting two statements on single line.
* Arm neon alignment parameter is bits rather than bytes, change it.
* Move qladd_avx2.cpp to intrinsics/avx2/ folder
* Formatting using mlas style.
* Double check mlas style for these files.
* change indent 2 to 4 for qladd_avx2.cpp
* Fix windows x86 build error due to sse2 no _mm_cvtsi128_si64
* To re-trigger all as old failed pipeline updated.
Co-authored-by: Lei Zhang <phill.zhang@gmail.com>
* Implement BiasDropout Fusion and Kernel
Dropout kernel for residual input
BiasDropout Fusion to take residual input
Fix BiasDropout Kernel
Optimize DropoutGrad with 4 elements per thread
* Add graph transformer UT
* MLTypeCallDispatcher for RatioData
* Use MLTypeDispatcher for ratio tensor
* Handle traing_mode input for BiasDropout fusion
* Add test case for missing ratio input
* Replace using FinalizeNodeFusion
* Make BiasDropout kernel template-less
* Make DropoutGrad template-less
* Make Dropout and TrainableDropout template-less
* Regenerate onnx file for UT
* Minior fix on divmod in BiasDropoutKernel
* Adjust pt frontend test due to dropout randomnesss
* Make dropout kernel opeartion in fp32
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Update function body initialization
* minor fix
* changes per review comments
* minor fix
* format fix
* add function initialization in mixed precision transformer
* more updates
* more fixes
* Support another two format of mask_index input: 2D attention mask, or 1D mask index with end and start positions.
* Update dynamic axes of gpt2 with past state
* Update script to fuse model with attention mask
* add support to internally transpose nchw input to nhwc and only transpose back if it is necessary
* more changes in nchw<->nhc, fixed small issue in concat
* Add option for NNAPI to run on [all device]s/[cpu onl]y/[non-cpu only]
* minor code style changes
* Move allocators to SessionState so they're decoupled from ExecutionProviders
- when looking up an allocator it's based on OrtMemoryInfo not the EP so SessionState is a more natural place for that infromation to be stored
- add device based lookup
- simplifies logic for copying feeds/fetches across devices
Cleanup SessionState and SessionStateInitializer
- provide more things to SessionState at construction time so we don't construct and instance and immediately after call a bunch of setters
- simplify SessionStateInitializer
- reduced down to FinalizeSessionState method
* Make quantization support GPT2 past state
* Make OpTester to be able to generate reference outputs with a model. With it, there is no need to compute outputs manually, which are impossible for some cases.
* cast support for layernormfusion
* cast support for layernormfusion
* bug fix
* fix build
* bug fix
* fix test
* minor refactor
* on comments
* on comments
* on comments
* on comments
Co-authored-by: Ethan Tao <ettao@microsoft.com>
* Move nnapi dnnlib to subfolder
* dnnlib compile settings
* add nnapi buildin build.py
* add onnxruntime_USE_NNAPI_BUILTIN
* compile using onnxruntime_USE_NNAPI_BUILTIN
* remove dnnlib from built in code
* Group onnxruntime_USE_NNAPI_BUILTIN sources
* add file stubs
* java 32bit compile error
* built in nnapi support 5-26
* init working version
* initializer support
* fix crash on free execution
* add dynamic input support
* bug fixes for dynamic input shape, add mul support, working on conv and batchnorm
* Add batchnormalization, add overflow check for int64 attributes
* add global average/max pool and reshape
* minor changes
* minor changes
* add skip relu and options to use different type of memory
* small bug fix for in operator relu
* bug fix for nnapi
* add transpose support, minor bug fix
* Add transpose support
* minor bug fixes, depthwise conv weight fix
* fixed the bug where the onnx model input has mismatch order than the nnapi model input
* add helper to add scalar operand
* add separated opbuilder to handle single operator
* add cast operator
* fixed reshape, moved some logs to verbose
* Add softmax and identity support, change shaper calling signature, and add support for int32 output
* changed the way to execute the NNAPI
* move NNMemory and InputOutputInfo into Model class
* add limited support for input dynamic shape
* add gemm support, fixed crash when allocating big array on stack
* add abs/exp/floor/log/sigmoid/neg/sin/sqrt/tanh support
* better dynamic input shape support;
* add more check for IsOpSupportedImpl, refactored some code
* some code style fix, switch to safeint
* Move opbuilders to a map with single instance, minor bug fixes
* add GetUniqueName for new temp tensors
* change from throw std to ort_throw
* build settings change and 3rd party notice update
* add readme for nnapi_lib, move to ort log, add comments to public functions, clean the code
* add android log sink and more logging changes, add new string for NnApiErrorDescription
* add nnapi execution options/fp16 relax
* fix a dnnlibrary build break
* addressed review comments
* address review comments, changed adding output for subgraph in NnapiExecutionProvider::GetCapability, minor issue fixes
* formatting in build.py
* more formatting fix in build.py, return fail status instead of throw in compute_func
* moved android_log_sink to platform folder, minor coding style changes
* addressed review comments
As a zero-cost wrapper around the C API, the current state of the C++ API is still pretty low-level and requires programmers to use C-style standards to interact with ONNX.
1. Increase job timeout, while we are investigating why the tests take much longer
2. Upgrade the linux docker image to manylinux2010, by request from Tianlei. (We had an offline discussion with Pranav and Tracy)
3. Remove the installation of "devtoolset-7" in the CUDA image. It was added for CUDA 10.0, it is not needed for CUDA 10.1. We have moved to CUDA 10.1.
* Add amd migraphx execution provider to onnx runtime
* rename MiGraphX to MIGraphX
* add migraphx EP to tests
* support multiple program output
* disable more tests
* backup changes related to program multiple outputs
* remove logging code
* remove unnecessary changes in migraphx_execution_provider.cc
* add migraphx EP to tests
* add input requests of the batchnorm operator
* add to support an onnx operator PRelu
* update migrapx dockerfile and removed one unused line
* chagnes related to support dynamic input shape
* fix build error
* code backup
* code backup
* version that has 106 models run correctly
* code backup
* code backup
* remove unnecessary print info
* code backup
* code backup
* code backup
* code backup
* code backup
* code backup
* changes corresponding to migraphx change
* fix merge conflict
* minor code cleanup
* code cleanup
* remove unnecessary code
* remove unnecessary code
* add to support more constant folding analysis
* more constant folding checking for shape input
* add env var to control whether fp16 is enabled. Modify docker file to use ROCM3.3
* fix function name to avoid build error
* add build and execution instruction for migraphx execution provider
* added more build instructions
* fixed a small format error
* a minor change
* fix review comments
* another minor change
* additional refinement of the documents
* additional changes
* remove unnecessary changes in the dockfile
* additional changes for the dockerfile
* code change backup
* fix errors related to a few unit tests
* fix a build error related to api change
* fix unit test errors by either disabling the test or fix related isssues
* remove unnecessary log info
* sync submodule tvm with master
* remove unnecessary changes
* remove an unnecessary code line
* refine documents for addition example
- Move thread hint vectors from thread-local struct
- Add static_assert that the per-thread state in the thread pool is trivially-destructible
- Rename "thread_data" to "worker_data" (only allocated for workers in the pool, not threads calling into the pool)
* fix python ep default ordering. cpu provider should be last.
* add comment.
* add test case to ensure no regressions for get_all_providers().
* expand on get_all_providers() api documentation
* Rename partition_optimizer -> deepspeed_zero
* Use ZeROConfig in orttraining_pybind_state.cc
* deepspeed_zero -> deepspeed_zero_stage for clarity
* Expose as deepspeed_zero_stage in pybind
* Avoid signed/unsigned warning on loops
* Report sizes when distributed world configuration is inconsistent
* Add DistributedRunContextTest for pipeline stage configuration
