* updating examples with current api calls
* Fixing capitalization in api calls, adding RKNPU update
* Correcting nuphar and rknpu ep api calls
* Include creating session in readme
* Model test start with float
* Clean up code and add environment variable detection
* Move into namespace
* PR comments
* Fix linker errors in latest merge to master and also fix warning
* add skipping model test mechanism
* Return std::string instead of writing to buffer
* Address case where env variable is larger than max_path
* use const static string for test reason
* Disable x86 tests and don't build if ort memory checker is enabled
* Add comment
* Add additional failing x86 tests and ifdef for checking fo rx86 build
* PR comments
* Add type inference for BroadcastGradientArgs
This change enables the ONNX shape and type inference to work on a function body containing a BroadcastGradientArgs op. Without this change, the dummy inference function is used, and no types are inferred for the output here:
531e6dd459/onnx/shape_inference/implementation.cc (L467-L469)
* Handle optional outputs.
* Created shared version of InferenceSession wrapper class and update relevant tests to use it.
Include domain in the ops counting helper so it's more general and we don't need to duplicate it in the nchwc tests. Update tests to include domain in key being checked.
* Fix some training tests
* Fix prefixing of contrib op names in test
* Add session option and global thread pool option to set denormal as zero.
* Revert unneccessary changes.
* Add cpuinfo submodule
* Add more comments
* Remove cpuinfo submodule dependency and check only SSE3 support for ftz and daz inspired by Tensorflow
* Preserve API order in C api
* Clean up and utilize SSE3 detection logic from existeing cpuid_info.h
* Keep the same order with header file
* Fix build issue with Linux pipeline, which has old g++ compiler
* Fix broken build on Linux and remove a duplicated unit test
* Remove reformatting at eigen thread pool
* Remove flatbuffers which is not intentionally added
* Revert "Remove flatbuffers which is not intentionally added"
This reverts commit 9f509a9aaaa3c7832d88854c82fd26b234770b7f.
* Remove flatbuffers which is not intentionally added
* Resolve comments
- Put details on APIs
- Add a log for ftz/daz initialization
- Add clang
- Fix typo
* Remove unnecessary header include
* Resolve comments
* Cmake changes for 2021.1
* added new ov version 2020.1 for faster rcnn
* Added missing defs
* equal op modified
* changes to incoroporate faster rcnn
* backend util.cc
* hddl_plugin_config.hpp is depreceated . instead use hddl_config.hpp
* changing myriad precision bool to i32
* gather is not enabled for gpu
* conv2D and pooltest auto_pad attribute should not be null
* negative indices are not valid for scatter op in myriad
* non max suppression op only supported in faster rcnn mode
* maxpool indices output is not supported
* Cleaned redundant code in backends
* Added ifdefs for HDDL config
* cast output dimensions check
topk operator k input it seems only resolved for myriad as it is
throwing issues for ask rcnn . need to verify
* we are limiting the subgraph size to 3 here
* taking care of review comments
* Fixed minor bugs
* Modified Slice op checks
* Added NonZero, Upsample
* Removed TopK if it's in the middle of a subgraph
* incorporated upsample conditions too
* Dockerfile changes for 2021.1 release
* dockerfile aptkey update
* Minor fixes
* ceil condition added again
* Fixed few gpu models
* Disabled LSTM and yolov3 in ModelTests
* python softmax cross entropy tests and negative log likelihood
* Update Build.md
Updated for openvino 2021.1
* Update OpenVINO-ExecutionProvider.md
update openvino execution provider for 2021.1
* Update READMe.md
updated new openvino version
* Update Dockerfile.openvino
added environment variable for DEBIAN Frontend
* Fixed myriad models
* Fixed gather condition
* Fixed mask rcnn model on myriad
* Modified Gather condition
* set default target of MCR dockerfile to MYRIAD_FP16
* Fixed tinyolov3 on CPU
* Update OpenVINO-ExecutionProvider.md
update openvino execution provider documentation
* Update Dockerfile.openvino
Removed environment variable
* Update OpenVINO-ExecutionProvider.md
update image manipulation networks supported
* Update onnx_backend_test_series_filters.jsonc
removed test_upsample_nearest from cpu test cases
* New InternalCI changes for 2021.1
* Full protobuf removed for OpenVINO
* Protobuf added
* Updated with apt installation for openvino
* Revert the testing changes
* Reverted testing changes
* File permessions are changed to original
* Deleted openvino installation and cmake change
* Optimized Dockerfile
Removed unnecessary cmake installation, numpy
* Added missing ifdefs
* delete array fix
* backend_utils.cc output_shape
* Revert "set default target of MCR dockerfile to MYRIAD_FP16"
This reverts commit 928d3e2b71e2f589cf51dacd3a133951cf9ca18d.
Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
Co-authored-by: sfatimar <sahar.fatima@intel/com>
Co-authored-by: suryasidd <48925384+suryasidd@users.noreply.github.com>
Co-authored-by: S. Manohar Karlapalem <manohar.karlapalem@intel.com>
Co-authored-by: Aravind <aravindx.gunda@intel.com>
Co-authored-by: Aravind Gunda <38353114+gundaarx@users.noreply.github.com>
* add Python API for getProfilingStartTime
* debug for using Python API
* add in C# api
* use uint intead of uint64_t to prevent warning
* typo for GetProfilingStartTimeNs
* remove const
* Update onnxruntime/python/session.py
Co-authored-by: Pranav Sharma <emailpranav@gmail.com>
* remove unnecessary return
* Add Python unit test
* Add C# unit test and refactor Python test
* use ulong in C# for uint64_t in C++
* remove time.monotonic_ns
* syntax: remove public for inner function
* correct the API's order
* getprofilingstarttime after run
* Correct the right order in NativeMethod.cs
* update order
* nit: remove spaces
* Update csharp/src/Microsoft.ML.OnnxRuntime/InferenceSession.cs
Co-authored-by: Guoyu Wang <62914304+gwang-msft@users.noreply.github.com>
* use the updated function
* add comment about the precision
* add more comments
* add session.py back
* fix flake8
* remove session.py
* Add comments in C, C#, Python APIs about precision
Co-authored-by: Pranav Sharma <emailpranav@gmail.com>
Co-authored-by: Guoyu Wang <62914304+gwang-msft@users.noreply.github.com>
* Add CUDA option to run copy in default stream
This change fixes#4829. Thanks @maherzog for providing the repro!
The bug is caused by memory reuse in BFC arena, where copy and
compute stream in CUDA has a racing condition.
BFC arena is an arena allocator on top of cudaMalloc/Free to
reduce the cost in syncing CPU and GPU when alloc/free. It means
when CPU alloc/free the memory, GPU might not finished previous
work on the memory, so that CPU and GPU could run asynchronously.
This is OK if there's only one stream, where the execution order
in CPU and GPU are consistent. For example, if we have two kernels
A and B, CPU runs allocA->computeA->freeA->allocB->computeB->freeB,
A and B could shares the same memory since computeA and computeB
will not have racing as long as they run in the same GPU compute
stream.
However, if CPU runs allocA->CopyA->freeA->allocB->computeB->freeB,
the order of execution in GPU could have copyA happen after computeB,
if copy and compute happens in different GPU streams.
This change makes copy to run in default compute stream, while adding
an option to fall back to previous behavior if there's perf hit. This
is a short term fix before BFC arena could support multiple streams.
User may use following options to revert to previous behavior:
C API:
struct OrtCUDAProviderOptions cudaProviderOpt;
cudaProviderOpt.do_copy_in_default_stream = false;
C++ API:
CUDAExecutionProviderInfo cudaEPInfo;
cudaEPInfo.do_copy_in_default_stream = false;
C# API:
pending...
Python:
import onnxruntime
onnxruntime.capi._pybind_state.set_do_copy_in_default_stream(False)
* Confirmed the test failes in CI when doing copy in separate stream
Revert the test to get CI pass now
* Fix Windows test
* Address CR
* Update MaxBatchSize and include recompute mode
* Minor fix for frontend test
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
**Description**: Add missing gradient registration for the `Exp` op.
**Motivation and Context**
* Adding support for training a model that uses the `Exp` op.
Co-authored-by: Derek Murray <demurra@microsoft.com>
* t5 layer norm changes
* add t5 layer norm kernel
* use template for t5 layer norm
* template definition changes
* no build error
* add CPU cuda kernel
* first unit test
* other forward unit tests
* add T5LayerNormGrad
* Add c++ transform and test for T5 LN
* fix and some debug prints
* fix cuda error
* rename from t5 to simplified
* PR comments
* revert change on invertible LM code path
* remove duplicate forward computation
* add GradientCheckerTest.SimplifiedLayerNormGrad
* change back macro
* Fix SimplifiedLayerNorm Gradient
* merge with Sherlockss changes
* changed cuda kernel
* reapply cpu kernel changes
Co-authored-by: Jingyan Wang <jingywa@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: aishwarya bhandare <aibhanda@microsoft.com>
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Avoid inserting other CUDA calls in-between NCCL Send's and Recv's
* Add a comment
* Place CUDA EP on the right device
* Fix a warning
* Address a comment
* use run_orttraining_test_orttrainer_frontend_separately to work around a sporadic segfault.
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* - Link with libatomic if needed
- Install pip differently so it doesn't clash with the system pip which may involve a wrapper script
- Remove ability to specify offset when Tensor allocates the data. The data prior to offset isn't accessible by anything.
- Fix use of offset in TensorOpTest to work on armv7 where it must be aligned to the type it points to.
- Fix ActivationOpNoInfTest.Softsign to allow for armv7 behavior
- Fix ReductionOpTest.ReduceMean_*keepdims to allow for armv7 floating point inaccuracy
* Address PR comments
