* Mention OrtCreateSessionFromArray in C API doc
* fix seq of tensors
* changes on 9/30
* All tests passing
* Add SequenceAt op
* Fix shared_lib non_tensor_types test
* Address some PR comments
* Address PR comments
* Add support in python bindings to accept seq(tensor)
* Change data type from vector<Tensor> to TensorSeq
* Change data type from vector<Tensor> to TensorSeq
* Added some documentation
* Added missing test model
* Fix Linux build
* Fix Mac build
* Fix Mac build
* Add Unique operator.
* Enable onnx tests. Disable one with incorrect expected output and add unit test to validate ORT behavior. Need onnx update to fix (will address that separately but don't want to block this checkin on that change).
Remove gsl subodule and replace with a local copy of gsl-lite
Refactor for onnxruntime::make_unique
gsl::span size and index are now size_t
Remove lambda auto argument type detection.
Remove constexpr from fail_fast in gsl due to Linux not being happy.
Comment out std::stream support due to MacOS std lib broken.
Move make_unique into include/core/common so it is accessible for server builds.
Relax requirements for onnxruntime/test/providers/cpu/ml/write_scores_test.cc
due to x86 build.
Add ONNXRUNTIME_ROOT to Server Lib includes so gsl is recognized
* Don't return shape for non-const initializer in InferenceContextImpl::getInputType
Don't return initializer for non-const initializer in InferenceContextImpl::getInputData
Update graph_utils to support these scenarios
- fix GetConstantInitializer to make sure a name is for an outer scope value before checking a parent graph, as local name could shadow an outer scope initializer.
* Mention OrtCreateSessionFromArray in C API doc
* Add C API for free dim override
* Add C API for free dim override, fix missing API mention in InferenceTest.cs, fix confusing print statement in perf_test.
* Remaining C#files
* fix c# build
* Run the tests in blame mode. This option is helpful in isolating a problematic test causing the test host to crash.
* fix order
Description: Refine threading control options and move inter op thread pool to session state.
Added thread_utils.h/cc to centralize the decision around the thread pool size under various conditions.
Motivation and Context
Currently the thread pool size of the parallel executor is hardcoded to 32 for some reason. This PR makes the options to configure the thread pool sizes clearer.
* Fix symbolic shape inference for faster_rcnn, mask_rcnn, yolov3
Force merge when --auto_merge, on symbolic dims which sympy cannot simplify
Add symbolic inference for Resize opset 10
Add support for step != 1 in Slice
Add support for computed dim in TopK
Bug fixes in passing symbolic dims from subgraph
Fix an outdate comment in Nuphar provider header
* Bump onnx to latest
Update onnx.in.proto with changes for SparseTensor.
* add temp skip tests
* remove passed tests from skip list
* skip more tests for new ops in opset 11
* skip crashing tests
* update handling of new attribute types sparse tensor and sparse tensors
* advance onnx commit and remove skip cpu_flaky_tests
* temporarily skip yolo3 model test due to resize opset10 shape inference regression
* update proto for onnxruntime server
* advance onnx commit further
C/C++ Opage APIs
Add new virtual interfaces for NonTensorType
Implement entry points.
Add shared header for the data container.
Add export symbols.
Add serialization/deserialization.
Implement model with Opaque types.
Rework opqaue_api_test as a standalone executable.
* Mention OrtCreateSessionFromArray in C API doc
* Add GetDataTransfer() interface in the EP.
* Check return status of RegisterDataTransfer
* Address PR comments
* Rework the feed/fetch copy setup so that it can be calculated upfront by the control flow nodes. Also simplifies how it all works.
Update the control flow nodes to do the calculation prior to graph execution.
* Implement Nuphar execution provider
Nuphar execution provider is a TVM-based compilation provider. It has shown great speedups for RNN models using Scan.
This PR is mainly for a preview of the shared codegen library for other TVM-based providers.
* Fix submodules
* Fix TVM submodule
* Update Nuphar to latest and resolve confliction
* Remove stale files caused by merge -X theirs
* Revert heap buffer change to not introduce onnxruntime_framework into onnxruntime_perf_test
* Fix bad merge
* Merge from Nuphar
* Fix warning treated as error, revert some unnecessary changes
* Revert some more test changes
* Some more test revert or comments to make review easier
New tests could be added later
* One more revert of unnecessary changes
* More change revert. Test could be added back later.
* Mention OrtCreateSessionFromArray in C API doc
* Don't create the default allocator every single time. Rename API accordingly.
* Don't create the default allocator every single time. Rename API accordingly.
* updates...
* updates...
* PR comments
* fix typo in license header
* fix build
1.Let mlas use session thread pool
2.Remove onnxruntime_USE_MLAS cmake option
3. Remove the win32 thread pool code inside mlas
mlas will:
1.use ort thread pool if it get passed in
2.use openmp if the threadpool parameter is nullptr
3.run single threaded if the threadpool parameter is nullptr and openmp is disabled.
Added Sample Featurizer and Infrastructure
Make featurizers and unit tests compile and run with GTest.
Create definitions for the first featurizer kernel.
Add new operator domain.
Create datetime_transformer kernel and build.
Move OPAQUE types definitions for featurizers kerneles out to a separate cc.
Register them with the type system.
Provide unit tests for new AutoML DateTimeTransformer kernel.
Make necessary adjustments to the test infrastructure to make it run
with new types.
- Added python script for generating markdown doc from the registered opkernels.
- Made some conditional changes in the pybind to expose necessary python API
- Added some missing type-constraints in the op kernel registrations
* Mention OrtCreateSessionFromArray in C API doc
* review changes
* use enum for graph optimization level
* Use explicit values for enums
* updates...
* Add friendly enum for graph optimization levels in C, C# and Python APIs.
* Fix linux build
* Fix build breakage due to master merge
* PR comments
* Mention OrtCreateSessionFromArray in C API doc
* Fix perf test executable due to removal of certain C APIs
* fix linux build
* Avoid duplication
* Fix mem leak
* remove memory copy between CUDA and TRT
* add info to RegisterExecutionProvider input
* use new IDeviceAllocator for trt allocator
* remove SetDefaultInputsMemoryType from TRT EP
* remove onnx-tensorrt 5.0
* add submodule onnx-tensorrt branch 5.1
* remove redundancy
* Update transformer_memcpy.cc
* Update tensorrt_execution_provider.cc
* switch to TensorRT 5.1.5.0
* update python binding
* disable failed test case on TensorRT
* Update activation_op_test.cc
* upgrade to TensorRT container 19.06
* update according to feedback
* add comments
* remove tensorrt allocator and use cuda(gpu) allocator
* update onnx-tensorrt submodule
* change ci build cuda directory name
* A few performance improvements:
- Make the iteration in NonZero more efficient by using a raw pointer and simplifying the increment logic
- add another unit test to check the new logic works with 3 dimensional tensor
- gains about 2% for ssd_mobilenet
- Avoid floating point operations on each iteration on Concat
- about 0.5% for ssd_mobilenet and ssd_resnet34
- Put common case first in ExecutionFrame::AllocateAsPerAllocationPlan to avoid unnecessary call to IsSparseTensor
- about 0.05% for ssd_mobilenet
- Minor tweak to put some ctors in the TensorShape header so they can be inlined more easily
* If there is an outer scope value that matches a subgraph input, don't create an implicit input from the outer scope value.
Minor unrelated change for issue noticed while debugging: Use unordered_set for implicit inputs so we don't add them multiple times.
* Add unit test based on onnx issue.
* Add string attribute interface for C API.
* Add string attribute interface for C++ API accordingly.
* Update comment to say that string is also valid
* Use INFO instead of WARNING for an unused graph input.
* Drop severity of unused initializer as well
* Update to output a warning level message if removing an initializer that is never used, and an info level message if removing an initializer that optimization has made redundant.
* Now that we check for a constant initializer in an ancestor graph we also need to be able to retrieve and replace that initializer.
Add helpers to do so.
Update optimizers to use the new helpers.
Fix bug in UnsqueezeElimination where it wasn't checking if the initializer it was replacing was constant.
This change integrates the NCHWc support recently added to MLAS into ONNX Runtime. When using "-o 3" optimizations, then the runtime will do a NCHWc layout optimization pass to convert standard ONNX operators such as Conv/MaxPool to the com.microsoft.nchwc domain with weights and biases reordered for speed.
