* support ort device tensor in ort module inference
* fallback aten equal to cpu; add ortmodule inference test case
* fix python format
Co-authored-by: Cheng Tang <chenta@microsoft.com@orttrainingdev9.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
* draft kernel creation
* setup eager context
* call into kernel in eager mode
* redefine test case
* refact eager context
* add comment
* remove header
* rename argument
* redefine API definition with types
* list outputs as argument
* switch to int to represent length
* fix compile err
* create attribute API
* add test case for topk
* remove bool from c api
* add gru test case
* remove var
* fix compile warnings
* rename status
* fix compile err
* exclude sparse tensor
* fix comments
* fix comments
* fix build err
* rename file and move location
* format code
* move file to session folder
* fix comments
Co-authored-by: Randy <Randy@randysmac.attlocal.net>
* Move some of the tranpose kernel code to onnxruntime_framework.lib
* Fix C4244 warnings in the tranpose code
* Rename IsMovingSingleAxis to IsTransposeMovingSingleAxis
This reverts commit 4983d6e5d6. We can't destroy OrtEnv through python's atexit function, because at that time there might be many other ORT python objects alive.
* move all logic for ubuntu dockerfiles
* pass in trt version
* update trt 8.0 file
* downgrade protobuf
* uncomment
* and
* change to 8.0
* update dockerfiles
* checkout protobuf based on version
* adding last dockerfile:
:
* checkout 3.10 protobuf
* fix checkout version
* update to 8.2
* keep only one submodule sync
* cleanup
* Delete Dockerfile.custom-trt-perf
* create checkout submodules script
* properly compare decimals in bin/sh
* combine build ort paths
* deprecate TRT 7.2
* only checkout protobuf if we checkout older onnx-tensorrt
* only pull nvidia container if true, update image
* downgrade protobuf only if we checkout onnx-trt
* Update linux-gpu-tensorrt-daily-perf-pipeline.yml for Azure Pipelines
* Update linux-gpu-tensorrt-daily-perf-pipeline.yml for Azure Pipelines
* Add quotes to avoid path splitting
* address shellcheck
* use shellcheck suggestions
* Create new pipeline to sign ov ep binaries
* make codesign available
* make codesign available
* Update sign_ov_ep_binaries.yml for Azure Pipelines
* Update sign_ov_ep_binaries.yml for Azure Pipelines
* add codesign task
* Update sign_ov_ep_binaries.yml for Azure Pipelines
* Update sign_ov_ep_binaries.yml for Azure Pipelines
* windows
* reduce timeout to 15 minutes
* initial fix
* refactor the function handle
* update the implementation
* fix linux build break
* fix training build
* fix minmal build
* fix gradient checker
* deprecate the local function members in graph. host it in model
* fix changming's comments
* fix comments about inlined containers
* fix a missed inlined container
* fix training build
* avoid const for std string_view
Co-authored-by: Cheng Tang <chenta@microsoft.com>
Fix the order of onnx and onnxruntime imports. Importing onnx before onnxruntime causes a dependency issue in the tensorrt containers that prevents onnxruntime_pybind11_state.so from finding the system libstdc++. This is a workaround to get the EP Perf pipeline working until we can investigate the issue more closely.
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
* add aten export for max, max.dim
* rewrite grad of max (no dim); add cases for min
* update UT cases
* mod sym shape infer
* resolve comments: shape infer, add comments, etc.
* add test for torch.max of two tensors
* resolve peng's comments: keepdim; test case
* correct python format
* fix recently introduced lint error
* Specify list/map capacity when initializing where possible
- This really depends on the use case, but in some cases the array/map resizing can be slightly costly, there is effectively no downside setting the initial capacity for a collection if we know for sure its final size
* Supply list/map capacity when initializing where possible
- This really depends on the use case, but in some cases the array/map resizing can be slightly costly, there is effectively no downside setting the initial capacity for a collection if we know for sure its final size
- Introduce an extra utility to help creating maps with expected capacity
* Move utility function to OrtUtil and drop MapUtil, also add Java doc to method
* Move test to the right class
Description: Set black's target version to be py37 - py310
Motivation and Context
Black by default targets its format for py3.10. Since our project supports python 3.7, we need to target version to all the python versions supported.
Re-ran black. 13 files reformatted.
* Add stub implementation of the NNAPI interface so that model builder code can be unit tested on all platforms.
Needed to fix a lot of type mismatch warnings. As these don't occur on Android builds used static_cast for simplicity.
* wip 1
* wip
* wip
* wip
* wip supported
* wip
* wip
* fix bias shape
* wip
* wip
* add gemm to binary op support checker
* op builder change
* add gemm bias handling
* update
* modify and add additional checks
* support matmul
* modify qdq matmul test case to make inputs an initializer
* support matmul test case pass
* modify several helper methods
* minor variable modify
* TestQDQGemm UT pass
* fix UT test pass
* uncomment minor fix
* fix build
* remove unnecessary line
* update to QDQMatMul tests
* add check to addinitializerstoskip
* address comments
* minor update
* update bool -> const bool
* update
* add tests and other minor update
Co-authored-by: rachguo <rachguo@rachguos-Mini.attlocal.net>
* Code refactor
* fix bug
* modify comment
* modify test for the new ORT TRT cache behavior
* update comment
* rename variable
* fix bug for not having trt context
* Custom parameters (#10964)
* get inputs independently for trtexec
* track one process only
* remove engine and profile files
* change time to commit time
* add runtime option for io binding
* move to commit date
* fixes
* add option for graph optimization
* cleanup docker script
* note second time creation
* allow for parameters to be configured from pipeline at runtime
* uncomment
* include optional arguments at runtime
* post second session creation
* update cmake version
* Revert "update cmake version"
This reverts commit 09a1364eae68610724c8e90eeea777b7ee03f74b.
* Move data format import
* Perf FasterRCNN + MaskRCNN (#11102)
* add faster mask
* fix paths
* add a test scenario that - if engine cache is present, trt ep should load the engine cache and run inference
* Revert "Merge branch 'trt_cache_refactor' of https://github.com/microsoft/onnxruntime into trt_cache_refactor"
This reverts commit 8edc574de1ea6055534f33a57b9365c721c2eb29, reversing
changes made to 0c92e5b2b1d453527001fe731ed4ccfc79e6adad.
Co-authored-by: Olivia Jain <oljain@microsoft.com>
Description: Format all python files under onnxruntime with black and isort.
After checking in, we can use .git-blame-ignore-revs to ignore the formatting PR in git blame.
#11315, #11316
Update the code to use OrtApis instead of the old onnxruntime::InferenceSession class. Mainly because the old one doesn't support custom op. We are trying to convert some EPs to custom ops. Hopefully they can continue to leverage this test set.
* increase timeout
* show mac agent info
* Revert "show mac agent info"
This reverts commit a646ebefff8940a3044f1984107856db33319eb8.
* increase timeout in PR test
Prior to this, certain shape and type errors were surfaced only when
the model was using the latest known op set version.
Providing users an explicit option allows for better testing of code
that produces models, which includes unit tests within this repo and
other repos such as the TF-ONNX and PT-ONNX converters.
Remove the previous behavior which seems quite counter-intuitive:
an otherwise identical model with a later op set version should be treated
identically in this regard.
The option defaults to false to avoid causing errors for users that
rely on the previous permissive behavior.
Turned on the strict enforcement by default in OpTester, which revealed a few
disagreements between ORT and ONNX on what the correct output shape should
be.
Fix shape inference bug in ReduceSumTraining with noop_with_empty_axes=1
which was revealed.
Fix TensorOpTest.Unsqueeze_scalar, which was testing negative axes on an
op set version where the op did not actually support negative axes.
Fixes#9506.
