onnxruntime/onnxruntime
Valery Chernov ba8a00f62f
[TVM EP] Support zero copying TVM EP output tensor to ONNX Runtime output tensor (#12593)
**Description**:
Support new feature of TVM Virtual Machine (method `set_outputs`) on TVM
Execution Provider side. It allows to avoid excess copying from TVM EP
output tensor to ONNX Runtime one

**Motivation and Context**
Tests with multiple output topologies and big output tensors shows that
there is overheads spent on copying from TVM EP to ONNX Runtime.
Returning output(s) on preallocated memory for VirtualMachine was
implemented on TVM side.

**Details**
`set_output_zero_copy` provider option for TVM EP switches on/off this
feature. It is true by default.
The feature works for both GraphExecutor and VirtualMachine from TVM.

---------

Co-authored-by: Valery Chernov <valery.chernov@deelvin.com>
2023-02-08 10:02:20 -08:00
..
contrib_ops Introduce collective ops to ort inference build (#14399) 2023-02-07 13:47:48 -08:00
core [TVM EP] Support zero copying TVM EP output tensor to ONNX Runtime output tensor (#12593) 2023-02-08 10:02:20 -08:00
python Adding RunOptions synchronization behaviour to C/C++ API (#14088) 2023-02-07 19:59:28 -08:00
test Introduce collective ops to ort inference build (#14399) 2023-02-07 13:47:48 -08:00
tool/etw
wasm [wasm] fix session option setting of mem_pattern (#13858) 2022-12-07 13:15:44 -08:00
__init__.py Bump ORT version number (#14226) 2023-01-26 12:33:47 -08:00
ReformatSource.ps1
ReformatSourcePython.bat
VSCodeCoverage.runsettings