mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-18 21:21:17 +00:00
1e5bda88f0
### Description This change enables caching `MLTensor`s between inferences runs. This is done by keeping a reference to `MLTensor`s alive after they have been released. `MLTensor`s are only destroyed once the sessions goes out of scope. ### Motivation and Context Creating and destroying `MTensor`s on every run has a non-trivial performance penalty. This performance penalty materializes when using `ort.Tensors`[location=cpu] for inputs/outputs or when using the CPU EP as a fallback EP for unsupported operators. The former could be mitigated by developer using `ort.Tensors`[location=ml-tensor]. The latter cannot be mitigated by developers. |
||
|---|---|---|
| .. | ||
| jsep | ||
| proxy-worker | ||
| proxy-messages.ts | ||
| proxy-wrapper.ts | ||
| run-options.ts | ||
| session-handler-inference.ts | ||
| session-options.ts | ||
| wasm-common.ts | ||
| wasm-core-impl.ts | ||
| wasm-factory.ts | ||
| wasm-types.ts | ||
| wasm-utils-env.ts | ||
| wasm-utils-import.ts | ||
| wasm-utils-load-file.ts | ||
| wasm-utils.ts | ||