saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-14 18:12:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Document on the usage of mimalloc allocator (#9974)

Browse source 
[Enable proper mimalloc override](https://github.com/microsoft/onnxruntime/pull/9944) PR has fixed a number of issues with regards to building onnxruntime with mimalloc. This PR documents usage of mimalloc.
This commit is contained in:
Dmitri Smirnov 2021-12-08 12:25:04 -08:00 committed by GitHub
parent 70bd575cdd
commit 50127feab2
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
docs/performance/tune-performance.md
View file

@ -153,6 +153,17 @@ Below are some suggestions for things to try for various EPs for tuning performa
### Shared arena based allocator
Memory consumption can be reduced between multiple sessions by configuring the shared arena based allocation. See the `Share allocator(s) between sessions` section in the [C API documentation](../get-started/with-c.md).
### MiMalloc allocator usage
Onnxruntime supports overriding memory allocations using Mimalloc allocator.
MiMalloc allocator is a general-purpose fast allocator. See [mimalloc github](https://github.com/microsoft/mimalloc).
Depending on your model and usage it can deliver single- or double-digits improvements. The GitHub README page describes various scenarios on how mimalloc can be leveraged to support your scenarios.
Mimalloc is a submodule in the Onnxruntime source tree. On Windows one can employ `--use_mimalloc` build flag which would build a static version of mimalloc and link it to Onnxruntime. This would redirect Onnxruntime allocators and all new/delete calls to mimalloc.
Currently, there are no special provisions to employ Mimalloc on Linux. This can be done via LD_PRELAOD mechanism using pre-built binaries that you can build/obtain separately.
### Thread management
* If ORT is built with OpenMP, use the OpenMP env variable to control the number of intra op num threads.