7faa4b1eb4
### Description This PR updates the documentation regarding the ACL execution provider to reflect the changes made in #22032. It also updates various references to Arm's trademarks in accordance with the Arm trademark use guidelines. ### Motivation and Context Ensures documentation is accurate following recent changes. --------- Signed-off-by: Michael Tyler <michael.tyler@arm.com>
2.1 KiB
| title | description | grand_parent | parent | nav_order | redirect_from |
|---|---|---|---|---|---|
| Arm - ACL | Instructions to execute ONNX Runtime with the ACL Execution Provider | Execution Providers | Community-maintained | 2 | /docs/reference/execution-providers/ACL-ExecutionProvider |
ACL Execution Provider
{: .no_toc }
The ACL Execution Provider enables accelerated performance on Arm®-based CPUs through Arm Compute Library{:target="_blank"}.
Build
For build instructions, please see the build page.
Usage
C/C++
{: .no_toc }
Ort::Env env = Ort::Env{ORT_LOGGING_LEVEL_ERROR, "Default"};
Ort::SessionOptions sf;
bool enable_fast_math = true;
Ort::ThrowOnError(OrtSessionOptionsAppendExecutionProvider_ACL(sf, enable_fast_math));
The C API details are here.
Python
{: .no_toc }
import onnxruntime
providers = [("ACLExecutionProvider", {"enable_fast_math": "true"})]
sess = onnxruntime.InferenceSession("model.onnx", providers=providers)
Performance Tuning
Arm Compute Library has a fast math mode that can increase performance with some potential decrease in accuracy for MatMul and Conv operators. It is disabled by default.
When using onnxruntime_perf_test{:target="_blank"}, use the flag -e acl to enable the ACL Execution Provider. You can additionally use -i 'enable_fast_math|true' to enable fast math.
Arm Compute Library uses the ONNX Runtime intra-operator thread pool when running via the execution provider. You can control the size of this thread pool using the -x option.
Supported Operators
| Operator | Supported types |
|---|---|
| AveragePool | float |
| BatchNormalization | float |
| Concat | float |
| Conv | float, float16 |
| FusedConv | float |
| FusedMatMul | float, float16 |
| Gemm | float |
| GlobalAveragePool | float |
| GlobalMaxPool | float |
| MatMul | float, float16 |
| MatMulIntegerToFloat | uint8, int8, uint8+int8 |
| MaxPool | float |
| NhwcConv | float |
| Relu | float |
| QLinearConv | uint8, int8, uint8+int8 |