a0f3e30de6
* initial setup and rename "how to" to "setup" * move API to main nav * move api to main nav * add get starated, rework nav order * rename to install move mds out of install section * update api nav and home page * add install docs and python qs updates * python get started work * remove c and obj c for now * move java, python, and obj-c docs under api folder * move java api html to iframe (ugh) * remove api docs w/o details, move api text getstar * remove api docs wo detail updates get started * remvoe iframes * move eco system to main nav * fix api buttons * added more examples moved intro to ORT * fix links * fix get started titles * fix get started titles * fix more links * fix more links * more link fixes * fix nav remove inferencing and training subnav * fix top nav remove inference and training nav * fix title * fix tutorials nav hierarchy * fix python api button * add tenorflow keras example * fix quickstart toc * add imports fix spacing * fix links * update nav and python get started page * move ort training example, add coming soon for iot * update C# get started * fix spacing on quantization * Add some js get started content * fix formatting * fix typo * removed onnx-pytorch and onnx-tf * updated pip install torch and added links iot page * added pytorch tutorial heirarchy * updated web to docs soon added release blog link * add web link
2.9 KiB
| title | parent | nav_order |
|---|---|---|
| CoreML | Execution Providers | 2 |
{::options toc_levels="2" /}
CoreML Execution Provider
Core ML is a machine learning framework introduced by Apple. It is designed to seamlessly take advantage of powerful hardware technology including CPU, GPU, and Neural Engine, in the most efficient way in order to maximize performance while minimizing memory and power consumption.
Contents
{: .no_toc }
- TOC placeholder {:toc}
Requirements
The CoreML Execution Provider (EP) requires iOS devices with iOS 13 or higher, or Mac computers with macOS 10.15 or higher.
It is recommended to use Apple devices equipped with Apple Neural Engine to achieve optimal performance.
Install
Pre-built binaries of ONNX Runtime Mobile with CoreML EP for iOS are published to CocoaPods.
- C/C++: onnxruntime-mobile-c
- Objective-C: onnxruntime-mobile-objc
Build
For build instructions for iOS devices, please see How to: Build for Android/iOS.
Usage
The ONNX Runtime API details are here.
The CoreML EP can be used via the C or C++ APIs currently. Additional support via the Objective-C API is in progress.
The CoreML EP must be explicitly registered when creating the inference session. For example:
Ort::Env env = Ort::Env{ORT_LOGGING_LEVEL_ERROR, "Default"};
Ort::SessionOptions so;
uint32_t coreml_flags = 0;
Ort::ThrowOnError(OrtSessionOptionsAppendExecutionProvider_CoreML(so, coreml_flags));
Ort::Session session(env, model_path, so);
Configuration Options
There are several run time options available for the CoreML EP.
To use the CoreML EP run time options, create an unsigned integer representing the options, and set each individual option by using the bitwise OR operator.
uint32_t coreml_flags = 0;
coreml_flags |= COREML_FLAG_ONLY_ENABLE_DEVICE_WITH_ANE;
Available Options
COREML_FLAG_USE_CPU_ONLY
Limit CoreML to running on CPU only.
This may decrease the performance but will provide reference output value without precision loss, which is useful for validation.
COREML_FLAG_ENABLE_ON_SUBGRAPH
Enable CoreML EP to run on a subgraph in the body of a control flow operator (i.e. a Loop, Scan or If operator).
COREML_FLAG_ONLY_ENABLE_DEVICE_WITH_ANE
By default the CoreML EP will be enabled for all compatible Apple devices.
Setting this option will only enable CoreML EP for Apple devices with a compatible Apple Neural Engine (ANE). Note, enabling this option does not guarantee the entire model to be executed using ANE only.
For more information, see Which devices have an ANE?