454f77cd94
# Motivation Currently, ORT minimal builds use kernel def hashes to map from nodes to kernels to execute when loading the model. As the kernel def hashes must be known ahead of time, this works for statically registered kernels. This works well for the CPU EP. For this approach to work, the kernel def hashes must also be known at ORT format model conversion time, which means the EP with statically registered kernels must also be enabled then. This is not an issue for the always-available CPU EP. However, we do not want to require that any EP which statically registers kernels is always available too. Consequently, we explore another approach to match nodes to kernels that does not rely on kernel def hashes. An added benefit of this is the possibility of moving away from kernel def hashes completely, which would eliminate the maintenance burden of keeping the hashes stable. # Approach In a full build, ORT uses some information from the ONNX op schema to match a node to a kernel. We want to avoid including the ONNX op schema in a minimal build to reduce binary size. Essentially, we take the necessary information from the ONNX op schema and make it available in a minimal build. We decouple the ONNX op schema from the kernel matching logic. The kernel matching logic instead relies on per-op information which can either be obtained from the ONNX op schema or another source. This per-op information must be available in a minimal build when there are no ONNX op schemas. We put it in the ORT format model. Existing uses of kernel def hashes to look up kernels are replaced with the updated kernel matching logic. We no longer store kernel def hashes in the ORT format model’s session state and runtime optimization representations. We no longer keep the logic to generate and ensure stability of kernel def hashes. |
||
|---|---|---|
| .. | ||
| app | ||
| .gitignore | ||
| build.gradle | ||
| gradle.properties | ||
| README.md | ||
| settings.gradle | ||
Android Test Application for ORT-Mobile
This directory contains a simple android application for testing ONNX Runtime AAR package.
Background
For general usage and build purpose of ORT-Mobile Android, please see the documentation here.
Test Android Application Overview
This android application is mainly aimed for testing:
- Model used: A simple sigmoid ONNX model (converted to ORT format under
app\src\androidTest\assetsfolder).- Here's documentation about how you can convert an ONNX model into ORT format.
- Run
python -m onnxruntime.tools.convert_onnx_models_to_ort --optimization_style=Fixed /path/to/model.onnxand rename the resulting .ort file accordingly.
- Main test file: An android instrumentation test under
app\src\androidtest\java\ai.onnxruntime.example.javavalidator\SimpleTest.kt - The main dependency of this application is
onnxruntimeaar package underapp\libs. - The MainActivity of this application is set to be empty.
Requirements
- JDK version 8 or later is required.
- The Gradle build system is required for building the APKs used to run android instrumentation tests. Version 6 or newer is required.
Building
Use the android's build instructions with --build_java and --android_run_emulator option.
Please note that you may need to set the --android_abi=x86_64 (the default option is arm64-v8a). This is because android instrumentation test is run on an android emulator which requires an abi of x86_64.
Build Output
The build will generate two apks which is required to run the test application in $YOUR_BUILD_DIR/java/androidtest/android/app/build/outputs/apk:
androidtest/debug/app-debug-androidtest.apkdebug/app-debug.apk
After running the build script, the two apks will be installed on ort_android emulator and it will automatically run the test application in an adb shell.