onnxruntime/js

ONNX Runtime JavaScript API

This directory contains multiple NPM projects:

• onnxruntime-common
• onnxruntime-node
• onnxruntime-web
• onnxruntime-react-native

Development

This folder contains a .vscode folder for Visual Studio Code workspace configs. Using VSCode to open this folder will allow code-formatting and linting features on typescript and C/C++ source code inside this folder. Following files are used for code-formatting and linting features for developers:

• .vscode/**
• package.json
• packages-lock.json
• .eslintrc.js
• .clang-format

Using VSCode:

1. in <ORT_ROOT>/js, run:

   npm ci

2. use VSCode to open folder <ORT_ROOT>/js
3. install VSCode extension if not installed yet:
   • Clang-Format
   • ESLint

To populate typescript type declarations, in each projects, run npm ci.

Run code formatter and linter manually

in <ORT_ROOT>/js, use npm run lint to run ESLint , and use npm run format to run clang-format.

onnxruntime-common

language: typescript

dependency:

folder: <ORT_ROOT>/js/common

This project is designed to include all "common" code, which are pure javascript that can run in both Node.js and browsers.

Requirements

Node.js v12+ (recommended v14+)

Build

Use following command in folder <ORT_ROOT>/js/common to install NPM packages, build typescript files and generate bundles:

npm ci

Distribution

It should be able to consumed by both from projects that uses NPM packages (through a Node.js folder structure of node_modules folder that generated by npm install onnxruntime-common) and from a CDN service that serves a .min.js bundle file.

Features

Following features are included in onnxruntime-common:

• InferenceSession interfaces
• Tensor/OnnxValue interfaces, implementation and a set of utility functions
• Backend interfaces and a set of functions for backend registration

onnxruntime-node

language: typescript/C++

dependency: onnxruntime-common, ONNXRuntime.dll

folder: <ORT_ROOT>/js/node

This project is designed to be used as a NPM package to enable Node.js users to consume ONNX Runtime via Node.js binding, in Node.js or any Node.js compatible environment.

Requirements

Node.js v12+ (recommended v14+)

Build

Build ONNX Runtime and Node.js binding

Follow instructions for building ONNX Runtime Node.js binding

Build Node.js binding only

Use following command in folder <ORT_ROOT>/js/node to install NPM packages and build typescript files:

npm ci

This will download the latest pre-built ONNX Runtime binaries for the current platform.

Distribution

It should be able to consumed by from projects that uses NPM packages (through a Node.js folder structure of node_modules folder that generated by npm install onnxruntime-node).

onnxruntime-web

language: typescript

dependency: onnxruntime-common, ONNXRuntime WebAssembly

folder: <ORT_ROOT>/js/web

This project is a library for running ONNX models on browsers. It is the successor of ONNX.js.

Requirements

Node.js v12+ (recommended v14+)

Build

1. Install NPM packages

   1. in <ORT_ROOT>/js/, run npm ci.
   2. in <ORT_ROOT>/js/common/, run npm ci.
   3. in <ORT_ROOT>/js/web/, run npm ci.

2. Follow instructions for building ONNX Runtime WebAssembly. (TODO: document is not ready. we are working on it.)

   in <ORT_ROOT>/, run either of the following commands to build WebAssembly:

   # In windows, use 'build' to replace './build.sh'
   
   # The following command build debug.
   ./build.sh --build_wasm
   
   # The following command build release.
   ./build.sh --config Release --build_wasm --skip_tests --disable_wasm_exception_catching --disable_rtti
   

   To build with multi-thread support, append flag --enable_wasm_threads to the command. Make sure to build both single-thread and multi-thread before next step.

3. Copy following files from build output folder to <ORT_ROOT>/js/web/dist/:

   • ort-wasm.wasm
   • ort-wasm-threaded.wasm (build with flag '--enable_wasm_threads')

4. Copy following files from build output folder to <ORT_ROOT>/js/web/lib/wasm/binding/:

   • ort-wasm.js
   • ort-wasm-threaded.js (build with flag '--enable_wasm_threads')
   • ort-wasm-threaded.worker.js (build with flag '--enable_wasm_threads')

5. Use following command in folder <ORT_ROOT>/js/web to build:

   npm run build
   

Distribution

It should be able to consumed by both from projects that uses NPM packages (through a Node.js folder structure of node_modules folder that generated by npm install onnxruntime-web) and from a CDN service that serves a .min.js file and one or multiple .wasm file(s).

onnxruntime-react-native

language: typescript, java, objective-c

dependency: onnxruntime-common

folder: <ORT_ROOT>/js/react_native

This project provides an ONNX Runtime React Native JavaScript library to run ONNX models on React Native Android and iOS app.

Requirements

• Yarn
• Android SDK and NDK, which can be installed via Android Studio or sdkmanager command line tool
• A Mac computer with the latest macOS
• Xcode
• CMake
• Python 3

Build

1. Install NPM packages for ONNX Runtime common JavaScript library and required React Native JavaScript libraries

   1. in <ORT_ROOT>/js/, run npm ci.
   2. in <ORT_ROOT>/js/common/, run npm ci.
   3. in <ORT_ROOT>/js/react_native/, run yarn.

2. Build Android ONNX Runtime package

   1. Set up an Android build environment referring to instruction

   2. In <ORT_ROOT>, run this python script to build ONNX Runtime Android archive file. In windows, this requires admin account to build. If an app uses a fixed set of models, refer to instruction and build a mobile version package

   python tools/ci_build/github/android/build_aar_package.py js/react_native/scripts/aar_build_settings.json --config MinSizeRel --android_sdk_path <ANDROID_SDK_PATH> --android_ndk_path <ANDROID_NDK_PATH> --build_dir <BUILD_DIRECTORY>
   

   3. This generates onnxruntime-mobile-<version>.aar in <BUILD_DIRECTORY>/aar_out/MinSizeRel/com/microsoft/onnxruntime/onnxruntime-mobile/<version>. Copy aar file into <ORT_ROOT>/js/react_native/android/libs and rename it as onnxruntime.aar

   4. To verify, open Android Emulator and run this command from <ORT_ROOT>/js/react_native/android

   adb shell am instrument -w ai.onnxruntime.react_native.test/androidx.test.runner.AndroidJUnitRunner
   

3. Build iOS ONNX Runtime package

   1. Set up iOS build environment referring to instruction.

   2. Build ONNX Runtime library for iOS from <ORT_ROOT> using this command,

   ./build.sh --config MinSizeRel --use_xcode --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target 11
   

   Copy <ORT_ROOT>/build/iOS/MinSizeRel/MinSizeRel-iphoneos/libonnxruntime.<version>.dylib file into <ORT_ROOT>/js/react_native/ios/Libraries/onnxruntime/lib/iphoneos

   3. Clean up the previous build and build ONNX Runtime library for iOS Simulator from <ORT_ROOT>

   ./build.sh --config MinSizeRel --use_xcode --ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target 11
   

   Copy <ORT_ROOT>/build/iOS/MinSizeRel/MinSizeRel-iphonesimulator/libonnxruntime.<version>.dylib file into <ORT_ROOT>/js/react_native/ios/Libraries/onnxruntime/lib/iphonesimulator

   4. Edit onnxruntime-react-native.iphoneos.podspec and onnxruntime-react-native.iphonesimulator.podsepc in <ORT_ROOT>/js/react_native to change a version of ONNX Runtime library.

   5. Copy ONNX Runtime header files

   cp <ORT_ROOT>/include/onnxruntime/core/session/*.h <ORT_ROOT>/js/react_native/ios/Libraries/onnxruntime/include
   

   6. To verify, open iOS Simulator and run this command from <ORT_ROOT>/js/react_native/ios. Change a destination to specify a running iOS Simulator.

      pod install
      export ONNXRUNTIME_VERSION=<version>; xcodebuild test -workspace OnnxruntimeModule.xcworkspace -scheme OnnxruntimeModuleTest -destination 'platform=iOS Simulator,name=iPhone 11,OS=14.5'
      

4. Update a version in package.json to align with ONNX Runtime version.

5. Test an example for Android and iOS. In Windows, open Android Emulator first. From <ORT_ROOT>/js/react_native

   yarn bootstrap
   yarn example ios
   yarn example android
   

NPM Packaging

1. Update a version using npm verison <version> from <ORT_ROOT>/js/react_native folder. If it's for a dev, use npm version <version>-dev.<subversion>

2. Run yarn prepack to change onnxruntime-common to point to a published npm package

3. Run npm pack and verify NPM package contents

4. Run npm publish <tgz> --dry-run to see how it's going to be published

5. Run npm publish <tgz> to publish to npmjs

Distribution

It should be able to consumed by React Native projects that uses Yarn packages through yarn add onnxruntime-react-native.