Open Developer Command Prompt for Visual Studio version you are going to use. This will properly setup the environment including paths to your compiler, linker, utilities and header files.
If you want to build an ARM64 binary on a Windows ARM64 machine, you can use the same command above. Just be sure that your Visual Studio, CMake and Python are all ARM64 version.
If you want to cross-compile an ARM32 or ARM64 or ARM64EC binary on a Windows x86 machine, you need to add "--arm" or "--arm64" or "--arm64ec" to the build command above.
When building on x86 Windows without "--arm" or "--arm64" or "--arm64ec" args, the built binaries will be 64-bit if your python is 64-bit, or 32-bit if your python is 32-bit,
If you would like to use [Xcode](https://developer.apple.com/xcode/) to build the onnxruntime for x86_64 macOS, please add the `--use_xcode` argument in the command line.
Today, Mac computers are either Intel-Based or Apple silicon(aka. ARM) based. By default, ONNX Runtime's build script only generate bits for the CPU ARCH that the build machine has. If you want to do cross-compiling: generate ARM binaries on a Intel-Based Mac computer, or generate x86 binaries on a Mac ARM computer, you can set the "CMAKE_OSX_ARCHITECTURES" cmake variable. For example:
* Please note that these instructions build the debug build, which may have performance tradeoffs. The "--config" parameter has four valid values: Debug, Release, RelWithDebInfo and MinSizeRel. Compared to "Release", "RelWithDebInfo" not only has debug info, it also disables some inlines to make the binary easier to debug. Thus RelWithDebInfo is slower than Release.
* To build the version from each release (which include Windows, Linux, and Mac variants), see these [.yml files](https://github.com/microsoft/onnxruntime/tree/main/tools/ci_build/github/azure-pipelines/) for reference
* If you need to install protobuf from source code, please note:
* First, please open [cmake/deps.txt](https://github.com/microsoft/onnxruntime/blob/main/cmake/deps.txt) to check which protobuf version ONNX Runtime's offical packages use.
* As we statically link to protobuf, on Windows protobuf's CMake flag `protobuf_BUILD_SHARED_LIBS` should be turned OFF, on Linux if the option is OFF you also need to make sure [PIC](https://cmake.org/cmake/help/latest/prop_tgt/POSITION_INDEPENDENT_CODE.html) is enabled. After the installation, you should have the 'protoc' executable in your PATH. It is recommended to run `ldconfig` to make sure protobuf libraries are found.
* If you installed your protobuf in a non standard location it would be helpful to set the following env var:`export CMAKE_ARGS="-DONNX_CUSTOM_PROTOC_EXECUTABLE=full path to protoc"` so the ONNX build can find it. Also run `ldconfig <protobuf lib folder path>` so the linker can find protobuf libraries.
Then, it's better to uninstall protobuf before you start to build ONNX Runtime, especially if you have install a different version of protobuf other than what ONNX Runtime has.
If you want to build a binary for a 32-bit architecture, you might have to do cross-compiling since a 32-bit compiler might not have enough memory to run the build.
Building the code on Android/iOS is not supported. You need to use a Windows, Linux or macOS device to do so.
|**C# and C Nuget packages**|`--build_nuget`|Builds C# bindings and creates nuget package. Implies `--build_shared_lib`<br> Detailed instructions can be found [below](#build-nuget-packages).|
|**WindowsML**|`--use_winml`<br>`--use_dml`<br>`--build_shared_lib`|WindowsML depends on DirectML and the OnnxRuntime shared library|
|**Java**|`--build_java`|Creates an onnxruntime4j.jar in the build directory, implies `--build_shared_lib`<br>Compiling the Java API requires [gradle](https://gradle.org) v6.1+ to be installed in addition to the usual requirements.|
* dotnet is required for building csharp bindings and creating managed nuget package. Follow the instructions [here](https://dotnet.microsoft.com/download) to download dotnet. Tested with versions 2.1 and 3.1.
* nuget.exe. Follow the instructions [here](https://docs.microsoft.com/en-us/nuget/install-nuget-client-tools#nugetexe-cli) to download nuget
* On Windows, downloading nuget is straightforward and simply following the instructions above should work.
* On Linux, nuget relies on Mono runtime and therefore this needs to be setup too. Above link has all the information to setup Mono and nuget. The instructions can directly be found [here](https://www.mono-project.com/docs/getting-started/install/). In some cases it is required to run `sudo apt-get install mono-complete` after installing mono.
##### Build Instructions
###### Windows
```
.\build.bat --build_nuget
```
###### Linux
```
./build.sh --build_nuget
```
Nuget packages are created under <native_build_dir>\nuget-artifacts
See [onnxruntime/core/framework/debug_node_inputs_outputs_utils.h](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/framework/debug_node_inputs_outputs_utils.h) for details.
Running on a platform like RISC-V involves cross-compiling from a host platform (e.g. Linux) to a target platform (a specific RISC-V CPU architecture and operating system). Follow the instructions below to build ONNX Runtime for RISC-V 64-bit.
#### Cross compiling on Linux
1. Download and install GNU RISC-V cross-compile toolchain and emulator.
If you are using Ubuntu, you can obtain a prebuilt RISC-V GNU toolchain by visiting https://github.com/riscv-collab/riscv-gnu-toolchain/releases. Look for the appropriate version, such as `riscv64-glibc-ubuntu-22.04-llvm-nightly-XXXX-nightly.tar.gz`. After downloading, extract the archive to your build machine and add the "bin" folder to your $PATH environment variable. The QEMU emulator is also located in the toolchain folder. Skip this step if you've followed these instructions.
For users with other Linux distributions, the option to compile GCC from its source code is available. It is recommended to choose the a compiler version that corresponds to your target operating system, and opting for the latest stable release is recommended.
When you get the compiler, run `riscv64-unknown-linux-gnu-gcc -v` This should produce an output like below:
2. Configure the RISC-V 64-bit CMake toolchain file
The CMake toolchain file is located at `${ORT_ROOT}/cmake/riscv64.toolchain.cmake`. If there are alternative values for the variable that need to be configured, please include the corresponding settings in that file.
3. Cross compiling on Linux
Execute `./build.sh` with the `--rv64` flag, specify the RISC-V toolchain root using `--riscv_toolchain_root`, and provide the QEMU path using `--riscv_qemu_path` as build options to initiate the build process. Make sure that all installed cross-compilers are accessible from the command prompt used for building by configuring the `PATH` environment variable.
(Optional) If you intend to utilize xnnpack as the execution provider, be sure to include the `--use_xnnpack` option in your build configuration.
There are a few options for building ONNX Runtime for ARM.
First, you may do it on a real ARM device, or on a x86_64 device with an emulator(like qemu), or on a x86_64 device with a docker container with an emulator(you can run an ARM container on a x86_64 PC). Then the build instructions are essentially the same as the instructions for Linux x86_64. However, it wouldn't work if your the CPU you are targeting is not 64-bit since the build process needs more than 2GB memory.
* [Cross compiling for ARM with simulation (Linux/Windows)](#cross-compiling-for-arm-with-simulation-linuxwindows) - **Recommended**; Easy, slow, ARM64 only(no support for ARM32)
This method relies on qemu user mode emulation. It allows you to compile using a desktop or cloud VM through instruction level simulation. You'll run the build on x86 CPU and translate every ARM instruction to x86. This is much faster than compiling natively on a low-end ARM device. The resulting ONNX Runtime Python wheel (.whl) file is then deployed to an ARM device where it can be invoked in Python 3 scripts. The build process can take hours, and may run of memory if the target CPU is 32-bit.
This option is very fast and allows the package to be built in minutes, but is challenging to setup. If you have a large code base (e.g. you are adding a new execution provider to onnxruntime), this may be the only feasible method.
1. Get the corresponding toolchain.
TLDR; Go to https://www.linaro.org/downloads/, get "64-bit Armv8 Cortex-A, little-endian" and "Linux Targeted", not "Bare-Metal Targeted". Extract it to your build machine and add the bin folder to your $PATH env. Then skip this part.
You can use [GCC](https://gcc.gnu.org/) or [Clang](http://clang.llvm.org/). Both work, but instructions here are based on GCC.
When not cross compiling, usually "build" = "host" = "target". When you do cross compile, usually "build" = "host" != "target". For example, you may build GCC on x86_64, then run GCC on x86_64, then generate binaries that target aarch64. In this case,"build" = "host" = x86_64 Linux, target is aarch64 Linux.
You can either build GCC from source code by yourself, or get a prebuilt one from a vendor like Ubuntu, linaro. Choosing the same compiler version as your target operating system is best. If ths is not possible, choose the latest stable one and statically link to the GCC libs.
When you get the compiler, run `aarch64-linux-gnu-gcc -v` This should produce an output like below:
gcc version 9.2.1 20190827 (Red Hat Cross 9.2.1-3) (GCC)
```
Check the value of `--build`, `--host`, `--target`, and if it has special args like `--with-arch=armv8-a`, `--with-arch=armv6`, `--with-tune=arm1176jz-s`, `--with-fpu=vfp`, `--with-float=hard`.
You must also know what kind of flags your target hardware need, which can differ greatly. For example, if you just get the normal ARMv7 compiler and use it for Raspberry Pi V1 directly, it won't work because Raspberry Pi only has ARMv6. Generally every hardware vendor will provide a toolchain; check how that one was built.
Dump the root file system of the target operating system to your build machine. We'll call that folder "sysroot" and use it for build onnxruntime python extension. Before doing that, you should install python3 dev package(which contains the C header files) and numpy python package on the target machine first.
Below are some examples.
If the target OS is raspbian-buster, please download the RAW image from [their website](https://www.raspberrypi.org/downloads/raspbian/) then run:
```bash
$ fdisk -l 2020-02-13-raspbian-buster.img
```
Disk 2020-02-13-raspbian-buster.img: 3.54 GiB, 3787456512 bytes, 7397376 sectors
You'll find the the root partition starts at the 532480 sector, which is 532480 \* 512=272629760 bytes from the beginning.
Then run:
```bash
$ mkdir /mnt/pi
$ mount -r -o loop,offset=272629760 2020-02-13-raspbian-buster.img /mnt/pi
```
You'll see all raspbian files at /mnt/pi. However you can't use it yet. Because some of the symlinks are broken, you must fix them first.
In /mnt/pi, run
```bash
$ find . -type l -exec realpath {} \; |grep 'No such file'
```
It will show which are broken.
Then you can fix them by running:
```bash
$ mkdir /mnt/pi2
$ cd /mnt/pi2
$ sudo tar -C /mnt/pi -cf - . | sudo tar --transform 'flags=s;s,^/,/mnt/pi2/,' -xf -
```
Then /mnt/pi2 is the sysroot folder you'll use in the next step.
If the target OS is Ubuntu, you can get an image from [https://cloud-images.ubuntu.com/](https://cloud-images.ubuntu.com/). But that image is in qcow2 format. Please convert it before run fdisk and mount.
```bash
qemu-img convert -p -O raw ubuntu-18.04-server-cloudimg-arm64.img ubuntu.raw
```
The remaining part is similar to raspbian.
If the target OS is manylinux2014, you can get it by:
Besides, you may also set CMAKE_SYSTEM_PROCESSOR. CMake's official document suggests when doing cross-compiling a CMAKE_TOOLCHAIN_FILE should set the CMAKE_SYSTEM_PROCESSOR variable to match target architecture that it specifies. But the document doesn't provide a list of valid values and we found this setting is not necessary. Anyway, if you know which value the variable should be set to, please add the setting there. ONNX Runtime's build scripts do not use this variable, but ONNX Runtime's dependencies may.
Append `-Donnxruntime_ENABLE_CPUINFO=OFF -DCMAKE_TOOLCHAIN_FILE=path/to/tool.cmake` to your cmake args, run cmake and make to build it. If you want to build Python package as well, you can use cmake args like:
This is not needed if you only want to target a specfic Linux distribution (i.e. Ubuntu).
#### Cross compiling on Windows
**Using Visual C++ compilers**
1. Download and install Visual C++ compilers and libraries for ARM(64).
If you have Visual Studio installed, please use the Visual Studio Installer (look under the section `Individual components` after choosing to `modify` Visual Studio) to download and install the corresponding ARM(64) compilers and libraries.
2. Use `.\build.bat` and specify `--arm` or `--arm64` as the build option to start building. Preferably use `Developer Command Prompt for VS` or make sure all the installed cross-compilers are findable from the command prompt being used to build using the PATH environmant variable.
