saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-14 20:48:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/dockerfiles
History
rakelkar 0f7c01b49b Use exec form of ENTRYPOINT for docker server (#1690) 
* Use exec form of ENTRYPOINT for docker server

# Issue
The entrypoint currently uses the shell form - this prevents users from passing in any cmdline arguments... also passing a model_path in means the server only works in the envvar is set... however this is not what the error message says!
```
$ docker run -v /home/rakelkar/try/onnxzoo/style:/mnt/models -it   mcr.microsoft.com/onnxruntime/server --model_path /mnt/models/model.onnx
Version: local_build
Commit ID: default

model_path must be the location of a valid file
Allowed options:
  -h [ --help ]               Shows a help message and exits
  --log_level arg (=info)     Logging level. Allowed options (case sensitive): 
                              verbose, info, warning, error, fatal
  --model_path arg            Path to ONNX model
  --address arg (=0.0.0.0)    The base HTTP address
  --http_port arg (=8001)     HTTP port to listen to requests
  --num_http_threads arg (=4) Number of http threads
  --grpc_port arg (=50051)    GRPC port to listen to requests
```
# Fix
1. remove the env var
2. use the exec form

* Update readme to use model_path arg
2019-08-29 10:18:08 -07:00
..
Dockerfile.arm32v7 Treat attribute warning as non-error on cross compiling ARM (#1261) 2019-06-23 17:59:38 -07:00
Dockerfile.cuda Added license files in the base image (#1595) 2019-08-09 13:02:06 -07:00
Dockerfile.ngraph Allow building Docker container based on a different git repo. (#1222) 2019-06-20 09:55:42 -07:00
Dockerfile.openvino Added license files in the base image (#1595) 2019-08-09 13:02:06 -07:00
Dockerfile.server Use exec form of ENTRYPOINT for docker server (#1690) 2019-08-29 10:18:08 -07:00
Dockerfile.source Dockerfiles for TensorRT, CUDA, build from source (#922) 2019-07-09 02:03:55 -07:00
Dockerfile.tensorrt Added license files in the base image (#1595) 2019-08-09 13:02:06 -07:00
install_common_deps.sh Dockerfiles for TensorRT, CUDA, build from source (#922) 2019-07-09 02:03:55 -07:00
LICENSE-IMAGE.txt Dockerfiles for TensorRT, CUDA, build from source (#922) 2019-07-09 02:03:55 -07:00
README.md Use exec form of ENTRYPOINT for docker server (#1690) 2019-08-29 10:18:08 -07:00

README.md

Docker containers for ONNX Runtime

Build from Source

Linux 16.04, CPU, Python Bindings

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-source -f Dockerfile.source .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-source

CUDA

Linux 16.04, CUDA 10.0, CuDNN 7

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-cuda -f Dockerfile.cuda .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-cuda

nGraph (Public Preview)

Linux 16.04, Python Bindings

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-ngraph -f Dockerfile.ngraph .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-ngraph

TensorRT

Linux 16.04, TensorRT 5.0.2

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-trt -f Dockerfile.tensorrt .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-trt

OpenVINO (Public Preview)

Linux 16.04, Python Bindings

  1. Build the onnxruntime image for all the accelerators supported as below

    Retrieve your docker image in one of the following ways.

    • For building the docker image, download OpenVINO online installer version 2019 R1.1 from here and copy the openvino tar file in the same directory and build the image. The online installer size is only 16MB and the components needed for the accelerators are mentioned in the dockerfile. Providing the argument device enables onnxruntime for that particular device. You can also provide arguments ONNXRUNTIME_REPO and ONNXRUNTIME_BRANCH to test that particular repo and branch. Default values are http://github.com/microsoft/onnxruntime and repo is master 
      docker build -t onnxruntime --build-arg DEVICE=$DEVICE .
    • Pull the official image from DockerHub.

  2. DEVICE: Specifies the hardware target for building OpenVINO Execution Provider. Below are the options for different Intel target devices.

    Device Option Target Device
    CPU_FP32 | Intel CPUs |
    GPU_FP32 | ntel Integrated Graphics |
    GPU_FP16 | Intel Integrated Graphics |
    MYRIAD_FP16 | Intel MovidiusTM USB sticks |
    VAD-M_FP16 | Intel Vision Accelerator Design based on MovidiusTM MyriadX VPUs |

CPU

  1. Retrieve your docker image in one of the following ways.

    • Build the docker image from the DockerFile in this repository.

      docker build -t onnxruntime-cpu --build-arg DEVICE=CPU_FP32 --network host .

    • Pull the official image from DockerHub.

      # Will be available with next release

  2. Run the docker image

     docker run -it onnxruntime-cpu

GPU

  1. Retrieve your docker image in one of the following ways.

    • Build the docker image from the DockerFile in this repository. 
       docker build -t onnxruntime-gpu --build-arg DEVICE=GPU_FP32 --network host .
    • Pull the official image from DockerHub. 
        # Will be available with next release

  2. Run the docker image

    docker run -it --device /dev/dri:/dev/dri onnxruntime-gpu:latest

Myriad VPU Accelerator

  1. Retrieve your docker image in one of the following ways.
    • Build the docker image from the DockerFile in this repository. 
       docker build -t onnxruntime-myriad --build-arg DEVICE=MYRIAD_FP16 --network host .
    • Pull the official image from DockerHub. 
       # Will be available with next release
  2. Install the Myriad rules drivers on the host machine according to the reference in here
  3. Run the docker image by mounting the device drivers 
    docker run -it --network host --privileged -v /dev:/dev  onnxruntime-myriad:latest

=======

VAD-M Accelerator Version

  1. Retrieve your docker image in one of the following ways.
    • Build the docker image from the DockerFile in this repository. 
       docker build -t onnxruntime-vadr --build-arg DEVICE=VAD-M_FP16 --network host .
    • Pull the official image from DockerHub. 
       # Will be available with next release
  2. Install the HDDL drivers on the host machine according to the reference in here
  3. Run the docker image by mounting the device drivers 
    docker run -it --device --mount type=bind,source=/var/tmp,destination=/var/tmp --device /dev/ion:/dev/ion  onnxruntime-hddl:latest

ONNX Runtime Server (Public Preview)

Linux 16.04

  1. Build the docker image from the Dockerfile in this repository
docker build -t {docker_image_name} -f Dockerfile.server .
  1. Run the ONNXRuntime server with the image created in step 1
docker run -v {localModelAbsoluteFolder}:{dockerModelAbsoluteFolder} -p {your_local_port}:8001 {imageName} --model_path {dockerModelAbsolutePath}
  1. Send HTTP requests to the container running ONNX Runtime Server

Send HTTP requests to the docker container through the binding local port. Here is the full usage document.

curl  -X POST -d "@request.json" -H "Content-Type: application/json" http://0.0.0.0:{your_local_port}/v1/models/mymodel/versions/3:predict