saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-20 21:40:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/dockerfiles/README.md
Vinitra Swamy 6b32c77804 Dockerfiles for TensorRT, CUDA, build from source (#922) 
* dockerfile updates for BYOC scenario

* updates for 3 different build versions

* updating to remove libopenblas, python3, python3-pip

* Including LICENSE-IMAGE.txt for CUDA/TensorRT dockerfiles

* remove unnecessary cmake files

* fixing comment typo

* optimizing dockerfile.source as per review suggestions (not working currently)

* Optimizing dockerfiles with install_dependencies script

* update dockerfile with --cmake_extra_defines version number

* add &&\ for license copy lines

* updates, adding miniconda to path, reincluded clearing the pycache

* adding maintainer note

* update readme instructions

* update tensorrt versioning in dockerfile
2019-07-09 02:03:55 -07:00

3.3 KiB
Raw Blame History

Docker containers for ONNX Runtime

Build from Source Version (Preview)

Linux 16.04, CPU, Python Bindings

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-source -f Dockerfile.source .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-source

CUDA Version (Preview)

Linux 16.04, CUDA 10.0, CuDNN 7

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-cuda -f Dockerfile.cuda .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-cuda

nGraph Version (Preview)

Linux 16.04, Python Bindings

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-ngraph -f Dockerfile.ngraph .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-ngraph

TensorRT Version (Preview)

Linux 16.04, TensorRT 5.0.2

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-trt -f Dockerfile.tensorrt .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-trt

OpenVINO Version (Preview)

Linux 16.04, Python Bindings

  1. Build the docker image from the Dockerfile in this repository.
# If you have a Linux machine, preface this command with "sudo"

docker build -t onnxruntime-openvino -f Dockerfile.openvino .

To use GPU_FP32:

docker build -t onnxruntime-openvino --build-arg TARGET_DEVICE=GPU_FP32 -f Dockerfile.openvino .
  1. Run the Docker image
# If you have a Linux machine, preface this command with "sudo"

docker run -it onnxruntime-openvino

ONNX Runtime Server (Preview)

Linux 16.04

  1. Build the docker image from the Dockerfile in this repository
docker build -t {docker_image_name} -f Dockerfile.server .
  1. Run the ONNXRuntime server with the image created in step 1
docker run -v {localModelAbsoluteFolder}:{dockerModelAbsoluteFolder} -e MODEL_ABSOLUTE_PATH={dockerModelAbsolutePath} -p {your_local_port}:8001 {imageName}
  1. Send HTTP requests to the container running ONNX Runtime Server

Send HTTP requests to the docker container through the binding local port. Here is the full usage document.

curl  -X POST -d "@request.json" -H "Content-Type: application/json" http://0.0.0.0:{your_local_port}/v1/models/mymodel/versions/3:predict