mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-14 20:48:00 +00:00

Initial PR for RKNPU execution provider (#3609 )

* Initial RKNPU execution provider

    * Init

    * Support Ops:
        Conv, Relu, Clip, LeakyRelu,
        MaxPool, AveragePool, GlobalAveragePool,
        Concat, Softmax, BatchNormalization, Gemm,
        Add, Mul, Sub,
        Reshape, Squeeze, Unsqueeze,
        Flatten, Transpose,
        QLinearConv, DequantizeLinear

    * Add rknpu unittest

    * Update BUILD.md and Add RKNPU-ExecutionProvider.md

* misc code update

* fix CLIP accuracy issue.

* fix "Error: Duplicate definition of name".

* move rknpu_ddk out of onnxruntime submodule.

* remove temporary code.

* add rknpu namespace.

* update misc of node_attr_helper

* add const & comment for onnx_converter

* add const & comment for shaper

* unify variable name

Co-authored-by: dkm <dkm@rock-chips.com>
Co-authored-by: George Wu <jywu@microsoft.com>

2020-05-05 20:36:47 -07:00

2 KiB

Raw Blame History

RKNPU Execution Provider

RKNPU DDK is an advanced interface to access Rockchip NPU, currently support platform as follow:

RK1808 Linux

Note: RK3399Pro platform is not supported.

RKNPU Execution Provider enables deep learning inference on Rockchip NPU via RKNPU DDK.

Build

For build instructions, please see the BUILD page.

Using

C/C++

To use RKNPU as execution provider for inferencing, please register it as below.

string log_id = "Foo";
auto logging_manager = std::make_unique<LoggingManager>
(std::unique_ptr<ISink>{new CLogSink{}},
                                  static_cast<Severity>(lm_info.default_warning_level),
                                  false,
                                  LoggingManager::InstanceType::Default,
                                  &log_id)
Environment::Create(std::move(logging_manager), env)
InferenceSession session_object{so,env};
session_object.RegisterExecutionProvider(std::make_unique<::onnxruntime::RknpuExecutionProvider>());
status = session_object.Load(model_file_name);

The C API details are here.

ONNX Op supported using RKNPU

The table below shows the ONNX Ops supported using RKNPU Execution Provider and the mapping between ONNX Ops and RKNPU Ops.

ONNX Ops	RKNPU Ops
Add	ADD
Mul	MULTIPLY
Conv	CONV2D
QLinearConv	CONV2D
Gemm	FULLCONNECT
Softmax	SOFTMAX
AveragePool	POOL
GlobalAveragePool	POOL
MaxPool	POOL
GlobalMaxPool	POOL
LeakyRelu	LEAKY_RELU
Concat	CONCAT
BatchNormalization	BATCH_NORM
Reshape	RESHAPE
Flatten	RESHAPE
Squeeze	RESHAPE
Unsqueeze	RESHAPE
Transpose	PERMUTE
Relu	RELU
Sub	SUBTRACT
Clip(0~6)	RELU6
DequantizeLinear	DATACONVERT
Clip	CLIP

Model Supported

Below Models are supported from ONNX open model zoo using RKNPU Execution Provider

Image Classification

squeezenet
mobilenetv2-1.0
resnet50v1
resnet50v2
inception_v2

Object Detection

ssd
yolov3

2 KiB Raw Blame History