The ONNX runtime provides a C# .Net binding for running inference on ONNX models in any of the .Net standard platforms. The API is .Net standard 1.1 compliant for maximum portability. This document describes the API.
## Contents
{: .no_toc }
* TOC placeholder
{:toc}
## NuGet Package
The Microsoft.ML.OnnxRuntime Nuget package includes the precompiled binaries for ONNX runtime, and includes libraries for Windows and Linux platforms with X64 CPUs. The APIs conform to .Net Standard 1.1.
## Sample Code
The unit tests contain several examples of loading models, inspecting input/output node shapes and types, as well as constructing tensors for scoring.
Here is simple tutorial for getting started with running inference on an existing ONNX model for a given input data. The model is typically trained using any of the well-known training frameworks and exported into the ONNX format. To start scoring using the model, open a session using the `InferenceSession` class, passing in the file path to the model as a parameter.
```cs
var session = new InferenceSession("model.onnx");
```
Once a session is created, you can execute queries using the `Run` method of the `InferenceSession` object. Currently, only `Tensor` type of input and outputs are supported. The results of the `Run` method are represented as a collection of .Net `Tensor` objects (as defined in [System.Numerics.Tensor](https://www.nuget.org/packages/System.Numerics.Tensors)).
```cs
Tensor<float> t1, t2; // let's say data is fed into the Tensor objects
Here is a [complete sample code](https://github.com/microsoft/onnxruntime/blob/master/csharp/sample/Microsoft.ML.OnnxRuntime.InferenceSample) that runs inference on a pretrained model.
In some scenarios, you may want to reuse input/output tensors. This often happens when you want to chain 2 models (ie. feed one's output as input to another), or want to accelerate inference speed during multiple inference runs.
### Chaining: Feed model A's output(s) as input(s) to model B
```cs
InferenceSession session1, session2; // let's say 2 sessions are initialized
Tensor<float> t1; // let's say data is fed into the Tensor objects
var inputs2 = new List<NamedOnnxValue>() { input2 };
// session2 inference
using (var results = session2.Run(inputs2))
{
// manipulate the results
}
}
```
### Multiple inference runs with fixed sized input(s) and output(s)
If the model have fixed sized inputs and outputs of numeric tensors, you can use "FixedBufferOnnxValue" to accelerate the inference speed. By using "FixedBufferOnnxValue", the container objects only need to be allocated/disposed one time during multiple InferenceSession.Run() calls. This avoids some overhead which may be beneficial for smaller models where the time is noticeable in the overall running time.
An example can be found at `TestReusingFixedBufferOnnxValueNonStringTypeMultiInferences()`:
Holds some methods which can be used to tune the ONNX Runtime's runime environment
#### Constructor
No public constructor available.
#### Methods
```cs
static OrtEnv Instance();
```
Returns an instance of the singlton class `OrtEnv`.
```cs
void EnableTelemetryEvents();
```
Enables platform-specific telemetry collection where applicable. Please see [Privacy](https://github.com/microsoft/onnxruntime/blob/master/docs/Privacy.md) for more details.
```cs
void DisableTelemetryEvents();
```
Disables platform-specific telemetry collection. Please see [Privacy](https://github.com/microsoft/onnxruntime/blob/master/docs/Privacy.md) for more details.
Runs the model with the given input data to compute all the output nodes and returns the output node values. Both input and output are collection of NamedOnnxValue, which in turn is a name-value pair of string names and Tensor values. The outputs are IDisposable variant of NamedOnnxValue, since they wrap some unmanaged objects.
Runs the model on given inputs for the given output nodes only.
### System.Numerics.Tensor
The primary .Net object that is used for holding input-output of the model inference. Details on this newly introduced data type can be found in its [open-source implementation](https://github.com/dotnet/corefx/tree/master/src/System.Numerics.Tensors). The binaries are available as a [.Net NuGet package](https://www.nuget.org/packages/System.Numerics.Tensors).
### NamedOnnxValue
```cs
class NamedOnnxValue;
```
Represents a name-value pair of string names and any type of value that ONNX runtime supports as input-output data. Currently, only Tensor objects are supported as input-output values.
