saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-15 20:50:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/csharp/sample/InferenceSample
History
Dmitri Smirnov 322237f482
[C#] Implement OrtValue APIs (#16206) 
### Description

Expose `OrtValue` class API as first-class citizen.
Make it simular with C++ API.
Enable safe direct native memory access.
Make string tensor manipulation more efficient.
Avoid intermediate structures such as `NamedOnnxValue`,
`DisposableNamedOnnxvalue` and etc.

Provide more examples with `IOBinding`, although `OrtValue` API
potentially makes `IOBinding` redundant for most of scenarios, since
`OrtValue` can be created on top of any memory.

Run all the pre-trained models now with `OrtValue` API as well.
Obsolete `OrtExternalMemory class`. Obsolete IOBinding API that takes
`FixedBufferOnnxValue`.

### Motivation and Context
Make the API efficient and uniform with C++.

This aspires to address: 
https://github.com/microsoft/onnxruntime/issues/14918
https://github.com/microsoft/onnxruntime/issues/15381

Cc: @Craigacp
2023-06-29 08:59:23 -07:00
..
Microsoft.ML.OnnxRuntime.InferenceSample [C#] Implement OrtValue APIs (#16206) 2023-06-29 08:59:23 -07:00
Microsoft.ML.OnnxRuntime.InferenceSample.Forms
Microsoft.ML.OnnxRuntime.InferenceSample.Forms.Android
Microsoft.ML.OnnxRuntime.InferenceSample.Forms.iOS
Microsoft.ML.OnnxRuntime.InferenceSample.Maui
Microsoft.ML.OnnxRuntime.InferenceSample.NetCoreApp [C#] Implement OrtValue APIs (#16206) 2023-06-29 08:59:23 -07:00
readme.md

readme.md

To test the iOS or Android samples the native build of ONNX Runtime is required and must be in a specific location.

Only the native build for the platform you are testing on is required. e.g. if you're testing using an Android device that is arm64, you only need the libonnxruntime.so for arm64-v8a. The version of the native build should match the checked-out version of the ONNX Runtime repository you're currently using as closely as possible. Otherwise mismatches with the native entry points is possible and could cause crashes.

To acquire the native build you can:

  • build it yourself
    • Android build instructions
    • iOS build instructions
  • extract it from the Microsoft.ML.OnnxRuntime nuget package using NuGetPackageExplorer
    • release version is here
    • integration test version is here
      • this is frequently updated and should work if you're currently using the master branch of ONNX Runtime
  • or if you have access to the internal packaging pipelines
    • the Zip-Nuget-Java-Nodejs Packaging Pipeline produces the native package as an artifact under drop-signed-nuget-CPU
      • run a build for your current branch in the pipeline to ensure the native build matches exactly

For iOS the native build should be at one or more of:

  • \build\iOS\iphoneos\Release\Release-iphoneos\onnxruntime.framework for an iOS device
  • \build\iOS\iphonesimulator\Release\Release-iphonesimulator\onnxruntime.framework for an iOS simulator

For Android the native build should be at one or more of:

  • \build\Android\arm64-v8a\Release\libonnxruntime.so for an 64-bit arm device
  • \build\Android\armeabi-v7a\Release\libonnxruntime.so for an 32-bit arm device
  • \build\Android\x86\Release\libonnxruntime.so for an x86 Android emulator
  • \build\Android\x86_64\Release\libonnxruntime.so for an x86_64 Android emulator