mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-18 18:52:16 +00:00
Test staging website from a branch (#17440)
This commit is contained in:
parent
d25834c7f7
commit
09f9e6d6fa
2 changed files with 23 additions and 6 deletions
2
.github/workflows/publish-gh-pages.yml
vendored
2
.github/workflows/publish-gh-pages.yml
vendored
|
|
@ -28,8 +28,6 @@ jobs:
|
|||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
ref: gh-pages
|
||||
|
||||
- name: Setup Pages
|
||||
uses: actions/configure-pages@v3
|
||||
|
|
|
|||
|
|
@ -8,17 +8,17 @@ nav_order: 7
|
|||
|
||||
[](https://dev.azure.com/onnxruntime/onnxruntime/_build/latest?definitionId=209&branchName=main)
|
||||
|
||||
ONNXRuntime-Extensions is a library that extends the capability of the ONNX models and inference with ONNX Runtime, via the ONNX Runtime custom operator interface. It includes a set of Custom Operators to support common model pre and post-processing for audio, vision, text, and language models. As with ONNX Runtime, Extensions also supports multiple languages and platforms (Python on Windows/Linux/macOS, Android and iOS mobile platforms and Web-Assembly for web.
|
||||
ONNXRuntime-Extensions is a library that extends the capability of the ONNX models and inference with ONNX Runtime, via the ONNX Runtime custom operator interface. It includes a set of Custom Operators to support common model pre and post-processing for audio, vision, text, and language models. As with ONNX Runtime, Extensions also supports multiple languages and platforms (Python on Windows/Linux/macOS, Android and iOS mobile platforms and Web assembly for web).
|
||||
|
||||
The basic workflow is to add the custom operators to an ONNX model and then to perform inference on the enhanced model with ONNX Runtime and ONNXRuntime-Extensions packages.
|
||||
|
||||
|
||||
<img src="../../images/combine-ai-extensions-img.png" alt="Pre and post-processing custom operators for vision, text, and NLP models" width="100%"/>
|
||||
<sub>This image was created using <a href="https://github.com/sayanshaw24/combine" target="_blank">Combine.AI</a>, which is powered by Bing Chat, Bing Image Creator, and EdgeGPT.</sub>
|
||||
|
||||
## Quickstart
|
||||
|
||||
### **Python installation**
|
||||
|
||||
```bash
|
||||
pip install onnxruntime-extensions
|
||||
```
|
||||
|
|
@ -26,17 +26,23 @@ pip install onnxruntime-extensions
|
|||
#### **Nightly Build**
|
||||
|
||||
##### on Windows
|
||||
|
||||
```cmd
|
||||
pip install --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-extensions
|
||||
```
|
||||
Please ensure that you have met the prerequisites of onnxruntime-extensions (e.g., onnx and onnxruntime) in your Python environment.
|
||||
#### <strong>on Linux/macOS</strong>
|
||||
|
||||
The onnxruntime-extensions package depends on onnx and onnxruntime.
|
||||
|
||||
##### on Linux/MacOS
|
||||
|
||||
Please make sure the compiler toolkit like gcc(later than g++ 8.0) or clang are installed before the following command
|
||||
|
||||
```bash
|
||||
python -m pip install git+https://github.com/microsoft/onnxruntime-extensions.git
|
||||
```
|
||||
|
||||
### **NuGet installation (with .NET CLI)**
|
||||
|
||||
```bash
|
||||
dotnet add package Microsoft.ML.OnnxRuntime.Extensions --version 0.8.1-alpha
|
||||
```
|
||||
|
|
@ -81,13 +87,16 @@ In your Android Studio Project, make the following changes to:
|
|||
```
|
||||
|
||||
## Add pre and post-processing to the model
|
||||
|
||||
There are multiple ways to add pre and post processing to an ONNX graph:
|
||||
|
||||
- [Use the pre-processing pipeline API if the model and its pre-processing is supported by the pipeline API](https://github.com/microsoft/onnxruntime-extensions/blob/main/onnxruntime_extensions/tools/pre_post_processing/pre_post_processor.py)
|
||||
- [Export to ONNX from a PyTorch model](https://github.com/microsoft/onnxruntime-extensions/blob/main/tutorials/superresolution_e2e.py#L69)
|
||||
- [Create an ONNX model with a model graph that includes your custom op node](https://github.com/microsoft/onnxruntime-extensions/blob/main/onnxruntime_extensions/_ortapi2.py#L50)
|
||||
- [Compose the pre-processing with an ONNX model using ONNX APIs if you already have the pre processing in an ONNX graph](https://onnx.ai/onnx/api/compose.html)
|
||||
|
||||
If the pre processing operator is a HuggingFace tokenizer, you can also easily get the ONNX processing graph by converting from Huggingface transformer data processing classes such as in the following example:
|
||||
|
||||
```python
|
||||
import onnxruntime as _ort
|
||||
from transformers import AutoTokenizer
|
||||
|
|
@ -98,17 +107,21 @@ model = OrtPyFunction(gen_processing_models(tokenizer, pre_kwargs={})[0])
|
|||
```
|
||||
|
||||
For more information, you can check the API using the following:
|
||||
|
||||
```python
|
||||
help(onnxruntime_extensions.gen_processing_models)
|
||||
```
|
||||
|
||||
### What if I cannot find the custom operator I am looking for?
|
||||
|
||||
Find the custom operators we currently support [here](https://github.com/microsoft/onnxruntime-extensions/tree/main/operators). If you do not find the custom operator you are looking for, you can add a new custom operator to ONNX Runtime Extensions like [this](./add-op.md). Note that if you do add a new operator, you will have to [build from source](./build.md).
|
||||
|
||||
## Inference with ONNX Runtime and Extensions
|
||||
|
||||
### Python
|
||||
|
||||
There are individual packages for the following languages, please install it for the build.
|
||||
|
||||
```python
|
||||
import onnxruntime as _ort
|
||||
from onnxruntime_extensions import get_library_path as _lib_path
|
||||
|
|
@ -122,7 +135,9 @@ sess.run (...)
|
|||
```
|
||||
|
||||
### C++
|
||||
|
||||
Register Extensions with a path to the Extensions shared library.
|
||||
|
||||
```c++
|
||||
Ort::Env env = ...;
|
||||
|
||||
|
|
@ -141,6 +156,7 @@ Ort::Session session(env, model_uri, session_options);
|
|||
```
|
||||
|
||||
Register Extensions by calling the `RegisterCustomOps` function directly.
|
||||
|
||||
```c++
|
||||
Ort::Env env = ...;
|
||||
|
||||
|
|
@ -158,6 +174,7 @@ Ort::Session session(env, model_uri, session_options);
|
|||
```
|
||||
|
||||
### Java
|
||||
|
||||
```java
|
||||
var env = OrtEnvironment.getEnvironment();
|
||||
var sess_opt = new OrtSession.SessionOptions();
|
||||
|
|
@ -167,6 +184,7 @@ sess_opt.registerCustomOpLibrary(OrtxPackage.getLibraryPath());
|
|||
```
|
||||
|
||||
### C#
|
||||
|
||||
```java
|
||||
SessionOptions options = new SessionOptions();
|
||||
options.RegisterOrtExtensions();
|
||||
|
|
@ -176,6 +194,7 @@ session = new InferenceSession(model, options);
|
|||
## Tutorials
|
||||
|
||||
Check out some end to end tutorials with our custom operators:
|
||||
|
||||
- NLP: [An end-to-end BERT tutorial](https://github.com/microsoft/onnxruntime-extensions/blob/main/tutorials/bert_e2e.py)
|
||||
- Audio: [Using audio encoding and decoding for Whisper](https://github.com/microsoft/onnxruntime-extensions/blob/main/tutorials/whisper_e2e.py)
|
||||
- Vision: [The YOLO model with our DrawBoundingBoxes operator](https://github.com/microsoft/onnxruntime-extensions/blob/main/tutorials/yolo_e2e.py)
|
||||
|
|
|
|||
Loading…
Reference in a new issue