saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-17 21:10:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/js/common
History
Yulong Wang e5ca3f3dcb
[js/api] introducing IO binding for tensor (#16452) 
[//]: # (## Work In Progress. Feedbacks are welcome!)

### Description
This PR adds a few properties, methods and factories to Tensor type to
support IO-binding feature. This will allow user to create tensor from
GPU/CPU bound data without a force transferring of data between CPU and
GPU.

This change is a way to resolve #15312

### Change Summary
1. Add properties to `Tensor` type:
a. `location`: indicating where the data is sitting. valid values are
`cpu`, `cpu-pinned`, `texture`, `gpu-buffer`.
b. `texture`: sit side to `data`, a readonly property of `WebGLTexture`
type. available only when `location === 'texture'`
c. `gpuBuffer`: sit side to `data`, a readonly property of `GPUBuffer`
type. available only when `location === 'gpu-buffer'`

2. Add methods to `Tensor` type (usually dealing with inference
outputs):
- async function `getData()` allows user to download data from GPU to
CPU manually.
- function `dispose()` allows user to release GPU resources manually.

3. Add factories for creating `Tensor` instances:
    a. `fromTexture()` to create a WebGL texture bound tensor data
    b. `fromGpuBuffer()` to create a WebGPUBuffer bound tensor data
    c. `fromPinnedBuffer()` to create a tensor using a CPU pinned buffer

### Examples:

create tensors from texture and pass to inference session as inputs
```js
// when create session, specify we prefer 'image_output:0' to be stored on GPU as texture
const session = await InferenceSession.create('./my_model.onnx', {
  executionProviders: [ 'webgl' ],
  preferredOutputLocation: { 'image_output:0': 'texture' }
});

...

const myImageTexture = getTexture(); // user's function to get a texture
const myFeeds = { input0: Tensor.fromTexture(myImageTexture, { width: 224, height: 224 }) }; // shape [1, 224, 224, 4], RGBA format.
const results = await session.run(myFeeds);
const myOutputTexture = results['image_output:0'].texture;
```
2023-08-29 12:58:26 -07:00
..
lib [js/api] introducing IO binding for tensor (#16452) 2023-08-29 12:58:26 -07:00
test [js/common] a few fixes/revises to onnxruntime-common (#16853) 2023-08-01 11:17:39 -07:00
.gitignore [js/common] a few fixes/revises to onnxruntime-common (#16853) 2023-08-01 11:17:39 -07:00
.npmignore [js/common] a few fixes/revises to onnxruntime-common (#16853) 2023-08-01 11:17:39 -07:00
build.js [js/common] allow import onnxruntime-common as ESM and CJS (#15772) 2023-06-12 12:05:11 -07:00
package-lock.json Update VERSION_NUMBER (#15773) 2023-05-03 15:07:34 -07:00
package.json [js/common] a few fixes/revises to onnxruntime-common (#16853) 2023-08-01 11:17:39 -07:00
README.md Replace 'master' branch ref to 'main' in the code (#12547) 2022-08-22 10:48:12 -07:00
tsconfig.json [js] enable formatter for more file types (#16888) 2023-07-28 15:46:58 -07:00
typedoc.json [js] enable formatter for more file types (#16888) 2023-07-28 15:46:58 -07:00
webpack.config.js [js/common] allow import onnxruntime-common as ESM and CJS (#15772) 2023-06-12 12:05:11 -07:00

README.md

ONNX Runtime JavaScript API

ONNX Runtime JavaScript API is a unified API for all JavaScript usages. It's dependency of the following NPM packages:

  • onnxruntime-node
  • onnxruntime-web
  • onnxruntime-react-native

This package (onnxruntime-common) is not designed for using directly. Please consider to install one of the NPM packages above according to target platform.

License

License information can be found here.