Commit graph

25 commits

Author SHA1 Message Date
Yulong Wang
82786baed1
[js/web] add 'xnnpack' to EP list (#12723)
**Description**: This PR adds support for "XNNPACK EP" in ORTWeb and
changes the behavior of how ORTWeb deals with "backends", or "EPs" in
API.

**Background**: Term "backend" is introduced in ONNX.js to representing
a TypeScript type which implements a "backend" interface, which is a
similar but different concept to ORT's EP (execution provider). There
was 3 backends in ONNX.js: "cpu", "wasm" and "webgl".

When ORT Web is launched, the concept is derived to help users to
integrate smoothly. Technically, when "wasm" backend is used, users need
to also specify "EP" in the session options. Considering it may get
complicated and confused for users to figure out the difference between
"backend" and "EP", the JS API hide the "backend" concept and made a
mapping between names, backends and EPs:
"webgl" (Name) <==> "onnxjsBackend" (Backend)
"wasm" (Name) <==> "wasmBackend" (Backend) <==> "CPU" (EP)

**Details**:
The following changes are applied in this PR:
1. allow multi-registration for backends using the same name. This is
for use scenarios where both "onnxruntime-node" and "onnxruntime-web"
are consumed in a Node.js App ( so "cpu" will be registered twice in
this scenario. )
2. re-assign priority values to backends. I give 100 as base to "cpu"
for node and react_native, and 10 as base to "cpu" in web.
3. add "cpu", "xnnpack" as new names of backends.
4. update onnxruntime wasm exported functions to support EP
registration.
5. update implementations in ort web to handle execution providers in
session options.
6. add '--use_xnnpack' as default build flag for ort-web
2022-10-03 10:38:45 -07:00
shalvamist
851b0ce936
[js/web][Fix] - updating the C API to catch non-tensor data (#12811)
Added a check for tensor validation on the input - this change fixes the
quiet abort WASM takes when processing a non tensor data in
"OrtGetTensorData"

**Motivation and Context**
At the current status when we try to process non-tensor data through
OrtGetTensorData and exception is thrown which results in a quiet abort
from WASM (assuming WASM was built without exception handling).

I added a check in the C API to catch this case and output a meaningful
message to the user

[example_error_github_12622.zip](https://github.com/microsoft/onnxruntime/files/9464328/example_error_github_12622.zip)
2022-09-21 13:59:17 -07:00
Yulong Wang
af21a04977
[js] upgrade async@3.2.3 /js/ (#11421)
* [js] upgrade async@3.2.3 /js/

* format code
2022-05-03 23:41:36 -07:00
Yulong Wang
6c7090a829
[js/web] fix output type mapping (#11049) 2022-03-30 16:26:04 -07:00
Yulong Wang
893ee65e54
[js/web] fix lint error when run without ort-web TS types (#10429)
* [js/web] fix lint error when run without ort-web TS types

* update CI to run linter before 'npm ci' in /js/web
2022-02-17 22:34:38 -08:00
Yulong Wang
74ca417c0e
[js/web] optimize bundle file size (#9817)
* es2017 by default for ort-common

* add visualizer and define plugin

* es2017 for ort-web. also add build target for es5

* add multiple reduced size build for ort-web

* resolve comments, add e2e tests and add docs
2021-11-22 13:56:55 -08:00
Yulong Wang
634bb5ede0
fix CodeQL warning 'Remote property injection' (#9224) 2021-09-30 13:45:22 -07:00
Yulong Wang
8c57d51928
support WebAssembly SIMD for qgemm (#9191)
* support WebAssembly SIMD for qgemm

* remove '--experimental-wasm-bulk-memory' for test
2021-09-30 12:40:56 -07:00
Yulong Wang
750e2e0481
[js/web] check session ID in releaseSession() (#9105) 2021-09-20 17:49:53 -07:00
Yulong Wang
be80698698
[js/web] a bugfix and add tests for wasm proxy worker (#9048)
* [js/web] add tests for wasm proxy worker

* fix script src override
2021-09-14 10:38:58 -07:00
Sunghoon
450524359e
[js/web] WebAssembly profiling (#8932)
* add p50 in test

* Preallocate WebAssembly worker threads to minimize worker creation time

* WebAssembly profiling

* merge master

* merge with proxy changes

* disable profiling tests from WebAssembly build

* fix e2e test failure

Co-authored-by: Yulong Wang <yulongw@microsoft.com>
2021-09-07 17:18:08 -07:00
Yulong Wang
206537936f
[js/web] enable proxy worker for wasm backend (#8862) 2021-08-31 10:23:42 -07:00
Yulong Wang
cb67fca738
[js/web] enable 'use_ort_model_bytes_directly' by default (#8734) 2021-08-18 11:18:58 -07:00
Yulong Wang
4ceedbe933
[js/web] add SharedArrayBuffer check for wasm multi-thread (#8749) 2021-08-16 23:17:54 -07:00
Yulong Wang
f3a1aebb33
[js/web] support override wasm file path (#8610) 2021-08-05 18:01:03 -07:00
Gao, Chun
0f01de3b0b
[js/web] Add wasm SIMD backend to onnxruntime-web (#7896)
* [js/web] Add wasm SIMD backend to onnxruntime-web

* Import SIMD wasm artifacts enabled by PR #7839

* Detect SIMD capability of web engine

* Use SIMD wasm backend in both single-thread and multi-thread cases

* update optimized SIMD loading from ort web

* code lint and format

* fix WasmFileName in CI

* replace deprecated wasm SIMD functions

* fix unittest for simd

* optimize CI pipeline to merge build matrix

* make clean build for each config

* fix simd wasm to enable it.

* update script/pull-prebuilt-wasm-artifacts.ts

Co-authored-by: Yulong Wang <yulongw@microsoft.com>
Co-authored-by: Lei Zhang <zhang.huanning@hotmail.com>
2021-06-07 23:24:27 -07:00
Yulong Wang
896f32ec09
[js/web] support string tensor for wasm backend (#7891)
* [js/web] support string tensor for wasm backend

* disable v9/test_cast_STRING_to_FLOAT: test data is wrong

* add non-string check

* Update session-handler.ts

* Update session-handler.ts
2021-06-03 00:44:50 -07:00
Sunghoon
da5f24bd2d
Support additional session options and run options in WebAssembly (#7712)
* add all session options and run options in C API except AddInitializer and AddFreeDimensionOverride

* remove unnecessary comment

* change extra session and run options to object notation

* resolve comments

* use an optional chaining for options

* resolve comments
2021-05-17 14:57:19 -07:00
Tixxx
6d9f541442
[JS]moved logging level flag to global env (#7700)
* moved logging level flag to global env

* added setter and getter for loggingLevel in Env

* moved implementation of env to a separate file
2021-05-17 14:16:59 -07:00
Yulong Wang
97d9bcd644
[js/web] fix bundle for multi-thread, add e2e test and support nodejs (#7688)
* fix bundle for multi-thread, add e2e test and support nodejs

* add copyright banner

* resolve comments

* add comments for isMultiThreadSupported()
2021-05-14 18:15:38 -07:00
Sunghoon
1ab8a95eb6
Bind existing SessionOptions and RunOptions in Javascript API with WebAssembly (#7621)
* support session options and run options. use onnxruntime c api.

* fix lint errors

* add an error code on throwing an exception

* resolve comments. change remaining C++ APIs to C API
2021-05-13 10:50:04 -07:00
Yulong Wang
bdefc6c4d8
[js/web] support multi-thread for wasm backend (#7601) 2021-05-07 12:12:37 -07:00
Yulong Wang
3600c3e66e
[js/web] integrate latest changes from onnxjs (#7535)
* [js/web] integrate latest changes from onnxjs

* apply ESLint rules: filename-case and header

* remove filename-case rule for wasm .d.ts
2021-05-03 15:03:25 -07:00
Yulong Wang
7079dfb93d
[wasm] fix and unify webassembly target name (#7549) 2021-05-03 10:37:25 -07:00
Yulong Wang
4ebc9c3b5e
[JS] onnxruntime-web (#7394)
* add web

* add script and test

* fix lint

* add test/data/ops

* add test/data/node/ to gitignore

* modify scripts

* add onnxjs

* fix tests

* fix test-runner

* fix sourcemap

* fix onnxjs profiling

* update test list

* update README

* resolve comments

* set wasm as default backend

* rename package

* update copyright header

* do not use class "Buffer" in browser context

* revise readme
2021-04-27 00:04:25 -07:00