### Description
This PR resolves a part of non-critical comments from code review
comments in #14579.
- use `USE_JSEP` instead of `USE_JS` in build definition to make it less
ambiguous
- remove unused util functions from util.ts
- fix transpose.h
- other misc fixes
### Description
make `RunFunction` return `void`.
the return value is meaningless in the OpResolveRule context. Allows any
JavaScript error to be caught and returns non-zero return value from
`computeKernel()`
### Description
This change introduced the following new components into ONNX Runtime
Web:
- JavaScript Execution Provider (JSEP)
- Asynchronized inferencing execution powered by Emscripten's Asyncify
- WebGPU backend implemented in TypeScript
- initial implementation of kernels:
- elementwise operators (22)
- binary operators (5)
- tensor: Shape, Reshape, Transpose, Gemm
- nn: Conv, {Global}Maxpool, {Global}AveragePool
Code need to be polished. still working on it.
## Q&A
What is JSEP?
> JSEP, aka JavaScript Execution Provider, is a new ONNXRuntime
execution provider that specifically works on Web environment
(browsers). JSEP allows JavaScript code to kick in from various places
when ONNX Runtime inferences a model.
Why JSEP?
> JSEP is a hybrid mode EP that contains both C/C++ and
TypeScript/JavaScript implementation. There are 2 strong reasons why we
introduces JSEP:
> 1. the C/C++ part helps JSEP to leverage ONNX Runtime's capabilities
as much as possible including graph transformer, optimizers and also the
capabilities to fallback to CPU EP. TypeScript/JavaScript helps JSEP to
develop and debug much easier in the browser for the kernel
implementation.
> 2. the requirement of asynchronized execution from JavaScript API (eg.
`buffer.mapAsync()`) makes it impossible to run `OrtRun()` in a
synchronized context (see "async problem" section below). This is done
by using Emscripten's Asyncify.
What is WebGPU?
> WebGPU is the new GPU API that available in browser. It's one of the
only 2 APIs that currently available to access the GPU from browser (the
other is WebGL).
> WebGPU is designed with more advanced and stronger features comparing
to WebGL and is potentially solution that offer the best GPU performance
for model inferencing that currently available.
What is the async problem and why we have the problem?
> The "async problem" is a problem that you cannot call an async
function in a synchronous context. Think about the following C++ code:
> ```c
> // C-style declarations (API)
> typedef void (*ON_COMPLETE)(PVOID state, DATA *data);
> void read_data_from_file(FILEHANDLE file, ON_COMPLETE on_complete);
>
> // implementation
> DATA * my_impl_read_data_from_file_sync(FILEHANDLE file) {
> // how to implement?
> }
> ```
> The answer is, it's impossible to implement this function. Usually we
try to find a sync version API, or launch a thread to call the async
function and sync-wait on the main thread. Unfortunately, in browser
environment, neither is possible.
>
> WebGPU does not offer any synchronized API for data downloading (GPU
to CPU). This is the only operation that MUST be async. As `OrtRun()`
will eventually call into DataTransfer for copy data from GPU to CPU,
and `OrtRun()` is a synchronized function, this cannot be done in normal
way.
What is Emscripten? How is the Asyncify feature resolved the problem?
> Emscripten is the C/C++ compiler for WebAssembly. It's what we use to
compile ORT and generates the WebAssembly artifacts which runs on
browsers.
>
> Asyncify is a [compiler
feature](https://emscripten.org/docs/porting/asyncify.html) that allows
calling async functions from a synchronized context. In short, it
generates code to unwind and rewind call stack to emulate async
execution. With this feature, we are able to call the async function
inside `OrtRun()` call.
## Design Overview
**Inter-op**
JSEP is doing pretty much same thing to just another EP. It exposes an
interface for inter-op with JavaScript, which is defined in
onnxruntime/wasm/js_internal_api.js:
```js
// init JSEP
Module["jsepInit"] = function (backend, alloc, free, copy, copyAsync, createKernel, releaseKernel, run) {
Module.jsepBackend = backend;
Module.jsepAlloc = alloc;
Module.jsepFree = free;
Module.jsepCopy = copy;
Module.jsepCopyAsync = copyAsync;
Module.jsepCreateKernel = createKernel;
Module.jsepReleaseKernel = releaseKernel;
Module.jsepRun = run;
};
```
This simple JavaScript snippet defines all language barrier level
functions that requires by JSEP to achieve implementing kernels and data
transfers using JavaScript inside ONNX Runtime:
- `jsepBackend`: assign the singleton object to webassembly module
- `jsepAlloc` and `jsepFree`: implementation of data transfer's Alloc()
and Free()
- `jsepCopy`: synchronized copy ( GPU to GPU, CPU to GPU)
- `jsepCopyAsync`: asynchronized copy ( GPU to CPU)
- `jsepCreateKernel` and `jsepReleaseKernel`: a corresponding object
that maintained in JS to match lifecycle of Kernel in ORT
- `jsepRun`: OpKernel::Compute() should call into this
The abstraction above allows to tie as little as possible connections
and dependencies between C/C++ and TypeScript/JavaScript.
**Resource Management**
Lifecycle of tensor data and kernels are managed by ORT(C/C++) but the
implementation are left to JavaScript. JavaScript code are responsible
to implement the callbacks correctly.
For WebGPU, the GPU data is managed by JavaScript using a singleton map
(tensot_data_id => GPUBuffer). GPU pipeline is managed as singleton.
Shaders are managed using a singletonmap (shader_key => gpu_program),
while shader_key is generated by cache_key (OP specific, including
attributes) and input shapes.
**about data transfer**
`js::DataTransfer::CopyTensor` implemented to call either synchronized
or asynchronized copy callback, depending on the destination is GPU or
not. Emscripten's macro `EM_ASYNC_JS` is used to wrap the async function
to be called in the synchronized context.
**run kernel in JS**
Kernel class constructor calls once `jsepCreateKernel()` with an
optional per-kernel specific serialization to pass attributes into
JavaScript.
`Compute()` are implemented in a way that a metadata serialization is
performed in a base class and JavaScript code can access the data using
the Emscripten specific builtin macro `EM_ASM_*`.
**disabled features**
memory pattern is force disabled, because the WebGPU data is not
presented by a general memory model (a buffer can be represented by
offset + size).
concurrent run support is disabled. WebGPU is stateful and it also has
async function call. To support concurrent run will significantly
increase the complexity and we don't get any real benefit from it.
**prefer channels last**
JSEP prefers channels last and returns `DataLayout::NHWC` in method
`GetPreferredLayout()`. This will let the graph transformers to
preprocess the graph into a channels last form so that a more optimized
WebGPU shader can be used.
**Testing code**
It's impossible to test JSEP directly because JSEP itself does not
contain any kernel implementation. However, it has the kernel
registration which need to work together with the corresponding
JavaScript code. There are unit tests that run onnx models from
JavaScript API.
---------
Co-authored-by: Scott McKay <skottmckay@gmail.com>
### Description
optimize default session options parsing.
- do minimal property assignment to the passed in `options` object.
- modify default value of `enableCpuMemArena` and `enableMemPattern` to
`false`. We don't get benefits from enabling these 2 flags in web
assembly
### Description
This PR includes the following changes:
- upgrade js dependencies
- enable STRICT mode for web assembly build.
- corresponding fix for cmake-js upgrade
- corresponsing fix for linter upgrade
- upgrade default typescript compile option of:
- `moduleResolution`: from `node` to `node16`
- `target`: from `es2017` to `es2020`
- fix ESM module import in commonJS source file
## change explanation
### changes to onnxruntime_webassembly.cmake
`-s WASM=1` and `-s LLD_REPORT_UNDEFINED` in latest version is
by-default and deprecated.
### changes to onnxruntime_node.cmake
The npm package `cmake-js` updated its way to find file `node.lib`.
previously it downloads this file from Node.js public release channel,
and now it generates it from a definition file.
The node.js release channel does not contain a windows/arm64 version, so
previously cmake-js will fail to download `node.lib` for that platform.
this is why we made special handling to download the unofficial binary
to build. now this is no longer needed so we removed that from the cmake
file.
### changes to tsconfig.json
`node16` module resolution supports async import and `es2020` as target
supports top level await.
### Description
While browsing the sources I found several typos here and there.
I collected them to a single PR and fixed them.
Namely these typos are: operater, tranform, neccessary, trainig.
After fixing none of them was found anymore:
$ git grep "operater"
$ git grep "tranform"
$ git grep "neccessary"
$ git grep "trainig"
$
### Motivation and Context
Since some of the typos are in example notebooks and markdown files,
users can see them.
### Description
disable multi-thread test on Node.js in E2E test.
multi-thread test on Node.js in E2E test never worked, however the CI
does not pick up the error every time. So this became a flaky test case
which sometimes cause a build break.
Disable this test now and should enable it once it's get fixed.
### Description
* Support flag 'optimizedModelFilePath' in session options.
In Node.js, the model will be saved into filesystem just like its
behaviour on native platforms.
In browser, the new model is not saved to filesystem. the file path is
ignored. Instead, a new pop-up window will be launched in browser and
user can 'save' the file as onnx model.
* Add corresponding commandline args for the following session option
flags:
- optimizedModelFilePath
- graphOptimizationLevel
### Description
Data processing capabilities to ORT Web.
This PR will focus augmenting raw data to and from Tensors.
### Motivation and Context
Enabling different app building use cases to leverage ORT in a more
natural form.
Currently, the user needs to process the data and call Tensor
constructors - these util functions will provide a direct path to
generating ORT tensors.
Co-authored-by: shalvamist <shalva.mist@microsoft.com>
### Description
update versions of a few build dependencies for onnxruntime NPM
packages.
update nodejs version to v16.x in linux CI. v12 is too out-of-dated. see
[nodejs release
schedule](https://github.com/nodejs/release#release-schedule)
### Motivation and Context
- upgrade to latest webpack allows using of latest Node.js LTS version.
previous version of webpack does not work on Node.js v18 and it is fixed
in latest version
- upgrade to latest typescript, ts-loader and other dev deps to
accelerate the build and bundling.
- upgrade also helps to resolve security warnings that may be vulnerable
in out-of-dated version
Bumps [engine.io](https://github.com/socketio/engine.io) and
[socket.io](https://github.com/socketio/socket.io). These dependencies
needed to be updated together.
Updates `engine.io` from 6.1.3 to 6.2.1
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/socketio/engine.io/releases">engine.io's
releases</a>.</em></p>
<blockquote>
<h2>6.2.1</h2>
<p>⚠️ This release contains an important security fix
⚠️</p>
<p>A malicious client could send a specially crafted HTTP request,
triggering an uncaught exception and killing the Node.js process:</p>
<pre><code>Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:209:20)
Emitted 'error' event on Socket instance at:
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
errno: -104,
code: 'ECONNRESET',
syscall: 'read'
}
</code></pre>
<p>Please upgrade as soon as possible.</p>
<h3>Bug Fixes</h3>
<ul>
<li>catch errors when destroying invalid upgrades (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/658">#658</a>)
(<a
href="425e833ab1">425e833</a>)</li>
</ul>
<h2>6.2.0</h2>
<h2>Features</h2>
<ul>
<li>add the "maxPayload" field in the handshake details (<a
href="088dcb4dff">088dcb4</a>)</li>
</ul>
<p>So that clients in HTTP long-polling can decide how many packets they
have to send to stay under the maxHttpBufferSize
value.</p>
<p>This is a backward compatible change which should not mandate a new
major revision of the protocol (we stay in v4), as
we only add a field in the JSON-encoded handshake data:</p>
<pre><code>0{"sid":"lv_VI97HAXpY6yYWAAAC","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":5000,"maxPayload":1000000}
</code></pre>
<h4>Links</h4>
<ul>
<li>Diff: <a
href="https://github.com/socketio/engine.io/compare/6.1.3...6.2.0">https://github.com/socketio/engine.io/compare/6.1.3...6.2.0</a></li>
<li>Client release: <a
href="https://github.com/socketio/engine.io-client/releases/tag/6.2.0">6.2.0</a></li>
<li>ws version: <a
href="https://github.com/websockets/ws/releases/tag/8.2.3">~8.2.3</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/socketio/engine.io/blob/main/CHANGELOG.md">engine.io's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/socketio/engine.io/compare/6.2.0...6.2.1">6.2.1</a>
(2022-11-20)</h2>
<p>⚠️ This release contains an important security fix
⚠️</p>
<p>A malicious client could send a specially crafted HTTP request,
triggering an uncaught exception and killing the Node.js process:</p>
<pre><code>Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:209:20)
Emitted 'error' event on Socket instance at:
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
errno: -104,
code: 'ECONNRESET',
syscall: 'read'
}
</code></pre>
<p>Please upgrade as soon as possible.</p>
<h3>Bug Fixes</h3>
<ul>
<li>catch errors when destroying invalid upgrades (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/658">#658</a>)
(<a
href="425e833ab1">425e833</a>)</li>
</ul>
<h1><a
href="https://github.com/socketio/engine.io/compare/3.5.0...3.6.0">3.6.0</a>
(2022-06-06)</h1>
<h3>Bug Fixes</h3>
<ul>
<li>add extension in the package.json main entry (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/608">#608</a>)
(<a
href="3ad0567dbd">3ad0567</a>)</li>
<li>do not reset the ping timer after upgrade (<a
href="1f5d469986">1f5d469</a>),
closes <a
href="https://github-redirect.dependabot.com//github-redirect.dependabot.com/socketio/socket.io-client-swift/pull/1309/issues/issuecomment-768475704">socketio/socket.io-client-swift#1309</a></li>
</ul>
<h3>Features</h3>
<ul>
<li>decrease the default value of maxHttpBufferSize (<a
href="58e274c437">58e274c</a>)</li>
</ul>
<p>This change reduces the default value from 100 mb to a more sane 1
mb.</p>
<p>This helps protect the server against denial of service attacks by
malicious clients sending huge amounts of data.</p>
<p>See also: <a
href="https://github.com/advisories/GHSA-j4f2-536g-r55m">https://github.com/advisories/GHSA-j4f2-536g-r55m</a></p>
<ul>
<li>increase the default value of pingTimeout (<a
href="f55a79a28a">f55a79a</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="24b847be6a"><code>24b847b</code></a>
chore(release): 6.2.1</li>
<li><a
href="425e833ab1"><code>425e833</code></a>
fix: catch errors when destroying invalid upgrades (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/658">#658</a>)</li>
<li><a
href="99adb00ba1"><code>99adb00</code></a>
chore(deps): bump xmlhttprequest-ssl and engine.io-client in
/examples/latenc...</li>
<li><a
href="d196f6a6b7"><code>d196f6a</code></a>
chore(deps): bump minimatch from 3.0.4 to 3.1.2 (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/660">#660</a>)</li>
<li><a
href="7c1270f98c"><code>7c1270f</code></a>
chore(deps): bump nanoid from 3.1.25 to 3.3.1 (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/659">#659</a>)</li>
<li><a
href="535a01d889"><code>535a01d</code></a>
ci: add Node.js 18 in the test matrix</li>
<li><a
href="1b71a6f5cb"><code>1b71a6f</code></a>
docs: remove "Vanilla JS" highlight from README (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/656">#656</a>)</li>
<li><a
href="917d1d29e1"><code>917d1d2</code></a>
refactor: replace deprecated <code>String.prototype.substr()</code> (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/646">#646</a>)</li>
<li><a
href="020801ab8c"><code>020801a</code></a>
chore: add changelog for version 3.6.0</li>
<li><a
href="ed1d6f912c"><code>ed1d6f9</code></a>
test: make test script work on Windows (<a
href="https://github-redirect.dependabot.com/socketio/engine.io/issues/643">#643</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/socketio/engine.io/compare/6.1.3...6.2.1">compare
view</a></li>
</ul>
</details>
<br />
Updates `socket.io` from 4.4.1 to 4.5.3
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/socketio/socket.io/releases">socket.io's
releases</a>.</em></p>
<blockquote>
<h2>4.5.3</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>typings:</strong> accept an HTTP2 server in the constructor
(<a
href="d3d0a2d5be">d3d0a2d</a>)</li>
<li><strong>typings:</strong> apply types to
"io.timeout(...).emit()" calls (<a
href="e357daf585">e357daf</a>)</li>
</ul>
<h4>Links:</h4>
<ul>
<li>Diff: <a
href="https://github.com/socketio/socket.io/compare/4.5.2...4.5.3">https://github.com/socketio/socket.io/compare/4.5.2...4.5.3</a></li>
<li>Client release: <a
href="https://github.com/socketio/socket.io-client/releases/tag/4.5.3">4.5.3</a></li>
<li>engine.io version: <code>~6.2.0</code></li>
<li>ws version: <code>~8.2.3</code></li>
</ul>
<h2>4.5.2</h2>
<h3>Bug Fixes</h3>
<ul>
<li>prevent the socket from joining a room after disconnection (<a
href="18f3fdab12">18f3fda</a>)</li>
<li><strong>uws:</strong> prevent the server from crashing after upgrade
(<a
href="ba497ee3eb">ba497ee</a>)</li>
</ul>
<h4>Links:</h4>
<ul>
<li>Diff: <a
href="https://github.com/socketio/socket.io/compare/4.5.1...4.5.2">https://github.com/socketio/socket.io/compare/4.5.1...4.5.2</a></li>
<li>Client release: <a
href="https://github.com/socketio/socket.io-client/releases/tag/4.5.2">4.5.2</a></li>
<li>engine.io version: <code>~6.2.0</code></li>
<li>ws version: <code>~8.2.3</code></li>
</ul>
<h2>4.5.1</h2>
<h3>Bug Fixes</h3>
<ul>
<li>forward the local flag to the adapter when using fetchSockets() (<a
href="30430f0985">30430f0</a>)</li>
<li><strong>typings:</strong> add HTTPS server to accepted types (<a
href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4351">#4351</a>)
(<a
href="9b43c9167c">9b43c91</a>)</li>
</ul>
<h4>Links:</h4>
<ul>
<li>Diff: <a
href="https://github.com/socketio/socket.io/compare/4.5.0...4.5.1">https://github.com/socketio/socket.io/compare/4.5.0...4.5.1</a></li>
<li>Client release: <a
href="https://github.com/socketio/socket.io-client/releases/tag/4.5.1">4.5.1</a></li>
<li>engine.io version: <code>~6.2.0</code></li>
<li>ws version: <code>~8.2.3</code></li>
</ul>
<h2>4.5.0</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>typings:</strong> ensure compatibility with TypeScript 3.x
(<a
href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4259">#4259</a>)
(<a
href="02c87a8561">02c87a8</a>)</li>
</ul>
<h3>Features</h3>
<ul>
<li>add support for catch-all listeners for outgoing packets (<a
href="531104d332">531104d</a>)</li>
</ul>
<p>This is similar to <code>onAny()</code>, but for outgoing
packets.</p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/socketio/socket.io/blob/main/CHANGELOG.md">socket.io's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/socketio/socket.io/compare/4.5.2...4.5.3">4.5.3</a>
(2022-10-15)</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>typings:</strong> accept an HTTP2 server in the constructor
(<a
href="d3d0a2d5be">d3d0a2d</a>)</li>
<li><strong>typings:</strong> apply types to
"io.timeout(...).emit()" calls (<a
href="e357daf585">e357daf</a>)</li>
</ul>
<h2><a
href="https://github.com/socketio/socket.io/compare/4.5.1...4.5.2">4.5.2</a>
(2022-09-02)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>prevent the socket from joining a room after disconnection (<a
href="18f3fdab12">18f3fda</a>)</li>
<li><strong>uws:</strong> prevent the server from crashing after upgrade
(<a
href="ba497ee3eb">ba497ee</a>)</li>
</ul>
<h1><a
href="https://github.com/socketio/socket.io/compare/2.4.1...2.5.0">2.5.0</a>
(2022-06-26)</h1>
<h3>Bug Fixes</h3>
<ul>
<li>fix race condition in dynamic namespaces (<a
href="05e1278cfa">05e1278</a>)</li>
<li>ignore packet received after disconnection (<a
href="22d4bdf00d">22d4bdf</a>)</li>
<li>only set 'connected' to true after middleware execution (<a
href="226cc16165">226cc16</a>)</li>
<li>prevent the socket from joining a room after disconnection (<a
href="f223178eb6">f223178</a>)</li>
</ul>
<h2><a
href="https://github.com/socketio/socket.io/compare/4.5.0...4.5.1">4.5.1</a>
(2022-05-17)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>forward the local flag to the adapter when using fetchSockets() (<a
href="30430f0985">30430f0</a>)</li>
<li><strong>typings:</strong> add HTTPS server to accepted types (<a
href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4351">#4351</a>)
(<a
href="9b43c9167c">9b43c91</a>)</li>
</ul>
<h1><a
href="https://github.com/socketio/socket.io/compare/4.4.1...4.5.0">4.5.0</a>
(2022-04-23)</h1>
<h3>Bug Fixes</h3>
<ul>
<li><strong>typings:</strong> ensure compatibility with TypeScript 3.x
(<a
href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4259">#4259</a>)
(<a
href="02c87a8561">02c87a8</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="945c84be47"><code>945c84b</code></a>
chore(release): 4.5.3</li>
<li><a
href="d3d0a2d5be"><code>d3d0a2d</code></a>
fix(typings): accept an HTTP2 server in the constructor</li>
<li><a
href="19b225b0c8"><code>19b225b</code></a>
docs(examples): update dependencies of the basic CRUD example</li>
<li><a
href="8fae95dd18"><code>8fae95d</code></a>
docs: add jsdoc for each public method</li>
<li><a
href="e6f6b906db"><code>e6f6b90</code></a>
docs: add deprecation notice for the allSockets() method</li>
<li><a
href="596eb88af7"><code>596eb88</code></a>
ci: upgrade to actions/checkout@3 and actions/setup-node@3</li>
<li><a
href="e357daf585"><code>e357daf</code></a>
fix(typings): apply types to "io.timeout(...).emit()"
calls</li>
<li><a
href="10fa4a2690"><code>10fa4a2</code></a>
refactor: add list of possible disconnection reasons</li>
<li><a
href="8be95b3bd3"><code>8be95b3</code></a>
chore(release): 4.5.2</li>
<li><a
href="ba497ee3eb"><code>ba497ee</code></a>
fix(uws): prevent the server from crashing after upgrade</li>
<li>Additional commits viewable in <a
href="https://github.com/socketio/socket.io/compare/4.4.1...4.5.3">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/onnxruntime/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.0.4 to
3.0.5.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="707e1b231d"><code>707e1b2</code></a>
3.0.5</li>
<li><a
href="a8763f4388"><code>a8763f4</code></a>
Improve redos protection, add many tests</li>
<li><a
href="bafa295617"><code>bafa295</code></a>
Use master branch for travis badge</li>
<li><a
href="013d64dc24"><code>013d64d</code></a>
update travis</li>
<li>See full diff in <a
href="https://github.com/isaacs/minimatch/compare/v3.0.4...v3.0.5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/onnxruntime/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
**Description**:
1. add pytorch_half_pixel interpolation mode in resize-packed.ts
Changes: add the following case in createPackedResizeProgramInfo
function:
```
case 'pytorch_half_pixel':
getSourceFracIndex = `
vec4 getSourceFracIndex(ivec4 coords) {
vec4 fcoords = vec4(coords);
return vec4(
${outputWidth}.0 > 1.0 ? (fcoords.x + 0.5) / scaleWHWH.x - 0.5 : 0.0,
${outputHeight}.0 > 1.0 ? (fcoords.y + 0.5) / scaleWHWH.y - 0.5 : 0.0,
${outputWidth}.0 > 1.0 ? (fcoords.z + 0.5) / scaleWHWH.z - 0.5 : 0.0,
${outputHeight}.0 > 1.0 ? (fcoords.w + 0.5) / scaleWHWH.w - 0.5 : 0.0
);
}
`;
break;
```
2. fix "unrecognized input '' for node: Resize_$num" error when inputs
like [input_tensor, None, scale_factor] (roiInput not given) are fed
into the resize layer.
Changes: change in input handling logic in upsample.ts & node scanning
logic in graph.ts
**Motivation and Context**
Before this fix, we aren't able to use webGL backend when the neural
network contains pytorch resize layers. This fix adds
'pytorch_half_pixel' interpolation mode support and makes it possible to
use webGL backend for more kind of computer vision networks.
This commit solves:
#10430
Co-authored-by: neo <neo@icode-lab.com>
Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
### Description
Bumping up version number to 1.14.0
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
**Description**: This PR adds support for "XNNPACK EP" in ORTWeb and
changes the behavior of how ORTWeb deals with "backends", or "EPs" in
API.
**Background**: Term "backend" is introduced in ONNX.js to representing
a TypeScript type which implements a "backend" interface, which is a
similar but different concept to ORT's EP (execution provider). There
was 3 backends in ONNX.js: "cpu", "wasm" and "webgl".
When ORT Web is launched, the concept is derived to help users to
integrate smoothly. Technically, when "wasm" backend is used, users need
to also specify "EP" in the session options. Considering it may get
complicated and confused for users to figure out the difference between
"backend" and "EP", the JS API hide the "backend" concept and made a
mapping between names, backends and EPs:
"webgl" (Name) <==> "onnxjsBackend" (Backend)
"wasm" (Name) <==> "wasmBackend" (Backend) <==> "CPU" (EP)
**Details**:
The following changes are applied in this PR:
1. allow multi-registration for backends using the same name. This is
for use scenarios where both "onnxruntime-node" and "onnxruntime-web"
are consumed in a Node.js App ( so "cpu" will be registered twice in
this scenario. )
2. re-assign priority values to backends. I give 100 as base to "cpu"
for node and react_native, and 10 as base to "cpu" in web.
3. add "cpu", "xnnpack" as new names of backends.
4. update onnxruntime wasm exported functions to support EP
registration.
5. update implementations in ort web to handle execution providers in
session options.
6. add '--use_xnnpack' as default build flag for ort-web
Added a check for tensor validation on the input - this change fixes the
quiet abort WASM takes when processing a non tensor data in
"OrtGetTensorData"
**Motivation and Context**
At the current status when we try to process non-tensor data through
OrtGetTensorData and exception is thrown which results in a quiet abort
from WASM (assuming WASM was built without exception handling).
I added a check in the C API to catch this case and output a meaningful
message to the user
[example_error_github_12622.zip](https://github.com/microsoft/onnxruntime/files/9464328/example_error_github_12622.zip)
using ensureSymlinkSync might have issues with permissions when using 'dir' - changed to 'junction' to avoid this.
If the folder generation fails it will cause the test to fails as well.
* consume onnx test data from github
* ensure tests
* update script and allow opset specification
* fix python format
* fix python format
* consume new filter format
* fix linting error
* skip browserstack test at release pipeline
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* pool name as a parameter to run at lotus
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* Update web-ci-pipeline.yml for Azure Pipelines
* create a packaging pipeline for web
* Update web-packaging-pipeline.yml for Azure Pipelines
* make web-ci-pipeline as a template
* make web-ci-pipeline as a template
* make web-ci-pipeline as a template
* make web-ci-pipeline as a template
* change a paramter name checking a pipeline
* make a pool name changable for react native pipeline
* disable code sign validation for react native
* fix react native package.json publish
* fix indentation
* remove unnecessary comment
* test onnxruntime-common package publish
* ts and js files use lf as eol for windows
* use Linux style of ending line break
* change newLine at only tsconfig.json
* restore a commented code
* fix git restore directory for npm packaging
* fix a typo
* force eol to lf on windows for js directory in CI
* es2017 by default for ort-common
* add visualizer and define plugin
* es2017 for ort-web. also add build target for es5
* add multiple reduced size build for ort-web
* resolve comments, add e2e tests and add docs
* add p50 in test
* Support FusedConv in WebGL
* resolve comments
* add a comment for longToNumber change
Co-authored-by: Yulong Wang <yulongw@microsoft.com>
* add p50 in test
* support opset-13 of softmax
* update a operators.md
* resolve comments
* fix lint and format
Co-authored-by: Yulong Wang <yulongw@microsoft.com>
* Support opset-13 for squeeze, unsqueeze, maxpool, pad, cast, clip
* merge master and update a operators.md
* resolve comment. revise pool and cast kernel implementation.
* skip fusion when clip min and max is not in initializer
* Integrate BrowserStack with ONNX Runtime Web CI pipeline
* Change to Linux command for BrowserStack CI
* Set preferTriggeringPipeline as true
* Fix a commit fetching script
* Remove wasm binary download from the latest build
* Use release build of WebAssembly
* Disable check-out of commit for testing
* Use commit of WebAssembly build CI pipeline
* Need to issue two PRs to prevent build failure
* updates for picking pnnx commit
* add tests filter to c# tests
* plus test fixes
* fix versioning for contrib ops
* fix tests
* test filter for optional ops
* more versioning related updates
* fix test
* fix layernorm spec
* more updates
* update docs
* add more test filters
* more filters
* update binary size threshold
* update docs
* plus more fixes
* updates per review
* update to release commit
* add filters for optional type tests
* plus updates
* [js/web] Add wasm SIMD backend to onnxruntime-web
* Import SIMD wasm artifacts enabled by PR #7839
* Detect SIMD capability of web engine
* Use SIMD wasm backend in both single-thread and multi-thread cases
* update optimized SIMD loading from ort web
* code lint and format
* fix WasmFileName in CI
* replace deprecated wasm SIMD functions
* fix unittest for simd
* optimize CI pipeline to merge build matrix
* make clean build for each config
* fix simd wasm to enable it.
* update script/pull-prebuilt-wasm-artifacts.ts
Co-authored-by: Yulong Wang <yulongw@microsoft.com>
Co-authored-by: Lei Zhang <zhang.huanning@hotmail.com>
* [js/web] support string tensor for wasm backend
* disable v9/test_cast_STRING_to_FLOAT: test data is wrong
* add non-string check
* Update session-handler.ts
* Update session-handler.ts
* fixed bugs in packed mode and enable pack mode tests in ci
* removed unnecessary space
* pr comments
* pr comments
* disable an average pool test
* try disabling another avg pool
* disable more avg pool tests
* disable maxpool tests
* migrated changes to support running super resolution model using ortweb
* reverted benchmarking tool related changes which will be in a separate pr
* added kernel tests to op and node tests
* minor change to the order of variables
* added one more unit test for packed matmul
* add all session options and run options in C API except AddInitializer and AddFreeDimensionOverride
* remove unnecessary comment
* change extra session and run options to object notation
* resolve comments
* use an optional chaining for options
* resolve comments
* support session options and run options. use onnxruntime c api.
* fix lint errors
* add an error code on throwing an exception
* resolve comments. change remaining C++ APIs to C API
