The test limits GPU's running memory requirements to 20MB. It might be
enough in the past, but it seems not enough now when we upgrade CUDA to
a newer version or add more kernels/graph transformers to our code.
Therefore we need to increase it. Our test log shows sometimes the model
needs 128MB memory. So I set the limit to 256MB.
### Description
Reduce a number of auxillary objects created to reduce GC pressure.
Eliminate GCHandle type of memory pinning in most of the places.
Improve string marshalling by allocating unmanaged memory that does not
require pinning. Change native methods from `IntPtr` to `byte[]`
(marshalling pinning is more efficient).
Allocate input/output UTF-8 names in unmanaged heap for the lifetime of
InferenceSession. So we do not keep converting them and pinning on every
Run.
Introduce a new native API that allows to allocate and convert/copy
strings directly into a native tensor.
The PR delivers around 50% latency improvements and less GC pauses.
Inspired by: https://github.com/microsoft/onnxruntime/pull/15520
### Motivation and Context
Client experience GC pressure and performance degradation when dealing
with string tensors.
Co-Authored-By: @tannergooding
### Description
Run clang-format in CI. Formatted all c/c++, objective-c/c++ files.
Excluded
```
'onnxruntime/core/mlas/**',
'onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/**',
```
because they contain assembly or is data heavy
### Motivation and Context
Coding style consistency
### Description
Implement Optional Type metadata support in the library.
Implement optional support in C# API along with metadata.
Implement Sequence, Map, Optional test data support
and test execution.
Prune tests and provide more details for failing tests in C# code.
Note, this PR does not enable running onnx test models in C++.
### Motivation and Context
Opset18 optional type support.
WindowsAI build failing due to deprecated .NET5 SDK missing in build
image
.NET5 was deprecated last year, and recently the build machine images
have been updated to not include this SDK.
Unblock failing builds by force insalling .NET5 SDK as part of the build
pipeline.
### Description
Add sbyte specific test case support.
### Motivation and Context
C# Test Data loading code and comparators are missing sbyte (Int8)
specializations.
This fails a test
### Description
<!-- Describe your changes. -->
I fixed some broken links in the C API documentation, but then did a
quick pass over all of the links I could find and then fixed those.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
I got some 404's when exploring the documentation and wanted to fix it.
### Description
Reenable disabled opset18 tests
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
Re-work `OrtApi::GetAvailableProviders` in a way that the data is
returned in a single allocation.
Fix exception safety issues and fix `Release` function.
Remove warning suppressions.
Fix exception safety issue in C++ API.
Fix exception safety issue in C# API.
Move EP name length enforcement to the implementation.
### Motivation and Context
The original motivation comes from
https://github.com/microsoft/onnxruntime/issues/14378.
However, the API is already implemented.
Cc: @prabhat00155
### Description
<!-- Describe your changes. -->
Remove exclusions for ONNX model tests that now pass due to kernels
being implemented.
Update ONNX update doc to point to correct location for tests.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Run as many tests as possible.
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
### Description
To Implement Resize 18.
This PR depends on https://github.com/microsoft/onnxruntime/pull/13765.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
<!-- Describe your changes. -->
Skip tests for opset18 models that we haven't implemented kernels for
yet.
Slice was checked in today so those failures should go away.
Resize: #13890 (all resize failures are fixed by this PR as confirmed in
output
[here](https://dev.azure.com/aiinfra/530acbc4-21bc-487d-8cd8-348ff451d2ff/_apis/build/builds/264725/logs/729))
Col2Im: #12311
ScatterND and ScatterElement: #14224
Pad (should also fix CenterCropPad failures): #14219 Bitwise ops: #14197
Optional: Unknown if we're intending to support this in 1.14
Not sure about SoftPlus as that is failing due to `Could not find an
implementation for Exp(1)`. ORT supports Exp from opset 6 and on, and it
seems incorrect for the test model created for opset 18 to be using a
version of Exp that is so old. Would have expected it to use the latest
- Exp(13). @liqunfu is this something that requires a fix to the ONNX
model?
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Fix pipeline
### Description
Enable creating dedicated build for on device training. With this PR we
can build a lean binary for on device training using flag
--enable_training_apis. This binary includes only the essentials like
training ops, optimizers etc and NOT features like Aten fallback,
strided tensors, gradient builders etc . This binary also removes all
the deprecated components like training::TrainingSession and OrtTrainer
etc
### Motivation and Context
This enables our partners to create a lean binary for on device
training.
### Description
1. Renames all references of on device training to training apis. This
is to keep the naming general. Nothing really prevents us from using the
same apis on servers\non-edge devices.
2. Update ENABLE_TRAINING option: With this PR when this option is
enabled, training apis and torch interop is also enabled.
3. Refactoring for onnxruntime_ENABLE_TRAINING_TORCH_INTEROP option:
- Removed user facing option
- Setting onnxruntime_ENABLE_TRAINING_TORCH_INTEROP to ON when
onnxruntime_ENABLE_TRAINING is ON as we always build with torch interop.
Once this PR is merged when --enable_training is selected we will do a
"FULL Build" for training (with all the training entry points and
features).
Training entry points include:
1. ORTModule
2. Training APIs
Features include:
1. ATen Fallback
2. All Training OPs includes communication and collectives
3. Strided Tensor Support
4. Python Op (torch interop)
5. ONNXBlock (Front end tools for training artifacts prep when using
trianing apis)
### Motivation and Context
Intention is to simply the options for building training enabled builds.
This is part of the larger work item to create dedicated build for
learning on the edge scenarios with just training apis enabled.
### Description
<!-- Describe your changes. -->
Add ability to set RunOptions config entries. Largely a cut-and-paste of
the existing code for setting SessionOptions config entries.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
#13936
### Description
This PR fixes some typos in the training apis.
We need to add more tests and make sure they are all run on the CIs to
capture such issues. These changes are out of scope of this PR.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Ashwini Khade <askhade@microsoft.com@orttrainingdev8.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
[//]: # (dependabot-start)
⚠️ **Dependabot is rebasing this PR** ⚠️
Rebasing might not happen immediately, so don't worry if this takes some
time.
Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.
---
[//]: # (dependabot-end)
Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from
13.0.1 to 13.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/JamesNK/Newtonsoft.Json/releases">Newtonsoft.Json's
releases</a>.</em></p>
<blockquote>
<h2>13.0.2</h2>
<ul>
<li>New feature - Add support for DateOnly and TimeOnly</li>
<li>New feature - Add UnixDateTimeConverter.AllowPreEpoch property</li>
<li>New feature - Add copy constructor to JsonSerializerSettings</li>
<li>New feature - Add JsonCloneSettings with property to disable copying
annotations</li>
<li>Change - Add nullable annotation to JToken.ToObject(Type,
JsonSerializer)</li>
<li>Change - Reduced allocations by reusing boxed values</li>
<li>Fix - Fixed MaxDepth when used with ToObject inside of a
JsonConverter</li>
<li>Fix - Fixed deserializing mismatched JToken types in properties</li>
<li>Fix - Fixed merging enumerable content and validate content</li>
<li>Fix - Fixed using $type with arrays of more than two dimensions</li>
<li>Fix - Fixed rare race condition in name table when deserializing on
device with ARM processors</li>
<li>Fix - Fixed deserializing via constructor with ignored base type
properties</li>
<li>Fix - Fixed MaxDepth not being used with ISerializable
deserialization</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4fba53a324"><code>4fba53a</code></a>
Remove prerelease for 13.0.2</li>
<li><a
href="b15df4b50d"><code>b15df4b</code></a>
Add missing headers</li>
<li><a
href="789bfd3bbc"><code>789bfd3</code></a>
Update to 13.0.2-beta3</li>
<li><a
href="b13717a1c1"><code>b13717a</code></a>
Add JsonCloneSettings to disable copy annotations (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2757">#2757</a>)</li>
<li><a
href="d0a328e8a4"><code>d0a328e</code></a>
Fix MaxDepth not being used with ISerializable deserialization (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2736">#2736</a>)</li>
<li><a
href="aae9284e20"><code>aae9284</code></a>
Update SDK</li>
<li><a
href="bd989708b1"><code>bd98970</code></a>
Update to 13.0.2-beta2</li>
<li><a
href="4dc9af66e0"><code>4dc9af6</code></a>
Add roll forward to global.json (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2726">#2726</a>)</li>
<li><a
href="b8f4ef0f98"><code>b8f4ef0</code></a>
Fixing misspelling (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2698">#2698</a>)</li>
<li><a
href="cb9eed9666"><code>cb9eed9</code></a>
Fix deserializing via constructor with ignored base type properties (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2711">#2711</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/JamesNK/Newtonsoft.Json/compare/13.0.1...13.0.2">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/onnxruntime/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[//]: # (dependabot-start)
⚠️ **Dependabot is rebasing this PR** ⚠️
Rebasing might not happen immediately, so don't worry if this takes some
time.
Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.
---
[//]: # (dependabot-end)
Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from
13.0.1 to 13.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/JamesNK/Newtonsoft.Json/releases">Newtonsoft.Json's
releases</a>.</em></p>
<blockquote>
<h2>13.0.2</h2>
<ul>
<li>New feature - Add support for DateOnly and TimeOnly</li>
<li>New feature - Add UnixDateTimeConverter.AllowPreEpoch property</li>
<li>New feature - Add copy constructor to JsonSerializerSettings</li>
<li>New feature - Add JsonCloneSettings with property to disable copying
annotations</li>
<li>Change - Add nullable annotation to JToken.ToObject(Type,
JsonSerializer)</li>
<li>Change - Reduced allocations by reusing boxed values</li>
<li>Fix - Fixed MaxDepth when used with ToObject inside of a
JsonConverter</li>
<li>Fix - Fixed deserializing mismatched JToken types in properties</li>
<li>Fix - Fixed merging enumerable content and validate content</li>
<li>Fix - Fixed using $type with arrays of more than two dimensions</li>
<li>Fix - Fixed rare race condition in name table when deserializing on
device with ARM processors</li>
<li>Fix - Fixed deserializing via constructor with ignored base type
properties</li>
<li>Fix - Fixed MaxDepth not being used with ISerializable
deserialization</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4fba53a324"><code>4fba53a</code></a>
Remove prerelease for 13.0.2</li>
<li><a
href="b15df4b50d"><code>b15df4b</code></a>
Add missing headers</li>
<li><a
href="789bfd3bbc"><code>789bfd3</code></a>
Update to 13.0.2-beta3</li>
<li><a
href="b13717a1c1"><code>b13717a</code></a>
Add JsonCloneSettings to disable copy annotations (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2757">#2757</a>)</li>
<li><a
href="d0a328e8a4"><code>d0a328e</code></a>
Fix MaxDepth not being used with ISerializable deserialization (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2736">#2736</a>)</li>
<li><a
href="aae9284e20"><code>aae9284</code></a>
Update SDK</li>
<li><a
href="bd989708b1"><code>bd98970</code></a>
Update to 13.0.2-beta2</li>
<li><a
href="4dc9af66e0"><code>4dc9af6</code></a>
Add roll forward to global.json (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2726">#2726</a>)</li>
<li><a
href="b8f4ef0f98"><code>b8f4ef0</code></a>
Fixing misspelling (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2698">#2698</a>)</li>
<li><a
href="cb9eed9666"><code>cb9eed9</code></a>
Fix deserializing via constructor with ignored base type properties (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2711">#2711</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/JamesNK/Newtonsoft.Json/compare/13.0.1...13.0.2">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/onnxruntime/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from
13.0.1 to 13.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/JamesNK/Newtonsoft.Json/releases">Newtonsoft.Json's
releases</a>.</em></p>
<blockquote>
<h2>13.0.2</h2>
<ul>
<li>New feature - Add support for DateOnly and TimeOnly</li>
<li>New feature - Add UnixDateTimeConverter.AllowPreEpoch property</li>
<li>New feature - Add copy constructor to JsonSerializerSettings</li>
<li>New feature - Add JsonCloneSettings with property to disable copying
annotations</li>
<li>Change - Add nullable annotation to JToken.ToObject(Type,
JsonSerializer)</li>
<li>Change - Reduced allocations by reusing boxed values</li>
<li>Fix - Fixed MaxDepth when used with ToObject inside of a
JsonConverter</li>
<li>Fix - Fixed deserializing mismatched JToken types in properties</li>
<li>Fix - Fixed merging enumerable content and validate content</li>
<li>Fix - Fixed using $type with arrays of more than two dimensions</li>
<li>Fix - Fixed rare race condition in name table when deserializing on
device with ARM processors</li>
<li>Fix - Fixed deserializing via constructor with ignored base type
properties</li>
<li>Fix - Fixed MaxDepth not being used with ISerializable
deserialization</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4fba53a324"><code>4fba53a</code></a>
Remove prerelease for 13.0.2</li>
<li><a
href="b15df4b50d"><code>b15df4b</code></a>
Add missing headers</li>
<li><a
href="789bfd3bbc"><code>789bfd3</code></a>
Update to 13.0.2-beta3</li>
<li><a
href="b13717a1c1"><code>b13717a</code></a>
Add JsonCloneSettings to disable copy annotations (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2757">#2757</a>)</li>
<li><a
href="d0a328e8a4"><code>d0a328e</code></a>
Fix MaxDepth not being used with ISerializable deserialization (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2736">#2736</a>)</li>
<li><a
href="aae9284e20"><code>aae9284</code></a>
Update SDK</li>
<li><a
href="bd989708b1"><code>bd98970</code></a>
Update to 13.0.2-beta2</li>
<li><a
href="4dc9af66e0"><code>4dc9af6</code></a>
Add roll forward to global.json (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2726">#2726</a>)</li>
<li><a
href="b8f4ef0f98"><code>b8f4ef0</code></a>
Fixing misspelling (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2698">#2698</a>)</li>
<li><a
href="cb9eed9666"><code>cb9eed9</code></a>
Fix deserializing via constructor with ignored base type properties (<a
href="https://github-redirect.dependabot.com/JamesNK/Newtonsoft.Json/issues/2711">#2711</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/JamesNK/Newtonsoft.Json/compare/13.0.1...13.0.2">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/onnxruntime/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This reverts commit 87d5703b14.
### Motivation and Context
There was a bug in Linux CUDA installation.
The OS image is updated.
The TestCUDAProviderOptions could be reenabled.
### Description
This PR enables building nuget packages locally for on device training
using --build_nuget arg.
This PR also enables the C# bindings by default in the managed package.
If a user triggers any training apis when the native binary is not built
for training, an exception with message "Training is disabled in the
current build. Please build ONNXRuntime from source with the build flags
enable_training and enable_training_on_device. " is thrown.
Build command for creating nuget packes for on device training:
build.bat --enable_training --enable_training_on_device --build_nuget
2 Nuget packages are built
1. Microsoft.ML.OnnxRuntime.Managed
2. Microsoft.ML.OnnxRuntime.Training OR
Microsoft.ML.OnnxRuntime.Training.Gpu
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
Skip quantized model C# tests on GPU too.
### Motivation and Context
It looks the current test result isn't reasonable.
https://github.com/onnx/models/issues/581
Once we update the image, the quantized model [test data will be
generated with
VNNI](ba629906dd),
the CI would be broken.
### Description
<!-- Describe your changes. -->
Skip the test with --filter in runtest.sh
### Motivation and Context
Recently, the Zip-Nuget-Java-Nodejs Packaging Pipeline always failed in
Nuget_Test_Linux_GPU.
To unblock the packaging workflow, skip the test in Nuget_Test_Linux_GPU
temporally.
the exception message is below.
```
[xUnit.net 00:07:26.28] TestCUDAProviderOptions [FAIL]
Failed TestCUDAProviderOptions [1 m 19 s]
Error Message:
Microsoft.ML.OnnxRuntime.OnnxRuntimeException : [ErrorCode:RuntimeException] Non-zero status code returned while running FusedConv node. Name:'' Status Message: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:342 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool) Available memory of 11416064 is smaller than requested bytes of 134217728
Stack Trace:
at Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus)
at Microsoft.ML.OnnxRuntime.InferenceSession.RunImpl(RunOptions options, IntPtr[] inputNames, IntPtr[] inputValues, IntPtr[] outputNames, DisposableList`1 cleanupList)
at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection`1 inputs, IReadOnlyCollection`1 outputNames, RunOptions options)
at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection`1 inputs, IReadOnlyCollection`1 outputNames)
at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection`1 inputs)
at Microsoft.ML.OnnxRuntime.Tests.CUDATest.TestCUDAProviderOptions() in /mnt/vss/_work/1/s/csharp/test/Microsoft.ML.OnnxRuntime.Tests.NetCoreApp/InferenceTest.netcore.cs:line 93
Failed! - Failed: 1, Passed: 0, Skipped: 0, Total: 1, Duration: < 1 ms - /mnt/vss/_work/1/s/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/bin/Debug/netcoreapp3.1/Microsoft.ML.OnnxRuntime.EndToEndTests.dll (netcoreapp3.1)
Done executing task "Microsoft.TestPlatform.Build.Tasks.VSTestTask" -- FAILED.
1>Done building target "VSTest" in project "Microsoft.ML.OnnxRuntime.EndToEndTests.csproj" -- FAILED.
1>Done Building Project "/mnt/vss/_work/1/s/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests/Microsoft.ML.OnnxRuntime.EndToEndTests.csproj" (VSTest target(s)) -- FAILED.
```
Patch Protobuf and ONNX's cmake files and enforce BinSkim check.
This PR has overlap with #13523 . I would prefer to get this one merged
first so that we can finished the BinSkim work, and I try to make this
PR as small as possible.
### Description
* Add getter/setter to access and update C# OrtEnv log level
* Add C API about updating ort env with custom log level to support the
setter above (Following [pybind
implementation](952c99304a/onnxruntime/python/onnxruntime_pybind_state.cc (L923-L924)))
* Add test case to verify getter & setter
### Motivation and Context
* For C++/Python, the log level can be adjusted via OrtEnv, and this
feature is missing in C# binding
### Description
Training C# bindings (ReleaseTrainingSession and ReleaseCheckpointState)
broke after an API order change in Training C API. This PR fixes this
issue.
### Motivation and Context
Bug Fix for Training C# bindings
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
* Add asm statement to model.mm to force linker to link against CoreML.Framework.
Update targets.xml as per Rolf's suggestions
* Remove explicit numpy version from macos build. We don't specify it for other CIs and the version specified doesn't have a pre-built 3.10 wheel. This leads to the CI attempting to build numpy which fails.
* Fix SAME_UPPER/SAME_LOWER (auto_pad attribute) in ConvTranspose
* Bump ONNX 1.10.2 globally
* load ONNX_VERSION from VERSION_NUMBER
* /
* revert deprecate warning in ORT 1.12
* add a comment about why removing cntk_simple_seg
* correct the implem in DML as well
Add csharp\sample\InferenceSample\Microsoft.ML.OnnxRuntime.InferenceSample.Maui so we have an equivalent setup for MAUI as for the other platforms.
This provides a setup to do some basic local testing of using an InferenceSession in a MAUI app.
The generated bindings causes C# build errors that require workaround code. Disabling generation should avoid the need for any workarounds.
As the user has the C# ORT package with the C# to C bindings there's no need for binding generation that calls the ORT Java API (which is C# -> Java ->C).
* Update C# runtest.sh for opset 17
Should have been part of https://github.com/microsoft/onnxruntime/pull/11924
* get appropriate opset version from onnx doc
* use absolute rather than relative path
* fix typo in var name
* Add net6 targets.
Remove maccatalyst as we don't have a native build targetting that.
* Set platform in macos targets
* Add targetFramework entries
* Move NativeLib.DllName definition and set using preprocessor values for simplicity. Couldn't get it to build with the preprocessor based setup when it was in a separate file.
Update the nuspec generation to set platform version for .net6 targets. TODO: Validate versions. I copied them from the managed nuget package the packaging pipeline generated prior to adding targets. Possibly w could/should lower some of the versions.
Hopefully the need to specify a version goes away when the release version of VS2022 supports .net6.
* Try android 31.1 as https://github.com/actions/virtual-environments/blob/main/images/win/Windows2022-Readme.md suggests that should be available on the CI machines
* Fix patch version mismatch
Add some extra debug info in case it helps
* Debug nuget location in CI
* Add workspace entry back in
* Add steps
* One more attempt with hardcoded nuget.exe path and original android31.0 version
* Better fix - found explicit nuget download and updated version there.
* flake8 fixes
* Fix black complaints.
* Exit Microsoft_ML_OnnxRuntime_CheckPrerequisites for net6 iOS.
* Removed outdated comment
* Add .net6 support to the C# nuget package.
Currently requires jumping through a lot of hoops due to .net 6 only being supported in the preview release of VS 2022.
Build existing targets using msbuild.
Add .net6 targets and build using dotnet.
Create nuget package with combined targets.
A few misc automated changes from VS to spacing and adding a couple of properties.
* Rework the EP factory creation setup so we're not cut-and-pasting function declarations in multiple places.
Convert append EP for SNPE to be generic, and also use for XNNPACK.
Add XNNPACK to C# API
* Don't need stub for MIGraphX as it's using provider bridge.
* Remove old 'create' functions that aren't applicable now that the EPs are built as separate libraries.
* Only use EPs that require the layout transform if the opset is supported by the layout transformer.
* Update wasm registration of xnnpack.
* Initiate Ort SNPE EP
* fix snpe ep windows build which is caused by the utility method (ToUTF8String) name change on master
* correct the source path for libonnxruntime.so while building for andorid package
* add AdditionalDependencies for amr64
* On MS-Windows, the patchfile must be a text file, i.e. CR-LF must be used as line endings. A file with LF may give the error: "Assertion failed, hunk, file patch.c, line 343," unless the option '--binary' is given.
* fix build failure if snpe is not enabled
* update doc for contrib op
* separate out snpe ep settings to onnxruntime_snpe_provider.cmake
* renaming according review comments
* update according review comments
Description: Format all python files under onnxruntime with black and isort.
After checking in, we can use .git-blame-ignore-revs to ignore the formatting PR in git blame.
#11315, #11316
* Include onnxruntime binary when not using pacakge referene or uap app.
* Remove the lib\uap10.0 build from the nuget package - causing conflicts
* Add UWP test
* remove build files
* remove local change
* reset mimalloc and onnx-tensorrt
* change username to Microsoft
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* squashed commit for standalone tvm execution provider
* critical fix for correct python build with stvm ep
* get tuning log file from ep options. It has priority over AUTOTVM_TUNING_LOG
* updates and fixes
* update parsing of stvm provider options
* add support of external data for onnx model
* add conditional dump of subgraphs
* remove unused code
* get input tensor shapes through provider options. get output shapes for fixed input ones by TVM API
* support AUTO_TVM tuning log file inside ORT. Selector for Ansor and Auto_TVM is provider option (tuning_type)
* add fp16
* add functionality of conversion of model layout to NHWC if need. Necessary parameter was added to STVM provider options
* fix license text in header. fix log format
* small fixes
* fix issues from flake8
* remove model proto construction from GetCapability
* reserve memory for vector of DLTensors
* add simple tutorial for STVM EP
* STVM docs
* jroesch/tvm -> apache/tvm
* remove dead code, unneccessary logs and comments
* fix in readme
* improve tutorial notebook
* tvm update
* update STVM_EP.md
* fix default value
* update STVM_EP.md
* some TODOs for the future development
* shorten long lines
* add hyperlink to STVM_EP.md
* fix Linux CI error
* fix error in csharp test
Co-authored-by: Jared Roesch <jroesch@octoml.ai>
Co-authored-by: Valery Chernov <valery.chernov@deelvin.com>
Co-authored-by: KJlaccHoeUM9l <wotpricol@mail.ru>
- Only set them as targets for the ORT nuget package
- Use OrtPackageId as the condition for inclusion, if installed
- need to do the nuget restore via msbuild so that this property is set correctly
- Add desktop-only version of the C# sln as there is no way to exclude the mobile specific csproj's from an sln
- use this when applicable if someone is running build.py with the `--build_nuget` flag
Other
- remove attempt to include symbols in the nuget package as nuget doesn't support symbols in native packages
- update build.py to use `nuget` and not a windows specific path and filename for a linux build with `--build_nuget`
Add Xamarin support to the ORT nuget packages.
- Update C# code to support Xamarin builds for iOS and Android
- refactor some things to split out common code
- include iOS and Android ORT native shared library in native nuget package
* re-hipify all rocm EP sources
* fix all other files affected by re-hipify
* add cuda_provider_factory.h to amd_hipify.py
* do not use cudnn_conv_algo_search in ROCm EP, missing reduce min registration
* Fix ReduceConsts template specialization introduced in #9101.
Fixes the error when building for ROCm 4.3.1:
error: too many template headers for onnxruntime::rocm::ReduceConsts<__half>::One (should be 0)
* fix flake8 error in amd_hipify.py
* speed up hipify with concurrent.futures
* flake8 fix in amd_hipify.py
* Add netstandard2.0 to nuget managed package.
Re-does PR that was backed out due to packaging pipeline changes.
Allows deprecation of netstandard1.1 in the following release as netstandard2 is the preferred lowest level framework.
* Revert "Cleanup C# bindings to add EP (#8810)"
This reverts commit b21ea00020.
* Add back in a minimal set of changes.
Provide stubs in for a limited set of things
- things called from C# using a static lib of ORT built for mac/ios
- things in OrtApis that are not included in the build by default
- things in OrtApis that are excluded in a minimal build
* Cleanup order or EPs in test
* Fix unused function in ROCM build
Fix C# add EP bindings.
Add stubs to ORT so that if EP is not included in the build we return a graceful error message.
Move declaration of stubs into C API and out for EP so they're in one place and are easier to use (no extra header required in the C/C++ world and consistent with the CUDA EP setup).
Fix inconsistency in ROCM EP.
Cleanup a few other things.
