saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-18 21:21:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/csharp/test
History
Dmitri Smirnov a5dec8eedf
[C# ] Improve string marshalling and reduce GC pressure (#15545) 
### Description

  Reduce a number of auxillary objects created to reduce GC pressure.
Eliminate GCHandle type of memory pinning in most of the places.
Improve string marshalling by allocating unmanaged memory that does not
require pinning. Change native methods from `IntPtr` to `byte[]`
(marshalling pinning is more efficient).

Allocate input/output UTF-8 names in unmanaged heap for the lifetime of
InferenceSession. So we do not keep converting them and pinning on every
Run.

Introduce a new native API that allows to allocate and convert/copy
strings directly into a native tensor.

The PR delivers around 50% latency improvements and less GC pauses.

Inspired by: https://github.com/microsoft/onnxruntime/pull/15520

### Motivation and Context
Client experience GC pressure and performance degradation when dealing
with string tensors.


Co-Authored-By: @tannergooding
2023-04-20 15:12:51 -07:00
..
Microsoft.AI.MachineLearning.Tests Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
Microsoft.AI.MachineLearning.Tests.DotNet5_0
Microsoft.AI.MachineLearning.Tests.Uwp Fix Microsoft.AI.MachineLearning NuGet App failure with multiple binaries copied to same destination (#10076) 2021-12-21 12:34:03 -08:00
Microsoft.ML.OnnxRuntime.EndToEndTests Implement Optional Metadata support and C# test support (#15314) 2023-04-11 09:41:59 -07:00
Microsoft.ML.OnnxRuntime.EndToEndTests.Mobile Bump Newtonsoft.Json from 13.0.1 to 13.0.2 in /csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Mobile/EndToEndTests.Mobile.Automation (#13884) 2022-12-09 13:07:05 -08:00
Microsoft.ML.OnnxRuntime.Tests.Common [C# ] Improve string marshalling and reduce GC pressure (#15545) 2023-04-20 15:12:51 -07:00
Microsoft.ML.OnnxRuntime.Tests.Devices Add Xamarin support (#9436) 2021-10-27 20:07:07 +10:00
Microsoft.ML.OnnxRuntime.Tests.Droid Add csharp docfx (#12596) 2022-08-25 09:51:32 -05:00
Microsoft.ML.OnnxRuntime.Tests.iOS Add .net6 support to the C# nuget package. (#11908) 2022-06-22 08:08:24 +10:00
Microsoft.ML.OnnxRuntime.Tests.NetCoreApp [C#] Allow passing various options when creating singleton Environment object. (#14723) 2023-04-18 21:49:55 -07:00