saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-27 22:45:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
8250 commits 2856 branches 57 tags 1.3 GiB
Commit graph

8250 commits

This branch
This branch
All branches
Author SHA1 Message Date
JiCheng
be1416d032
prefast C26451 (#14933) 
Fixed
[AB#13290](https://aiinfra.visualstudio.com/6a833879-cd9b-44a4-a9de-adc2d818f13c/_workitems/edit/13290)
 2023-03-07 15:16:50 +08:00
Changming Sun
3e08a67dd6
Add Linux ARM64 CI pipeline (#14904) 2023-03-06 21:47:10 -08:00
Adrian Lizarraga
d45b47945c
Linux QNN Pipeline: fix build error reporting (#14922) 
### Description
Split up the ORT build step in the Linux QNN CI Pipeline.


### Motivation and Context
Build errors were not being immediately reported at the end of the build
step. The build step currently concatenates multiple shell commands, and
the return code for the last (mkdir) was being reported. This PR ensures
that the return code of the `python build.py ...` command is reported
for the build step.
 2023-03-06 17:49:35 -08:00
Sheil Kumar
f88b97ede2
Cast to int32_t->size_t to avoid prefast overflow warning (#14902) 
Cast to int32_t->size_t to avoid prefast overflow warning
 2023-03-05 06:21:46 -08:00
Tianlei Wu
6c8538f086
Fix prefast warning (#14895) 
Fix a prefast warning: `The const variable 'spatial_dim_start' can be
computed at compile-time. Consider using constexpr (con.5).`
 2023-03-03 12:54:28 -08:00
Changming Sun
c1155b70c5
Remove 37 and 50 from CUDA compute archs (#14874) 
### Description
To reduce CUDA package's size a little bit. 37 is for Tesla K80. Azure's
NC-series uses it, but in most cases CUDA can dynamic generate device
code .
 2023-03-03 12:24:21 -08:00
George Wu
289f7dbcdd
enable pybind for qnn ep (#14897) 
enable python bindings for QNN EP.
tested on Windows Dev Kit 2023 (ARM64) with python 3.11 (ARM64) from 
https://www.python.org/ftp/python/3.11.1/python-3.11.1-arm64.exe
 2023-03-03 07:26:53 -08:00
pengwa
f6c81d8aca
Introduce padding inspector in ORTModule (#14652) 
### Introduce padding inspector in ORTModule

In some Transformer-based LLM training recipes, high data sparsity is
observed due to 1). token padding (to max sequence length), 2). labels
contains many ignore_index for calculate loss.

This PR introduces a switch to enable data sparsity inspection, which 
1). in short term, can inform training users to use techniques like
dynamic batching to amortize the issue.
2). in medium and longer term, also helps us (training team) to have
better understanding what our training customers' models looks like from
perspective of data sparsity (and potentially motivate us to improve
with runtime).

Here is an example of different data sparsity with same training model
arch, same training input, but with different user models.

**Low Embed Density, High Label Density Case - Sentence Classification**
`
python -m torch.distributed.launch --nproc_per_node=4
examples/onnxruntime/training/text-classification/run_glue.py
--model_name_or_path roberta-large-openai-detector --task_name mnli
--do_train --do_eval --max_seq_length 128 --per_device_train_batch_size
32 --learning_rate 2e-5 --num_train_epochs 3 --overwrite_output_dir
--output_dir ./outputs/ --per_device_eval_batch_size 32 --seed 1137
--fp16 True --ignore_mismatched_sizes True --optim adamw_ort_fused
`
```
>>>Valid token/label density (e.g. valid/total) in passing 10 steps:
        | STEP       | INPUT TYPE |  INPUT NAME     | PAD IDX    | DENSITY    | VALID TOKENS    | TOTAL TOKENS    | VALID TOKENS/BATCH |
        | 60         | EMBED      | input_ids       | 1          | 35.21    % | 1442            | 4096            | [50, 81, 35, 11, 29, 36, 66, 19, 40, 22, 21, 42, 17, 37, 40, 41, 26, 58, 38, 54, 41, 73, 48, 57, 50, 51, 49, 85, 48, 36, 79, 62] |
        | 61         | LABEL      | labels          | -100       | 100.00   % | 32              | 32              | N/A             |
        | 62         | EMBED      | input_ids       | 1          | 30.00    % | 1229            | 4096            | [36, 73, 13, 47, 27, 33, 53, 25, 51, 28, 36, 42, 42, 32, 39, 52, 27, 13, 31, 66, 42, 45, 52, 45, 58, 42, 37, 66, 12, 18, 29, 17] |
        | 63         | LABEL      | labels          | -100       | 100.00   % | 32              | 32              | N/A             |
        | 64         | EMBED      | input_ids       | 1          | 26.73    % | 1095            | 4096            | [37, 28, 20, 53, 16, 20, 44, 52, 27, 28, 16, 19, 16, 24, 63, 31, 24, 42, 33, 41, 44, 60, 44, 67, 54, 30, 20, 19, 33, 23, 24, 43] |
        | 65         | LABEL      | labels          | -100       | 100.00   % | 32              | 32              | N/A             |
        | 66         | EMBED      | input_ids       | 1          | 30.03    % | 1230            | 4096            | [22, 46, 36, 41, 46, 43, 26, 50, 60, 16, 24, 42, 56, 35, 35, 59, 29, 39, 34, 20, 66, 23, 47, 53, 19, 35, 44, 23, 34, 81, 21, 25] |
        | 67         | LABEL      | labels          | -100       | 100.00   % | 32              | 32              | N/A             |
        | 68         | EMBED      | input_ids       | 1          | 31.62    % | 1295            | 4096            | [75, 36, 48, 20, 38, 21, 49, 54, 38, 41, 26, 28, 80, 45, 48, 16, 22, 41, 34, 28, 37, 16, 74, 63, 62, 34, 22, 45, 23, 27, 37, 67] |
        | 69         | LABEL      | labels          | -100       | 100.00   % | 32              | 32              | N/A             |
<<<
```

**High Embed Density, Low Label Density Case - masked language model** 
`
python -m torch.distributed.launch --nproc_per_node=4
examples/onnxruntime/training/language-modeling/run_mlm.py
--model_name_or_path bert-base-uncased --dataset_name wikitext
--dataset_config_name wikitext-2-raw-v1 --num_train_epochs 10
--per_device_train_batch_size 8 --per_device_eval_batch_size 8
--do_train --do_eval --overwrite_output_dir --output_dir ./outputs/
--seed 1137 --fp16 --report_to none --optim adamw_ort_fused
`
```
>>>Valid token/label density (e.g. valid/total) in passing 10 steps:
        | STEP       | INPUT TYPE |  INPUT NAME     | PAD IDX    | DENSITY    | VALID TOKENS    | TOTAL TOKENS    | VALID TOKENS/BATCH |
        | 710        | EMBED      | input_ids       | 0          | 100.00   % | 4096            | 4096            | [512, 512, 512, 512, 512, 512, 512, 512] |
        | 711        | LABEL      | labels          | -100       | 13.77    % | 564             | 4096            | N/A             |
        | 712        | EMBED      | input_ids       | 0          | 100.00   % | 4096            | 4096            | [512, 512, 512, 512, 512, 512, 512, 512] |
        | 713        | LABEL      | labels          | -100       | 14.48    % | 593             | 4096            | N/A             |
        | 714        | EMBED      | input_ids       | 0          | 100.00   % | 4096            | 4096            | [512, 512, 512, 512, 512, 512, 512, 512] |
        | 715        | LABEL      | labels          | -100       | 14.18    % | 581             | 4096            | N/A             |
        | 716        | EMBED      | input_ids       | 0          | 100.00   % | 4096            | 4096            | [512, 512, 512, 512, 512, 512, 512, 512] |
        | 717        | LABEL      | labels          | -100       | 14.53    % | 595             | 4096            | N/A             |
        | 718        | EMBED      | input_ids       | 0          | 100.00   % | 4096            | 4096            | [512, 512, 512, 512, 512, 512, 512, 512] |
        | 719        | LABEL      | labels          | -100       | 15.31    % | 627             | 4096            | N/A             |
<<<
```

#### Next Step

Let's see how we leverage the data sparsity for improvement.
Optimizations on the way around compute optimizer wave 2:
> Loss compute flops reduction.
> Flatten/Unflatten embedding tokens to save compute flops.
 2023-03-03 18:36:08 +08:00
Yi Zhang
8c454a76e0
Check Mac silicon package name (#14898) 
### Description
1. add comments 
2. check Mac silicon package name 

### Motivation and Context
There isn't Mac silicon Agent in ADO.
We couldn't add smoking test to test the wheel can be installed.
But We can check whether the package name is correct to avoid the
mistake in 1.14 release.

Test run

https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=283100&view=logs&j=fe710151-df7c-5aa4-0cea-cf5331faa499&t=3182cefe-2612-53c6-4445-e5b3e0c4ac57
 2023-03-03 18:27:54 +08:00
cloudhan
a997bb46b6
Refactor rocm attention (#14688) 
Extract QKV projection and attention computation into pipelines (composed from gemms and kernel launch). 

This will allow us to introduce ck flash attention in next PR
 2023-03-03 12:16:11 +08:00
Changming Sun
f3b6664384
Remove Python 3.7 from the python packaging pipeline (#14887) 
### Description
1. Remove Python 3.7 from the python packaging pipeline. It is planned
for the next release and approved by the PMs. Also we will add 3.11, but
it will be addressed in another PR.
2. Stop generating python packages based on Ubuntu 18.04 which will
reach EOL next month. We will either replace them with Ubuntu 20.04 or a
CentOS 8 variant.
 2023-03-02 19:44:49 -08:00
guyang3532
c49f250a14
Del ort_model._modules to foward its accessing to torch_model._modules (#14563) 
Missing '_modules' attribute in ORTModule will cause load_state_dict for
wrapped_ortmodule fail.

reference:https://github.com/microsoft/onnxruntime/pull/7847
 2023-03-03 10:12:37 +08:00
Dmitri Smirnov
8d87fdcfa1
Add GetVersionSting API for C++, C# and Python (#14873) 
### Description
Added APIs.

### Motivation and Context
Addresses https://github.com/microsoft/onnxruntime/issues/14584

Cc: @Craigacp cp
 2023-03-02 17:11:07 -08:00
Zachary Streeter
6e2ca15140
added miopenGetConvolutionSpatialDim if ROCm5.5 (#14772) 
The API should be `miopenGetConvolutionSpatialDim(cdesc, &spatial_dim)`, NOT `miopenGetConvolutionDescriptorSize(cdesc, &spatial_dim)`
 2023-03-03 08:02:32 +08:00
Yuriy Chernyshov
0ebe8e34f8
Do not use ADL to invoke std algos (#14851) 
This is a follow up for #14716
 2023-03-02 15:38:33 -08:00
Chun-Wei Chen
70a31e047a
Consume ONNX 1.13.1 in ONNX Runtime (#14812) 
### Description
<!-- Describe your changes. -->
Consume ONNX 1.13.1 in ONNX Runtime. (ONNX 1.13.0 to ONNX 1.13.1)


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
ONNX 1.13.1 patch was just released yesterday. This PR is making ORT's
ONNX submodule consistent with the latest released ONNX. Not sure
whether this PR is really needed, but let me make it ready. Previous PR
for testing ONNX 1.13.1rc2 :
https://github.com/microsoft/onnxruntime/pull/14634.

Fixed
[AB#13174](https://aiinfra.visualstudio.com/6a833879-cd9b-44a4-a9de-adc2d818f13c/_workitems/edit/13174)
.
 2023-03-02 14:57:35 -08:00
G. Ramalingam
2facc5efe6
Fix function inliner bug re. outer-scope names (#14734) 
### Description

Fix the function inliner logic for renaming variables. Typically, a
FunctionProto does not contain references to outer-scope names. However,
special cases, such as the function-expansion of SequenceMap, can
generate such FunctionProtos. Extend the renaming logic to ensure that
references to outer-scope names are not renamed.

### Motivation and Context
Fixes https://github.com/onnx/onnx/issues/4892

Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
 2023-03-02 12:08:37 -08:00
Chen Fu
603026fb84
Transpose for 16b tensors (#14877) 
### Description
Matrix transpose for 16b tensors (shorts, and half precision floats)


### Motivation and Context

Need it for fp16 operations
 2023-03-02 11:32:05 -08:00
Rachel Guo
7cd4b334a9
[CoreML EP] Add Flatten Op and LRN Op support (#14857) 
### Description
<!-- Describe your changes. -->

As title.

CoreML Spec for reference: 


https://apple.github.io/coremltools/mlmodel/Format/NeuralNetwork.html#flattento2dlayerparams


https://apple.github.io/coremltools/mlmodel/Format/NeuralNetwork.html#lrnlayerparams

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Fill CoreML Clipchamp usage gaps.

---------

Co-authored-by: rachguo <rachguo@rachguos-Mini.attlocal.net>
 2023-03-02 09:43:15 -08:00
Hector Li
bf35ad2aa3
[Qnn EP] Call Qnn deviceCreate during backend setup and deviceFree during shutdown (#14875) 
### Description
Call Qnn deviceCreate during backend setup and call deviceFree during
shutdown

### Motivation and Context
Algin with Qnn formal setup and shutdown procedure.
 2023-03-02 08:54:13 -08:00
cao lei
d69823f764
Do not create Barrier and triggerDownstream steps if the corresponding nodes are split by yield Op in training scenario (#14570) 
### Description
Do not create Barrier and triggerDownstream steps during execution plan
creation if the corresponding nodes are split by yield Op in training
scenario.



### Motivation and Context
In training scenario, forward and backward processes are running two
different partial nodes of a graph. If there are two nodes each in one
of the partial graph and separate in two streams, there are still
triggerDownstream/barrier steps between them which work quite different
from inference process as one of the steps will not be executed due to
it is not in the correct range. To make it work, there is a hacky way to
trigger the barrier step explicitly for training.
This PR is to do some check, and do not create Barrier and
triggerDownstream steps if the corresponding nodes are split by yield Op
in training scenario. So the hacky way is not needed.
 2023-03-02 07:08:29 -08:00
Tianlei Wu
c66af46fc1
Doc for Stable Diffusion CUDA Optimizations (#14830) 
Add document for stable diffusion optimizations and benchmark.
 2023-03-01 19:29:30 -08:00
Hector Li
c6074f3a4b
OnnxRuntime QNN EP (#14791) 
### Description
Integrate Qualcomm QNN SDK to enable inference on QC hexagon NPU devices

### Motivation and Context
Enable Ort inference on QC hexagon NPU devices.

---------

Co-authored-by: Satya Jandhyala <sajandhy@microsoft.com>
Co-authored-by: Adrian Lizarraga <adlizarraga@microsoft.com>
Co-authored-by: Adrian Lizarraga <adrianlm2@gmail.com>
 2023-03-01 13:48:20 -08:00
George Wu
6044312a43
fix TRT dockerfile documentation https://github.com/microsoft/onnxruntime/issues/14556 (#14600) 
address https://github.com/microsoft/onnxruntime/issues/14556
 2023-03-01 07:02:42 -08:00
Scott McKay
b7fde84341
Changes to support standalone custom ops in a minimal build. (#14497) 
### Description
<!-- Describe your changes. -->
Changes to support standalone custom ops in a minimal build. Also
incorporates changes from #14492 (needed to test builds prior to that
being checked in).

We first need to save the schema info from the operators used by the
standalone op invoker in the ORT format model. Add mechanism for that.

Merge the kernel lookup logic so the same is used in full and minimal
build. NOTE: the version matching is now consistent with all other
kernel lookups, and the call to CreateOp MUST use the exact version for
the operator. Previously matching wasn't as strict, but this can lead to
the incorrect kernel being chosen.

Add tests.

NOTE: There is currently no way to detect the ops/types/opsets used
inside these custom ops as they don't exist until we create kernels,
which is after model loading completes (which is the point the ORT
format model is saved). Due to that they have to be manually added to
the configuration used to do the reduced ops build. That shouldn't be
too hard for the custom op author to add given the custom op
implementation is specifying the op, opset and type constraints (i.e.
they have the info and it's just a case of capturing/formatting it
correctly).


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Enable usage of the standalone op invoker by custom ops in a minimal
build.

---------

Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
 2023-03-01 11:22:54 +10:00
Chen Fu
acc2ac627f
Fp16 Activations (#14722) 
### Description

NEON fp16 SIMD implementation of Activation functions


### Motivation and Context
Step 2 of fp16 SIMD support.

---------

Co-authored-by: Chen Fu <fuchen@microsoft.com>
 2023-02-28 17:20:40 -08:00
Yulong Wang
69c5edb11b
[wasm] upgrade emsdk from 3.1.19 to 3.1.32 (#14818) 
### Description
upgrade emsdk from 3.1.19 to 3.1.32

also add explicit config for stack size (1MB).
 2023-02-28 11:06:09 -08:00
Nat Kershaw (MSFT)
95c777b745
Fix link to High Level Design (#11786) 
Address #11661
 2023-02-28 11:05:54 -08:00
Yi Zhang
6320decf04
increase Test GPU Job's timeout to 8 hours (#14850) 
### Description
<!-- Describe your changes. -->

### Motivation and Context
In practice, 6 hours is not enough to finish the job.
 2023-02-28 18:52:03 +08:00
pengwa
79aa0acdd0
SCELoss(SCELossGrad) support half(float) input float(half) output (#13972) 
### Description

A follow up change for
https://github.com/microsoft/onnxruntime/pull/13616.

SoftmaxCrossEntropyLossInternal/SoftmaxCrossEntropyLossInternalGrad
support different type for input and output.

Add SCELoss(SCELossGrad) support half(float) input float(half) output

### Test Note

#### Add tests for variant input and output types. To add such tests,
have to refactor existing testing code for sce loss and scelossinternal
gradient.

Originally, 

FP32 input and output, the CPU kernels, runs with CPU kernels the
baseline, CUDA/RCOM then runs with same data, user CompareTester to
compare with CPU run results.

FP16 input and output, the CPU kernels (did not have half kernels), runs
with Cast_to_float->CPU kernel->cast_to_half as the baseline, CUDA/RCOM
then runs with same data but using Half implementation, user
CompareTester to compare with CPU run results.

Now, we want the support run different input and output types. The
proposed change here is, to run CPU kernels always with float input and
output as baseline (because CPU only have float type kernels impl), this
step is the very first thing for every test.

Then, we run CUDA/RCOM kernels using half_input_half_output,
float_input_float_output, half_input_float_output,
float_input_half_output if there is corresponding kernel registered.

Afterwards, compare the CUDA/ROCM run results with CPU float baselines. 

Be noted, there is one thing that deserved a special note:
CompareOpTester's result compare can be loose than OpTester's.
Roughly speaking: the former tolerant diff <= atol +
rtol*expected_value, while the later one telerant diff < atol && diff <
rtol*expected_value. When the expected value is super small in many
cases of our tests cases, the former one can pass but the later one
fails. So the refactoring also move the check outside of OpTester,
explicitly check the values using the way CompareOPTester did (to align
the previous behaviour).

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
 2023-02-28 18:02:08 +08:00
Justin Stoecker
08699c8052
Address SDL warnings in recent STFT changes (#14847) 
### Description
Addresses two separate SDL warnings, neither of which point to a cause
for concern:
1. `The expression '0<=_Param_(1)&&_Param_(1)<=3-1' is not true at this
call.
at
D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\Operators\\DmlSTFT.h@443,33`.
In other words, the tool thinks one of the calls to
`barriers[barrierCount++]` will be an index-of-of-range issue, even
those this is not currently possible. Switching a normal C array avoids
this complaint.
2. `'_Param_(1)' could be '0': this does not adhere to the specification
for the function 'CD3DX12_RESOURCE_BARRIER::UAV'`. The d3dx12 helper for
UAV barriers has the wrong SAL annotation and doesn't allow a null
resource (`_In_`), even though a null resource is legal and well
defined. Updated the annotation to `_In_out_` and created a PR upstream.

### Motivation and Context
Pacify SDL tasks in CI pipelines.
 2023-02-27 21:01:34 -08:00
kailums
9bdd42115c
add build flag for rocblas tune and fix bug (#14797) 
### Description
<!-- Describe your changes. -->
1. add a build flag for rocblas tuning feature.

2. fix a build bug when enable rocblas tuning.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
The rocblas tunning feature has no build flag to control, only using a
MACRO flag.

So I add an build flag, and fix a code bug when enable rocblas tunning.
 2023-02-28 10:37:07 +08:00
Yulong Wang
2d079c6333
[js/web] disable multi-thread test on Node.js in E2E test (#14844) 
### Description
disable multi-thread test on Node.js in E2E test.

multi-thread test on Node.js in E2E test never worked, however the CI
does not pick up the error every time. So this became a flaky test case
which sometimes cause a build break.

Disable this test now and should enable it once it's get fixed.
 2023-02-27 16:01:51 -08:00
Yi Zhang
0be20dc0f6
Run GPU test job after all CPU test jobs succeed. (#14833) 
### Description
Make GPU job depends on all CPU jobs

### Motivation and Context
GPU resources are very limited in packaging pipeline.
And GPU test job is very time consuming.
Only one CPU job fails, the workflow fails, so the GPU job is
meaningless.
To utilize GPU resources more efficiently, run GPU job only after all
CPU jobs succeed.

###test pipeline

https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=280905&view=results
 2023-02-28 07:44:51 +08:00
Chen Fu
26abaeb284
Fix half precision gemm test accumulation error (#14842) 
### Description

Half precision gemm test requirement relaxation

### Motivation and Context

Most CPUs does not support mixed precision accumulation, only mul & add
fuse. As a result, different striding on the K dimension may lead to
rounding error. Accumulation of these rounding error maybe very
significant. So setting an approximation ratio does NOT always work.
What's more, a relaxed test condition may hide real implementation
problem. So this is only a compromised fix.

More rigorous tests require manual efforts:
1. Change the K stride of the kernel under test to be 16.
2. Force the K stride of the fp16 kernel to 16
3. Change the test oracle to be exact match.
4. Pass this test and then change it back :-(.

Co-authored-by: Chen Fu <fuchen@microsoft.com>
 2023-02-27 13:23:14 -08:00
Albert Grigoryan
e097e4e93c
Objective-C lib: Added support for int64 and uint64. (#14405) 
### Description
Added support for int64 and uint64 in Objective-C lib.

### Motivation and Context
Int64 is rarely used, but we needed it. The Int64 inference worked after
the change (tested).
 2023-02-24 23:25:16 -08:00
satyajandhyala
dcc8fe656b
Fix type mismatch in CUDA Trilu op. (#12863) 
Added type cast to int64_t to avoid overflow errors/alerts.
 2023-02-24 23:24:26 -08:00
Ivan Komarov
9f6d452ca6
Fix ValueError when testing PyTorch performance (#14450) 
### Description
Fixed an exception that is thrown inside `transformers` when trying to
test PyTorch performance:
```
> python convert_generation.py -m gpt2 --output gpt2_greedy_search.onnx --num_beams 1 --num_return_sequences 1 --torch_performance
 2023-02-24 21:39:14 -08:00
Tianlei Wu
6603be26c6
Fix prefast warnings in CUDA ops (#14814) 
Fix some prefast warnings in CUDA ops.
 2023-02-24 21:23:08 -08:00
cloudhan
3bcdb0a83a
Fix TunableOp signature generation (#14709) 
Currently, all generated op sigs are `TunableOp<ParamsT, TimerT>` which
is causing signature collision when using rocblas solution api. This was
not a problem before `TuningContext`, because the `KernelMap`s were not
shared.

The root cause is the signature is initialized on Op consturction,
specificially, only in base class ctor, which is causing the type info
only caputre base class type info. That is, only the ParamsT + and base
class. After this change, the we will encode the derived class type in
the op sig.
 2023-02-25 12:05:07 +08:00
Chen Fu
bc5d0c83d1
Fix prefast scan complaints (#14823) 
### Description

Fix prefast warning
https://dev.azure.com/aiinfra/ONNX%20Runtime/_workitems/edit/12821/


### Motivation and Context

This one is just excessive and annoying. It warned that I should cast
value to 8bytes before using * operator. But the expression is:
(size_t(64) * size_t(1024))

size_t is 8 bytes on 64b systems. And when it is 4 bytes on 32b systems,
the cast is expensive and completely unnecessary.
 2023-02-24 17:23:22 -08:00
Yulong Wang
6b83ad9659
[js/web] allow unittest (onnxruntime_test_all) to run in browser (#14820) 
### Description
allow onnxruntime_test_all to run in browser for WebAssembly build (use
flag `--wasm_run_tests_in_browser`).

To output the logs from stdout correctly, this test needs to be build
with `--enable_wasm_threads`.
 2023-02-24 16:45:33 -08:00
Yulong Wang
a631ed77c0
[js/web] support flag 'optimizedModelFilePath' in session options (#14355) 
### Description
* Support flag 'optimizedModelFilePath' in session options.

In Node.js, the model will be saved into filesystem just like its
behaviour on native platforms.

In browser, the new model is not saved to filesystem. the file path is
ignored. Instead, a new pop-up window will be launched in browser and
user can 'save' the file as onnx model.

* Add corresponding commandline args for the following session option
flags:
    - optimizedModelFilePath
    - graphOptimizationLevel
 2023-02-24 15:50:15 -08:00
Rachel Guo
0700788b6e
Disable e2e android react native CI test temporarily (#14803) 
### Description
<!-- Describe your changes. -->

Disable e2e android react native test temporarily to unblock the CI
failure with no easy fix.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Temp solution to unblock CI failure.
 2023-02-24 09:32:18 -08:00
Ted Themistokleous
702a61c3bb
Add verbose and optimization args for parity tests (Gelu, Layernorm, … (#14739) 
…GPT_Attention)

Some EPs require that onnxruntime and optimum optimizations are turned
off in order to run correctly. Allowing this option during test runs
allows the EP and library to perform their own optimization and be more
representative of actual use case conditions.

Important for EPs like MIGraphX which require optimizations to be offer
for certain operations

### Description
<!-- Describe your changes. -->

Allow flags to turn off optimizations and add verbose output to confirm
which EP is being used for the inference run and validate fallbacks

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Related to: #14702 & #14700

---------

Signed-off-by: Ted Themistokleous <tthemist@amd.com>
Co-authored-by: Ted Themistokleous <tthemist@amd.com>
 2023-02-24 18:43:13 +08:00
Tianlei Wu
d3785ef8f6
Fix decoder_attention scratch buffer size and prefast warnings (#14808) 
(1) Change `GetScratchBuffer<T>(element_count * element_size)` to
`GetScratchBuffer<T>(element_count)` to avoid allocating more memory
than needed.
(2) Fix prefast:Warning C26451: Arithmetic overflow: Using operator '*'
on a 4 byte value and then casting the result to a 8 byte value.
 2023-02-23 21:58:28 -08:00
Justin Stoecker
928289c414
STFT for DML EP (#14736) 
### Description
Implements the STFT operator for the DirectML execution provider. This
is implemented as a custom op, just like the DFT kernel, because it's
implemented as a composite of two operators (DML Mul/Identity + DFT). As
such, this inherits the same restrictions as the existing DFT kernel
(requires power-of-two window sizes for now).

This change also adds a native FP16 shader to DFT so that both DFT/STFT
kernels support float16 tensors. There is no typed UAV fallback or
emulation path, so the HW _needs_ to support native float16. It also
appears the stockham shader was compiled with all optimizations disabled
and debug symbols (tsk tsk, Sheil), and this has been fixed.

This is passing all existing STFT tests (i.e. all of 1). I'm adding some
additional collateral in the Windows AI conformance tests in parallel to
check some extra cases.

---------

Co-authored-by: Patrice Vignola <vignola.patrice@gmail.com>
 2023-02-23 21:12:22 -08:00
PeixuanZuo
687118a159
[ROCm] Fix Skiplayernorm error and open ke test with optional output (#14794) 
Fix Skiplayernorm error and open kernel explorer test with optional
output.
 2023-02-24 12:46:42 +08:00
Patrice Vignola
abb51ec975
[DML EP] Fix GetInputTensor crash when accessing null tensor (#14809) 2023-02-23 18:13:40 -08:00
Jian Chen
29428cd9dc
Cjian/pr into main for 1.14.1 fix (#14805) 
### Description
<!-- Describe your changes. -->

PR a change made to 1.14.1 into Main branch as well. 

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
 2023-02-23 18:10:57 -08:00