### Description
This limits the size of constant data nodes which the DML EP creates in
the DML graph following de-duplication of 1D quantization tensors. In
the process it reduces a check for the maximum size of the constant
node.
This is merged from: https://github.com/microsoft/onnxruntime/pull/18494
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
Cleanup and rebase from [this
PR](https://github.com/microsoft/onnxruntime/pull/18629)
### Motivation and Context
---------
Co-authored-by: Christian Larson <chrilaMSFT@users.noreply.github.com>
Co-authored-by: Christian Larson <28911437+chrilaMSFT@users.noreply.github.com>
Co-authored-by: Jeff Bloomfield <jeffbloo@microsoft.com>
Co-authored-by: Anagha Rao <anagrao@microsoft.com>
### Description
This addresses a bug in a fast path that was added for submission of
re-used command lists of fused graph kernels in the DML EP, addressing a
D3D debug layer error.
### Motivation and Context
The fast path in DmlCommandRecorder::ExecuteCommandList enabled a
current non-reused command list, if empty, to be used for commands
following submission of the fused command list. The fix ensures the
associated command allocator is only re-used after the next fence value
is completed, which is higher due to submission of the other command
list.
The command recorder design was intended to support batching of provided
command list execution, however it submits command lists immedately as
an implementation detail to maximize CPU/GPU parallelism. If that
heuristic was removed, it would expose additional issues in this same
fast path. Because of this and complexity and inefficiency of the old
batching mechanism, I also removed this.
### Description
[DirectML EP] Add DML EP registration for Col2Im operator
### Motivation and Context
Add Col2Im support for opset 18.
This operator is implemented as the DirectML Fold operator.
---------
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
Co-authored-by: Dwayne Robinson <dwayner@microsoft.com>
Update resource creation flag to avoid D3D12 WARNING
### Description
Update the DML DX12 allocator to use D3D12_RESOUCE_STATE_COMMON to avoid
DX12 Warning messages.
### Motivation and Context
When directML is created with debug layer there are warnings when
resources are created by ORT.
---------
Co-authored-by: Christian Larson <28911437+chrilaMSFT@users.noreply.github.com>
### Description
1. Expand input datatype support for Resize with uint8/int8.
2. Update the logic to compute output shape of Resize Op, roiRange is
got rid of to align with how tests compute the output shape to go around
the size asserting in MLOperatorAuthorImpl.cpp
`m_inputDimensions[i] * roiRange * scale` -> `m_inputDimensions[i] *
scale`
3. disable 4 tests because of the result mismatch. The results of DML
with float32 and uint8/int8 match each other, so it should be problem of
resize implementation, which is out the scope of this PR.
`ResizeOpTest.NhwcResizeOpLinearDownSampleTest_tf_crop_and_resize_without_extrapolation_uint8
ResizeOpTest.NhwcResizeOpLinearDownSampleTest_tf_crop_and_resize_without_extrapolation_int8
ResizeOpTest.NhwcResizeOpLinearDownSampleTest_4DBilinear_pytorch_half_pixel_uint8
ResizeOpTest.NhwcResizeOpLinearDownSampleTest_4DBilinear_pytorch_half_pixel_int8`
[Cherry pick Reviewed]
Re-add changes which were merged out...
---------
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Sheil Kumar <smk2007@gmail.com>
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Jeff Bloomfield <jeffbloo@microsoft.com>
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
---------
Co-authored-by: Jeff Bloomfield <jeffbloo@microsoft.com>
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Jeff Bloomfield <jeffbloo@microsoft.com>
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Jeff Bloomfield <jeffbloo@microsoft.com>
### Description
This PR also includes,
8b0a55e7cc DML constant pow operator
7520974970 Enable custom heaps based on query-
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
---------
Co-authored-by: Jeff Bloomfield <jeffbloo@microsoft.com>
[Cherry Pick Reviewed]
DML EP Implementation for
[QLinearAveragePool](https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#com.microsoft.QLinearAveragePool)
```
Note: Google Test filter = *QLinear*Pool*
[==========] Running 72 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 36 tests from QLinearGlobalAveragePool
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x1x32x32
[ OK ] QLinearGlobalAveragePool.Nhwc_1x1x32x32 (410 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x32x32x1
[ OK ] QLinearGlobalAveragePool.Nchw_1x32x32x1 (641 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x256x8x8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x256x8x8 (156 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x8x8x256
[ OK ] QLinearGlobalAveragePool.Nchw_1x8x8x256 (134 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x255x7x7
[ OK ] QLinearGlobalAveragePool.Nhwc_1x255x7x7 (160 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x7x7x255
[ OK ] QLinearGlobalAveragePool.Nchw_1x7x7x255 (145 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x255x8x8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x255x8x8 (148 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x8x8x255
[ OK ] QLinearGlobalAveragePool.Nchw_1x8x8x255 (129 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x256x7x7
[ OK ] QLinearGlobalAveragePool.Nhwc_1x256x7x7 (134 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x7x7x256
[ OK ] QLinearGlobalAveragePool.Nchw_1x7x7x256 (131 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x256x8x8
[ OK ] QLinearGlobalAveragePool.Nhwc_3x256x8x8 (159 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x8x8x256
[ OK ] QLinearGlobalAveragePool.Nchw_3x8x8x256 (168 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x255x7x7
[ OK ] QLinearGlobalAveragePool.Nhwc_3x255x7x7 (139 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x7x7x255
[ OK ] QLinearGlobalAveragePool.Nchw_3x7x7x255 (170 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x255x8x8
[ OK ] QLinearGlobalAveragePool.Nhwc_3x255x8x8 (155 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x8x8x255
[ OK ] QLinearGlobalAveragePool.Nchw_3x8x8x255 (156 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x256x7x7
[ OK ] QLinearGlobalAveragePool.Nhwc_3x256x7x7 (133 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x7x7x256
[ OK ] QLinearGlobalAveragePool.Nchw_3x7x7x256 (149 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x1x32x32_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x1x32x32_S8 (131 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x32x32x1_S8
[ OK ] QLinearGlobalAveragePool.Nchw_1x32x32x1_S8 (127 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x256x8x8_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x256x8x8_S8 (153 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x8x8x256_S8
[ OK ] QLinearGlobalAveragePool.Nchw_1x8x8x256_S8 (129 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x255x7x7_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x255x7x7_S8 (133 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x7x7x255_S8
[ OK ] QLinearGlobalAveragePool.Nchw_1x7x7x255_S8 (135 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x255x8x8_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x255x8x8_S8 (129 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x8x8x255_S8
[ OK ] QLinearGlobalAveragePool.Nchw_1x8x8x255_S8 (152 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_1x256x7x7_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_1x256x7x7_S8 (140 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_1x7x7x256_S8
[ OK ] QLinearGlobalAveragePool.Nchw_1x7x7x256_S8 (133 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x256x8x8_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_3x256x8x8_S8 (135 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x8x8x256_S8
[ OK ] QLinearGlobalAveragePool.Nchw_3x8x8x256_S8 (147 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x255x7x7_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_3x255x7x7_S8 (156 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x7x7x255_S8
[ OK ] QLinearGlobalAveragePool.Nchw_3x7x7x255_S8 (155 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x255x8x8_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_3x255x8x8_S8 (138 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x8x8x255_S8
[ OK ] QLinearGlobalAveragePool.Nchw_3x8x8x255_S8 (155 ms)
[ RUN ] QLinearGlobalAveragePool.Nhwc_3x256x7x7_S8
[ OK ] QLinearGlobalAveragePool.Nhwc_3x256x7x7_S8 (144 ms)
[ RUN ] QLinearGlobalAveragePool.Nchw_3x7x7x256_S8
[ OK ] QLinearGlobalAveragePool.Nchw_3x7x7x256_S8 (139 ms)
[----------] 36 tests from QLinearGlobalAveragePool (5968 ms total)
[----------] 36 tests from QLinearPoolTest
[ RUN ] QLinearPoolTest.AveragePool1D_ExcludePadPixel
[ OK ] QLinearPoolTest.AveragePool1D_ExcludePadPixel (480 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_IncludePadPixel
[ OK ] QLinearPoolTest.AveragePool1D_IncludePadPixel (481 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_ExcludePadPixel
[ OK ] QLinearPoolTest.AveragePool2D_ExcludePadPixel (512 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_IncludePadPixel
[ OK ] QLinearPoolTest.AveragePool2D_IncludePadPixel (455 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_MultiChannel
[ OK ] QLinearPoolTest.AveragePool2D_MultiChannel (463 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_ExcludePadPixel
[ OK ] QLinearPoolTest.AveragePool3D_ExcludePadPixel (448 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_IncludePadPixel
[ OK ] QLinearPoolTest.AveragePool3D_IncludePadPixel (458 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_ExcludePadPixel_nhwc
[ OK ] QLinearPoolTest.AveragePool1D_ExcludePadPixel_nhwc (171 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_IncludePadPixel_nhwc
[ OK ] QLinearPoolTest.AveragePool1D_IncludePadPixel_nhwc (169 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_ExcludePadPixel_nhwc
[ OK ] QLinearPoolTest.AveragePool2D_ExcludePadPixel_nhwc (152 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_IncludePadPixel_nhwc
[ OK ] QLinearPoolTest.AveragePool2D_IncludePadPixel_nhwc (660 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_MultiChannel_nhwc
[ OK ] QLinearPoolTest.AveragePool2D_MultiChannel_nhwc (150 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_ExcludePadPixel_nhwc
[ OK ] QLinearPoolTest.AveragePool3D_ExcludePadPixel_nhwc (145 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_IncludePadPixel_nhwc
[ OK ] QLinearPoolTest.AveragePool3D_IncludePadPixel_nhwc (146 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_BigImage
[ OK ] QLinearPoolTest.AveragePool2D_BigImage (505 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_BigImage_nhwc
[ OK ] QLinearPoolTest.AveragePool2D_BigImage_nhwc (161 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_Global
[ OK ] QLinearPoolTest.AveragePool2D_Global (481 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_Global_nhwc
[ OK ] QLinearPoolTest.AveragePool2D_Global_nhwc (152 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_ExcludePadPixel_S8
[ OK ] QLinearPoolTest.AveragePool1D_ExcludePadPixel_S8 (461 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_IncludePadPixel_S8
[ OK ] QLinearPoolTest.AveragePool1D_IncludePadPixel_S8 (448 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_ExcludePadPixel_S8
[ OK ] QLinearPoolTest.AveragePool2D_ExcludePadPixel_S8 (471 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_IncludePadPixel_S8
[ OK ] QLinearPoolTest.AveragePool2D_IncludePadPixel_S8 (473 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_MultiChannel_S8
[ OK ] QLinearPoolTest.AveragePool2D_MultiChannel_S8 (1507 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_ExcludePadPixel_S8
[ OK ] QLinearPoolTest.AveragePool3D_ExcludePadPixel_S8 (477 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_IncludePadPixel_S8
[ OK ] QLinearPoolTest.AveragePool3D_IncludePadPixel_S8 (493 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_ExcludePadPixel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool1D_ExcludePadPixel_nhwc_S8 (158 ms)
[ RUN ] QLinearPoolTest.AveragePool1D_IncludePadPixel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool1D_IncludePadPixel_nhwc_S8 (146 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_ExcludePadPixel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool2D_ExcludePadPixel_nhwc_S8 (146 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_IncludePadPixel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool2D_IncludePadPixel_nhwc_S8 (158 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_MultiChannel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool2D_MultiChannel_nhwc_S8 (157 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_ExcludePadPixel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool3D_ExcludePadPixel_nhwc_S8 (145 ms)
[ RUN ] QLinearPoolTest.AveragePool3D_IncludePadPixel_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool3D_IncludePadPixel_nhwc_S8 (147 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_BigImage_S8
[ OK ] QLinearPoolTest.AveragePool2D_BigImage_S8 (537 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_BigImage_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool2D_BigImage_nhwc_S8 (173 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_Global_S8
[ OK ] QLinearPoolTest.AveragePool2D_Global_S8 (457 ms)
[ RUN ] QLinearPoolTest.AveragePool2D_Global_nhwc_S8
[ OK ] QLinearPoolTest.AveragePool2D_Global_nhwc_S8 (150 ms)
[----------] 36 tests from QLinearPoolTest (12914 ms total)
[----------] Global test environment tear-down
[==========] 72 tests from 2 test suites ran. (18885 ms total)
[ PASSED ] 72 tests.
memleakdbg:
----- No memory leaks detected -----
```
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Adrian Tsai <adtsai@microsoft.com>
### Description
[Cherry Pick Reviewed]
```
[ OK ] QLinearConcatS8.ExpectFail_WrongZeroPointType_1 (372 ms)
[ RUN ] QLinearConcatS8.InputOne_Dynamic
[ OK ] QLinearConcatS8.InputOne_Dynamic (255 ms)
[ RUN ] QLinearConcatS8.InputOne_Const
[ OK ] QLinearConcatS8.InputOne_Const (255 ms)
[----------] 11 tests from QLinearConcatS8 (3385 ms total)
[----------] Global test environment tear-down
[==========] 21 tests from 3 test suites ran. (9355 ms total)
[ PASSED ] 21 tests.
```
[#16971](https://github.com/microsoft/onnxruntime/pull/16971)
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Xiang Zhang <xianz@microsoft.com>
### Description
<!-- Describe your changes. -->
If we fail to calculate the buffer size (due to overflow) we currently
return a nullptr. This is inconsistent as an actual memory allocation
failure throws. An overflow would typically be due to bad input so an
exception makes more sense given that.
Change to throw so code using MakeUniquePtr* and AllocArray* doesn't
need to check for nullptr.
Add some extra info to the log message to help debugging.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Should help with #18905 by avoiding the invalid attempted usage of a
nullptr from the allocation. Extra info _might_ help with figuring out
where the overflow is coming from which is the real issue.
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
Previously, shape and strides were added unconditionally even they are
not used. This PR fixes this issue and only adds shape and strides when
they are required.
It's useful when some shapes are not used as uniform if the program
depends on type instead of rank.
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
Add trilinear interpolation to Resize and changed activation_params attribute as optional for FuseConv.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Description
<!-- Describe your changes. -->
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
### Fix build when flash attention and memory efficient attention are
disabled
On a customer env with lower version of CUDA < 11.6. Both flash
attention and memory efficient attention is turned OFF according to
e8f33b54ba/cmake/CMakeLists.txt (L701).
So
e8f33b54ba/cmake/external/cutlass.cmake (L1)
condition check return false. No cutlass lib is built.
```
Turn off flash attention since CUDA compiler version < 11.6
```
While, the kernels in
https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/contrib_ops/cuda/moe/ft_moe
are depending on cutass for its build, so we get error like this:
```
[ 77%] Building CUDA object CMakeFiles/onnxruntime_providers_cuda.dir/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu.o
In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17:
/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory
23 | #include "cutlass/array.h"
| ^~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17:
/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory
23 | #include "cutlass/array.h"
| ^~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17:
/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory
23 | #include "cutlass/array.h"
| ^~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17:
/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory
23 | #include "cutlass/array.h"
| ^~~~~~~~~~~~~~~~~
compilation terminated.
fatal : Could not open input file /tmp/tmpxft_00044da3_00000000-11_moe_gemm_kernels_fp16_fp16.compute_60.cpp1.ii
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/build.make:6290: CMakeFiles/onnxruntime_providers_cuda.dir/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:2210: CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
Traceback (most recent call last):
File "/tmp/onnxruntime/tools/ci_build/build.py", line 2746, in <module>
sys.exit(main())
File "/tmp/onnxruntime/tools/ci_build/build.py", line 2639, in main
build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
File "/tmp/onnxruntime/tools/ci_build/build.py", line 1527, in build_targets
run_subprocess(cmd_args, env=env)
File "/tmp/onnxruntime/tools/ci_build/build.py", line 824, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
File "/tmp/onnxruntime/tools/python/util/run.py", line 49, in run
completed_process = subprocess.run(
File "/opt/conda/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
```
### Motivation and Context
To summarize, there are two cases we will have build failure for Linux
CUDA build:
1. User use cuda version < 11.6
2. User disabled Flash attention and memory efficient attention
explictly with onnxruntime_USE_FLASH_ATTENTION and
onnxruntime_USE_MEMORY_EFFICIENT_ATTENTION
### Description
This makes a minimal change to address a crash caused by the PadFusion
pass. This pass assumed that the "pads" attribute of a child node
existed, and it now skips when it's missing.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Co-authored-by: Jeff Bloomfield <38966965+jeffbloo@users.noreply.github.com>
### Description
1. Update donwload-artifacts to flex-downloadartifacts to make it eaiser
to debug.
2. Move the native files into Gpu.Windows and Gpu-linux packages.
Onnxruntime-Gpu has dependency on them.
3. update the package validation as well
4. Add 2 stages to run E2E test for GPU.Windows and GPU.Linux
for example:
### Motivation and Context
Single Onnxruntime.Gpu Package size has already excceded the Nuget size
limit.
We split the package into some smaller packages to make them can be
published.
For compatibility, the user can install or upgrade Onnxruntime.Gpu,
which will install Gpu.Windows and Gpu.Linux automatically.
And the user can only install Gpu.Windows and Gpu.Linux directly.
### Test Link
1. In ORT_NIGHTLY
2. Install the preview version in nuget-int. (nuget source:
https://apiint.nugettest.org/v3/index.json)
---------
Co-authored-by: Scott McKay <skottmckay@gmail.com>
### Description
<!-- Describe your changes. -->
ORT-CUDAFp16 model tests were all failed
due to the latest `onnxmltools` 1.12.0 started to remove
`onnxconverter-common` out of its dependencies, which is needed by the
ep perf env to test models with CUDA EP under fp16.
Add `onnxconverter-common` dep to env to fix.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
