The dnnl_binary ops need the memory format to match the format expected by
Onnxruntime. If the memory format of the inputs do not match each other
there will be an error in the calculated results.
Additionally, since the code manually pads the tensor dimensions for broadcasting
the inputs are expected to be in Onnxruntimes format.
Since detecting and reordering the memory to Ort format matches what was previously
done for the Reshape op the code was moved from dnnl_reshape to
dnnl_subgraph_primitive under the name GetMemoryInOrtFormat.
One small additional change made to the capability code log to also print the
percentage of nodes run by the dnnl execution provider.
Signed-off-by: George Nash <george.nash@intel.com>
```
Component for aggressive decoding. Find the bifurcation index of predicted tokens, between source tokens,
starting from previous suffix match index, and predicted tokens.
Concat predicted tokens, starting from bifurcation index, to the back
of current tokens. This forms the output tokens.
Detect suffix match index in source tokens, between source tokens and output tokens.
Detection is based on finding the appearances of last n-gram in output tokens
in source tokens.
A match is considered found if source tokens contain a single matching n-gram.
Return the index of the start of the n-gram in source tokens.
No matching if found if src tokens contain multiple or zero matching n-grams. Return -1.
```
Add kernels for QLinearConv with symmetric quantized filter, e.g., filter type is int8 and zero point of filter is 0. This PR includes kernels for avx2, avxvnni, avx512 and avx 512 vnni. Will adds kernels for ARM64 in following PR.
Kernels uses direct input buffer directly for pointwise, and in-direct buffer for depthwise and non-group conv.
The advantages of those new kernels are:
no need to compute the sum of each pixel output image, and sum/offset of filter can be combined with bias.
with in-direct buffer, im2col returns an array of buffer pointers instead of memcpy'ing the original data. This saves memcpy time and reduces the size of the intermediate buffer needed to hold the im2col transform. In the future, will compute im2col ahead of time for input with fixed input size.
* Exception when duplicated autograd.Function name detected
* reorder a bit for a bittle bit better perf
* fix a bug in previous PR :(
* correct the error message a bit
* Support opset-13 for squeeze, unsqueeze, maxpool, pad, cast, clip
* merge master and update a operators.md
* resolve comment. revise pool and cast kernel implementation.
* skip fusion when clip min and max is not in initializer
* re-hipify all rocm EP sources
* fix all other files affected by re-hipify
* add cuda_provider_factory.h to amd_hipify.py
* do not use cudnn_conv_algo_search in ROCm EP, missing reduce min registration
* Fix ReduceConsts template specialization introduced in #9101.
Fixes the error when building for ROCm 4.3.1:
error: too many template headers for onnxruntime::rocm::ReduceConsts<__half>::One (should be 0)
* fix flake8 error in amd_hipify.py
* speed up hipify with concurrent.futures
* flake8 fix in amd_hipify.py
* removing warnings which are causing errors from torch and changing flags for Windows
* adding MKL library resolution and comments
* cleaning up the code
* fixing onnxruntime_python file for windows build
* fix the include order to aovid the python_d.lib issue on win debug build
* changes for warnings, typos and other comments
* merge conflict
* adding fix for mkl library error
* Revert "adding fix for mkl library error"
This reverts commit 73b87c73c2.
* fix for dll path for windows
* typo for dll path
Co-authored-by: Cheng Tang <chenta@microsoft.com>
* Added null check before filling tensor with a value. Passing optional parameter for EinSum in case of MatMul type
* Addressed comment on the PR
Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com>
* The serialization can be very heavy for large models
* Only use the serialized model check on compatible onnx versions
* onnx version >= 1.10.0 supports serialized model check
Signed-off-by: IceTDrinker <49040125+IceTDrinker@users.noreply.github.com>
* Transpose for DNNL EP
Transpose reorders the memory to the right format but has the wrong
dimentions and memory::format. So a new memory descriptor is created
that points to the reordered memory. However, that memory is in a
different location than the output expects. An extra parameter was
added to the SetMemory to specify when memory must be copied if it is
output from the subgraph.
Signed-off-by: George Nash <george.nash@intel.com>
* Implementation of Reshape op for dnnl ep
Signed-off-by: George Nash <george.nash@intel.com>
* Add Pow op to dnnl execution provider
This Pow is limited; the exponent must be scaler or a one dimensional tensor
e.g. a tensor with only a single element. The exponent must also be a constant
initializer since it is only read when the primitive is created. OneDNN does
not have any way to change the exponent after the primitive is created.
The GraphViewer is now passed into the NodeCapability code since the GraphViewer
is needed to find out if an input is a constant initializer.
The unit tests for "Pow" did not make the exponent a constant initializer. To
help verify the dnnl execution providers Pow function a version of the Pow unit
tests was created for the DNNL execution provier that made the exponent a constant
initializer.
Signed-off-by: George Nash <george.nash@intel.com>
* Add LeakyRelu to DNNL execution provider
LeakyRelu was added to the dnnl elementwise ops.
In the elementwise op the GetAlpha method was modified
to take the default value for Alpha as a parameter instead
of reading it from a member varable. This felt like it would
be less likely to cause programer error.
Signed-off-by: George Nash <george.nash@intel.com>
* Switch dnnl_code_capability DataTypes from strings to enums
Signed-off-by: George Nash <george.nash@intel.com>
* Update DnnlSubgraphPrimitive.GetMemory function input
This updates the GetMemory member function to take DnnlTensor
instead of a string. This was done for two reasons. Every
time the function was called it was always done using
DnnlTensor.Name() this will reduce the code repition. We never
called it using a saved string.
This also makes the function inputs more closely match the
GetMemoryAndReshape function. Making less differences between
member functions.
Signed-off-by: George Nash <george.nash@intel.com>
* resolve the provider options before create training session in orttrainer
* Update orttraining/orttraining/python/orttraining_pybind_common.h
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
* support clear the training ep instance pool
* fix status error
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
* bias dropout improvement
* add transform case for same shape case
* combine kernel
* merge with vectorized kernel
* use "has_same_shape_bias"
* minor: a "N % 4 != 0" case
* add op UT for has_same_shape_bias
* address comments; add param case for 1d bias;
add param case tests for 1d and same-shape bias
* rewrite logic condition
Co-authored-by: Peng Wang <pengwa@microsoft.com>
