onnxruntime/onnxruntime
cloudhan 51b67fa15c
Make ROCm Attention biased+masked and biased+nomask scaling logic consistent (#14976)
The biased+masked and biased+nomask have different scaling logic in current ROCm implementation

Currently,

biased + masked:  (QK'+ bias) * scale + convert(mask)
biased + nomask:   QK' * scale + bias

which is not correct. What we want is

  QK' * scale [+ bias]

That is, bias should not be scaled.

This effectively follows
https://github.com/microsoft/onnxruntime/pull/14517/files?w=1#diff-e4768ce15a73499f584f9cd7d71adcb1ff2ed8d68ad7e496723a4775cbc35e33
2023-03-09 23:37:50 +08:00
..
contrib_ops Make ROCm Attention biased+masked and biased+nomask scaling logic consistent (#14976) 2023-03-09 23:37:50 +08:00
core fix rocBLAS extensions API issue; add batched- and strided_batched- cases (#14883) 2023-03-09 23:23:35 +08:00
python [ROCm] SkipLayerNorm: add more configs for block size; loosen constraints (#14900) 2023-03-09 22:27:01 +08:00
test [CUDA] Support decoding multihead self-attention implementation (#14848) 2023-03-08 09:17:54 -08:00
tool/etw
wasm [js/web] support flag 'optimizedModelFilePath' in session options (#14355) 2023-02-24 15:50:15 -08:00
__init__.py Add GetVersionSting API for C++, C# and Python (#14873) 2023-03-02 17:11:07 -08:00
ReformatSource.ps1
ReformatSourcePython.bat
VSCodeCoverage.runsettings