onnxruntime/onnxruntime
kunal-vaishnavi 72a3bde330
Add GQA on CPU in LLaMA scripts (#20720)
### Description
This PR adds support for adding GroupQueryAttention (GQA) in models that
are running on CPU.

### Motivation and Context
Previously, the LLaMA scripts supported creating models that have GQA
for CUDA only. With the recently added support for [GQA on
CPU](https://github.com/microsoft/onnxruntime/pull/20299), models where
`num_attention_heads != num_key_value_heads` can now use the GQA op and
[run much faster on
CPU](https://github.com/microsoft/onnxruntime/pull/20598).
2024-05-17 23:23:57 -07:00
..
contrib_ops [CUDA] Fix SparseAttention Kernel (#20716) 2024-05-17 22:42:19 -07:00
core [js/web] Refine conv attributes (#20684) 2024-05-16 18:00:57 -07:00
python Add GQA on CPU in LLaMA scripts (#20720) 2024-05-17 23:23:57 -07:00
test MatMulNBits + Add fusion (#20587) 2024-05-16 11:00:59 -07:00
tool/etw
wasm [js/web] rewrite backend resolve to allow multiple EPs (#19735) 2024-03-15 11:47:45 -07:00
__init__.py Bump up version in main from 1.18.0 to 1.19.0 (#20489) 2024-04-29 20:21:41 -07:00
ReformatSource.ps1
ReformatSourcePython.bat
VSCodeCoverage.runsettings