saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-30 20:18:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update index.md (#19941)

Browse source
This commit is contained in:
Nat Kershaw (MSFT) 2024-03-17 20:19:06 -07:00 committed by GitHub
parent e00773efce
commit 2e93b41fab
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/genai/index.md
View file

@ -9,10 +9,11 @@ nav_order: 6
_Note: this API is in preview and is subject to change._
Run generative AI models with ONNX Runtime.
Run generative AI models with ONNX Runtime. Source code: https://github.com/microsoft/onnxruntime-genai
This library provides the generative AI loop for ONNX models, including inference with ONNX Runtime, logits processing, search and sampling, and KV cache management.
Users can call a high level `generate()` method, or run each iteration of the model in a loop, generating one token at a time, and optionally updating generation parameters inside the loop.
It has support for greedy/beam search and TopP, TopK sampling to generate token sequences and built-in logits processing like repetition penalties. You can also easily add custom scoring.