saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-22 19:23:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

add set runtime option for C API in GenAI (#22748)

Browse source 
### Description
Add Set runtime option for C API in GenIA



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
This commit is contained in:
Abhishek Jindal 2024-11-07 14:42:09 -08:00 committed by GitHub
parent b57e2e121d
commit 09b0d8feae
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 19 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
docs/genai/api/c.md
View file

@ -414,6 +414,25 @@ A pointer to the token sequence
OGA_EXPORT const int32_t* OGA_API_CALL OgaGenerator_GetSequenceData(const OgaGenerator* generator, size_t index);
```
### Set Runtime Option
An API to set Runtime options, more parameters will be added to this generic API to support Runtime options. An example to use this API for terminating the current session would be to call the SetRuntimeOption with key as "terminate_session" and value as "1": OgaGenerator_SetRuntimeOption(generator, "terminate_session", "1")
More details on the current runtime options can be found [here](https://github.com/microsoft/onnxruntime-genai/blob/main/documents/Runtime_option.md).
#### Parameters
* Input: generator The generator on which the Runtime option needs to be set
* Input: key The key for setting the runtime option
* Input: value The value for the key provided
#### Returns
`void`
```c
OGA_EXPORT void OGA_API_CALL OgaGenerator_SetRuntimeOption(OgaGenerator* generator, const char* key, const char* value);
```
## Enums and structs
```c