saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-26 19:52:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Docs] Adding CUDA Graph docs for TRT (#19064)

Browse source 
### Description

Adding docs for the flag `trt_cuda_graph_enable`.
This commit is contained in:
Maximilian Müller 2024-01-10 02:11:48 +01:00 committed by GitHub
parent 7720b66295
commit e394a8e045
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/execution-providers/TensorRT-ExecutionProvider.md
View file

@ -99,6 +99,7 @@ There are two ways to configure TensorRT settings, either by **TensorRT Executio
| trt_profile_min_shapes | ORT_TENSORRT_PROFILE_MIN_SHAPES | string |
| trt_profile_max_shapes | ORT_TENSORRT_PROFILE_MAX_SHAPES | string |
| trt_profile_opt_shapes | ORT_TENSORRT_PROFILE_OPT_SHAPES | string |
| trt_cuda_graph_enable | ORT_TENSORRT_CUDA_GRAPH_ENABLE | bool |
> Note: for bool type options, assign them with **True**/**False** in python, or **1**/**0** in C++.
@ -179,6 +180,8 @@ TensorRT configurations can be set by execution provider options. It's useful wh
* `trt_build_heuristics_enable`: Build engine using heuristics to reduce build time.
* `trt_cuda_graph_enable`: This will capture a [CUDA graph](https://developer.nvidia.com/blog/cuda-graphs/) which can drastically help for a network with many small layers as it reduces launch overhead on the CPU.
* `trt_sparsity_enable`: Control if sparsity can be used by TRT.
* Check `--sparsity` in `trtexec` command-line flags for [details](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#trtexec-flags).