From 55edff104f892c0eb460d2e7ecbddec0669a14b8 Mon Sep 17 00:00:00 2001
From: Yifan Li <109183385+yf711@users.noreply.github.com>
Date: Wed, 17 Apr 2024 13:45:06 -0700
Subject: [PATCH] [TensorRT EP] Update instruction of `user_compute_stream`
 (#20343)

### Description
<!-- Describe your changes. -->
Update instruction and add sample code

### Preview the change:

https://yf711.github.io/onnxruntime/docs/execution-providers/TensorRT-ExecutionProvider.html#execution-provider-options

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
---
 .../TensorRT-ExecutionProvider.md             | 28 ++++++++++++++++++-
 1 file changed, 27 insertions(+), 1 deletion(-)

diff --git a/docs/execution-providers/TensorRT-ExecutionProvider.md b/docs/execution-providers/TensorRT-ExecutionProvider.md
index d616c87ca1..73f5e98342 100644
--- a/docs/execution-providers/TensorRT-ExecutionProvider.md
+++ b/docs/execution-providers/TensorRT-ExecutionProvider.md
@@ -114,8 +114,34 @@ TensorRT configurations can be set by execution provider options. It's useful wh
     * Default value: 0
 
 * `user_compute_stream`: Defines the compute stream for the inference to run on. It implicitly sets the `has_user_compute_stream` option. It cannot be set through `UpdateTensorRTProviderOptions`, but rather `UpdateTensorRTProviderOptionsWithValue`.
+
     * This cannot be used in combination with an external allocator.
-    * This can not be set using the python API.
+
+    * This can also be set using the python API.  
+
+      * i.e The cuda stream captured from pytorch can be passed into ORT-TRT. Click below to check sample code: 
+      
+        <Details>
+
+        ```python
+        import onnxruntime as ort
+        import torch
+        ...
+        sess = ort.InferenceSession('model.onnx')
+        if torch.cuda.is_available():
+            s = torch.cuda.Stream()
+            option = {"user_compute_stream": str(s.cuda_stream)}
+            sess.set_providers(["TensorrtExecutionProvider"], [option])
+            options = sess.get_provider_options()
+      
+            assert "TensorrtExecutionProvider" in options
+            assert options["TensorrtExecutionProvider"].get("user_compute_stream", "") == str(s.cuda_stream)
+            assert options["TensorrtExecutionProvider"].get("has_user_compute_stream", "") == "1"
+        ...
+        ```
+        </Details>
+    * To take advantage of user compute stream, it is recommended to use [I/O Binding](https://onnxruntime.ai/docs/api/python/api_summary.html#data-on-device) to bind inputs and outputs to tensors in device.
+
 
 * `trt_max_workspace_size`: maximum workspace size for TensorRT engine.
     * Default value: 1073741824 (1GB).