saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-29 20:14:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Doc] Add I/O binding example using onnx data type in python API summary (#22695)

Browse source 
### Description

Add I/O binding example using onnx data type in python API summary. The
API is available since 1.20 release.

### Motivation and Context

Follow up of https://github.com/microsoft/onnxruntime/pull/22306 to add
some documentation.
This commit is contained in:
Tianlei Wu 2024-11-02 12:51:37 -07:00 committed by GitHub
parent 4ffc1ff3b4
commit 120cb5a804
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 28 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/python/api_summary.rst
View file

@ -244,9 +244,36 @@ You can also bind inputs and outputs directly to a PyTorch tensor.
)
session.run_with_iobinding(binding)
You can also see code examples of this API in in the `ONNX Runtime inferences examples <https://github.com/microsoft/onnxruntime-inference-examples/blob/main/python/api/onnxruntime-python-api.py>`_.
Some onnx data type (like TensorProto.BFLOAT16, TensorProto.FLOAT8E4M3FN and TensorProto.FLOAT8E5M2) are not supported by Numpy. You can directly bind input or output with Torch tensor of corresponding data type
(like torch.bfloat16, torch.float8_e4m3fn and torch.float8_e5m2) in GPU memory.
.. code-block:: python
x = torch.ones([3], dtype=torch.float8_e5m2, device='cuda:0')
y = torch.empty([3], dtype=torch.bfloat16, device='cuda:0')
binding = session.io_binding()
binding.bind_input(
name='X',
device_type='cuda',
device_id=0,
element_type=TensorProto.FLOAT8E5M2,
shape=tuple(x.shape),
buffer_ptr=x.data_ptr(),
)
binding.bind_output(
name='Y',
device_type='cuda',
device_id=0,
element_type=TensorProto.BFLOAT16,
shape=tuple(y.shape),
buffer_ptr=y.data_ptr(),
)
session.run_with_iobinding(binding)
API Details
===========