mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-06-30 03:37:44 +00:00
c964da7ea2
### Description <!-- Describe your changes. --> We are introducing the FasterTransfomer model-level integration using ORT [custom op runtime wrapper](https://github.com/microsoft/onnxruntime/pull/13427). In order to make the FT wrapper/integration work, two things need to be done: - New API `KernelInfoGetConstantInput_tensor`. (Done in this PR) During custom op kernel initialization, it needs to get the model weights (saved as node's constant inputs) ready for FT's weights instantiation. What's why we need to add this new API to make kernel info capable of getting constant inputs. - Custom op and custom op kernel to wrap FT model. (Will provide in onnxruntime extensions or inference examples) During custom op kernel initialization, it can fetch attributes from kernel info to determine which kind of FT model instance create. During custom op kernel compute/inference, it can get input/output from kernel context and then assign input/output buffers for model instance to run. |
||
|---|---|---|
| .. | ||
| cuda_ops.cu | ||
| custom_op_utils.cc | ||
| custom_op_utils.h | ||
| fns_candy_style_transfer.c | ||
| onnx_protobuf.h | ||
| test_allocator.cc | ||
| test_fixture.h | ||
| test_inference.cc | ||
| test_io_types.cc | ||
| test_model_loading.cc | ||
| test_nontensor_types.cc | ||
| test_ort_format_models.cc | ||
| test_run_options.cc | ||
| test_session_options.cc | ||
| utils.cc | ||
| utils.h | ||