mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-18 21:21:17 +00:00
04030f64be
Adds QNN EP HTP shared memory allocator. The HTP shared memory allocator (`HtpSharedMemoryAllocator`) calls the rpcmem shared library (libcdsprpc.so/dll) to allocate and free memory that can be shared between HTP and CPU. The allocator can be enabled by setting QNN EP option `enable_htp_shared_memory_allocator` to `1`. `QNNExecutionProvider::CreatePreferredAllocators()` will then return an instance of `HtpSharedMemoryAllocator`. For each QNN context, we also need to register and unregister memory handles in order to use the HTP shared memory. This memory handle management is added to `QnnBackendManager`, which also manages the QNN context handles. For more information about using HTP shared memory with QNN, see: https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/htp_shared_buffer_tutorial.html#shared-buffer-tutorial Limitations: - HTP shared memory usage is only supported for graph inputs and outputs. Intermediate values are not supported. - An allocation is assigned to a single shared memory buffer. The allocator is not smart enough to have multiple allocations share a single shared memory buffer. Co-authored-by: Baiju Meswani <bmeswani@microsoft.com> |
||
|---|---|---|
| .. | ||
| alloc_kind.h | ||
| allocator.h | ||
| buffer_deleter.h | ||
| customregistry.h | ||
| data_types.h | ||
| data_types_internal.h | ||
| endian.h | ||
| execution_provider.h | ||
| float8.h | ||
| float16.h | ||
| framework_common.h | ||
| framework_provider_common.h | ||
| func_api.h | ||
| int4.h | ||
| kernel_def_builder.h | ||
| kernel_registry.h | ||
| op_kernel.h | ||
| op_kernel_context.h | ||
| op_kernel_info.h | ||
| op_node_proto_helper.h | ||
| ort_value.h | ||
| ortdevice.h | ||
| ortmemoryinfo.h | ||
| provider_options.h | ||
| provider_options_utils.h | ||
| provider_shutdown.h | ||
| run_options.h | ||
| sparse_tensor.h | ||
| stream_handles.h | ||
| tensor.h | ||
| tensor_shape.h | ||
| to_tensor_proto_element_type.h | ||