saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-06-26 03:00:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/onnxruntime/core/framework
History
cao lei 6b73ab3e3e
Introduce reused_buffer_index_per_stream in allocation planner which will be reset after computing the reuse buffer for each stream (#19515) 
### Description
<!-- Describe your changes. -->
Introduce reused_buffer_index_per_stream in allocation planner which
will be reset after computing the reuse buffer for each stream. So if a
NodeArg is an input of several Ops across different streams and reuses
other NodeArg, the reused NodeArg won't be involved when computing the
second stream's reuse plan.


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
This is to fix https://github.com/microsoft/onnxruntime/issues/19480,
which is a crash for the scenario mentioned above.

---------

Co-authored-by: Lei Cao <leca@microsoft.com>
2024-02-22 10:19:08 -08:00
..
allocation_planner.cc Introduce reused_buffer_index_per_stream in allocation planner which will be reset after computing the reuse buffer for each stream (#19515) 2024-02-22 10:19:08 -08:00
allocation_planner.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
allocator.cc Throw if unique_ptr or array allocation fails due to SafeInt overflow (#18941) 2024-01-03 07:57:51 +10:00
allocator_stats.h
allocator_utils.cc remove AllocatorMgr class (#16509) 2023-06-28 15:43:19 -07:00
allocator_utils.h remove AllocatorMgr class (#16509) 2023-06-28 15:43:19 -07:00
arena_extend_strategy.h
bfc_arena.cc New configuration to limit the arena extension (#15983) 2023-05-25 02:19:07 -07:00
bfc_arena.h Align bins_space_ storage (#17552) 2024-02-11 19:18:26 -08:00
callback.cc
callback.h
compute_capability.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
config_options.cc Log out ORT session options (#16259) 2023-12-11 17:56:27 -08:00
config_options.h Log out ORT session options (#16259) 2023-12-11 17:56:27 -08:00
copy.cc
copy.h Add support for other data types to Split CPU kernel. (#13900) 2022-12-12 09:29:15 -08:00
customregistry.cc
data_transfer.cc Multi-stream execution support (#13495) 2022-12-15 07:39:29 -08:00
data_transfer.h Multi-stream execution support (#13495) 2022-12-15 07:39:29 -08:00
data_transfer_manager.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
data_transfer_manager.h Multi-stream execution support (#13495) 2022-12-15 07:39:29 -08:00
data_transfer_utils.h Switch GSL to MS GSL 4.0.0 (#13416) 2022-10-29 04:15:20 -07:00
data_types.cc [C#, CPP] Introduce Float16/BFloat16 support and tests for C#, C++ (#16506) 2023-07-14 10:46:52 -07:00
debug_node_inputs_outputs_utils.cc dump tensor statistics (#15761) 2023-08-16 10:53:48 -07:00
debug_node_inputs_outputs_utils.h dump tensor statistics (#15761) 2023-08-16 10:53:48 -07:00
device_stream_collection.cc clean unused parameter in ORT_UNUSED_PARAMETER (#16538) 2023-07-07 13:20:36 -07:00
device_stream_collection.h ExecutionProvider API refactor - move allocator from EP level to SessionState level and indexed by OrtDevice (#15833) 2023-06-19 17:44:45 -07:00
element_type_lists.h Introduce float 8 types (#14731) 2023-05-30 13:25:58 -07:00
empty.cc
endian_utils.cc
endian_utils.h Switch GSL to MS GSL 4.0.0 (#13416) 2022-10-29 04:15:20 -07:00
error_code.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
error_code_helper.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
ex_lib_loader.cc Support loading widechar paths on windows (#14066) 2022-12-30 16:30:11 -08:00
ex_lib_loader.h
execution_frame.cc Zhijxu/cleanup cached tensors when oom (#19306) 2024-02-21 10:41:42 +08:00
execution_frame.h Zhijxu/cleanup cached tensors when oom (#19306) 2024-02-21 10:41:42 +08:00
execution_plan_base.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
execution_provider.cc ExecutionProvider API refactor - make GenerateMetaDefId a standalone function, decouple it from EP (#18977) 2024-01-26 07:39:08 -08:00
execution_providers.h Add capturestate / rundown ETW support logging for session and provider options (#19397) 2024-02-08 11:28:05 -08:00
execution_steps.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
execution_steps.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
fallback_cpu_capability.cc [ORTModule] ATen Efficient Attention and Triton Flash Attention (#17959) 2023-10-27 10:29:27 +08:00
fallback_cpu_capability.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
feeds_fetches_manager.cc
feeds_fetches_manager.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
func_kernel.cc
func_kernel.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
fuse_nodes_funcs.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
fuse_nodes_funcs.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
graph_partitioner.cc Multi-partition support for context binary cache feature (#18865) 2024-02-01 15:04:29 -08:00
graph_partitioner.h Fix issue that the generated context cache model inputs/outputs order is not guaranteed (#19195) 2024-01-19 15:16:17 -08:00
iexecutor.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
kernel_def_builder.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
kernel_lookup.h Switch GSL to MS GSL 4.0.0 (#13416) 2022-10-29 04:15:20 -07:00
kernel_registry.cc Refine cast optimizer for safety (#15658) 2023-04-28 09:32:54 -07:00
kernel_registry_manager.cc Make session configuration options available to kernels via OpKernelInfo (#18897) 2024-01-13 10:02:43 +10:00
kernel_registry_manager.h Switch GSL to MS GSL 4.0.0 (#13416) 2022-10-29 04:15:20 -07:00
kernel_type_str_resolver.cc Introduce float 8 types (#14731) 2023-05-30 13:25:58 -07:00
kernel_type_str_resolver.h Fix compilation with newer flatbuffers (#17164) 2023-08-29 10:28:26 -07:00
kernel_type_str_resolver_utils.cc Update transpose optimization to be more QDQ aware (#18444) 2023-11-23 08:27:47 +10:00
kernel_type_str_resolver_utils.h Switch GSL to MS GSL 4.0.0 (#13416) 2022-10-29 04:15:20 -07:00
library_handles.cc Improve custom op library handle cleanup (#14099) 2023-01-04 17:56:29 -08:00
library_handles.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
math.h Switch GSL to MS GSL 4.0.0 (#13416) 2022-10-29 04:15:20 -07:00
mem_buffer.h
mem_pattern.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
mem_pattern_planner.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
memcpy.cc Multi-stream execution support (#13495) 2022-12-15 07:39:29 -08:00
memcpy.h
memory_info.cc fix memory profile build (#16177) 2023-05-31 16:08:14 -07:00
memory_info.h fix memory profile build (#16177) 2023-05-31 16:08:14 -07:00
mldata_type_utils.cc
mldata_type_utils.h
model_metadef_id_generator.cc ExecutionProvider API refactor - make GenerateMetaDefId a standalone function, decouple it from EP (#18977) 2024-01-26 07:39:08 -08:00
model_metadef_id_generator.h ExecutionProvider API refactor - make GenerateMetaDefId a standalone function, decouple it from EP (#18977) 2024-01-26 07:39:08 -08:00
murmurhash3.cc
murmurhash3.h
node_index_info.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
node_index_info.h
onnxruntime_map_type_info.cc Introduce float 8 types (#14731) 2023-05-30 13:25:58 -07:00
onnxruntime_map_type_info.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
onnxruntime_optional_type_info.cc Implement Optional Metadata support and C# test support (#15314) 2023-04-11 09:41:59 -07:00
onnxruntime_optional_type_info.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
onnxruntime_sequence_type_info.cc Implement Optional Metadata support and C# test support (#15314) 2023-04-11 09:41:59 -07:00
onnxruntime_sequence_type_info.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
onnxruntime_typeinfo.cc Introduce float 8 types (#14731) 2023-05-30 13:25:58 -07:00
onnxruntime_typeinfo.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
op_kernel.cc ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
op_kernel_context_internal.h Multi-stream execution support (#13495) 2022-12-15 07:39:29 -08:00
op_kernel_info.cc Make session configuration options available to kernels via OpKernelInfo (#18897) 2024-01-13 10:02:43 +10:00
op_kernel_type_control_utils.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
op_node_proto_helper.cc MLAS AArch64 quantized int4 Gemm kernel (#18031) 2023-11-15 09:31:54 -08:00
ort_stl_allocator.h
ort_value_name_idx_map.h
ort_value_pattern_planner.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
ort_value_pattern_planner.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
ort_value_tensor_slicer.cc
ort_value_tensor_slicer.h
partial_graph_execution_state.cc Enable conditional optimization automatically (#15885) 2023-05-23 13:08:05 +08:00
partial_graph_execution_state.h Avoid the lock for device stream impact the cpu build (#14131) 2023-01-05 09:01:33 -08:00
prepacked_weights.cc
prepacked_weights.h Refactor prepack buffer code (#16280) 2023-06-08 14:42:02 -07:00
prepacked_weights_container.cc remove AllocatorMgr class (#16509) 2023-06-28 15:43:19 -07:00
prepacked_weights_container.h
print_tensor_statistics_utils.h dump tensor statistics (#15761) 2023-08-16 10:53:48 -07:00
print_tensor_utils.h Enable -Wshorten-64-to-32 warning if available. (#16524) 2023-07-07 08:11:44 -07:00
program_region.h Multi-stream execution support (#13495) 2022-12-15 07:39:29 -08:00
random_generator.cc
random_generator.h
random_seed.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
random_seed.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
run_options.cc
sequential_execution_plan.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
sequential_executor.cc ORT ETW dynamic logging that improves ORT diagnosability & performance (#18882) 2024-01-11 12:43:27 -08:00
sequential_executor.h Fix memory profiler (#14695) 2023-02-23 18:05:21 +08:00
session_options.cc Improve custom op library handle cleanup (#14099) 2023-01-04 17:56:29 -08:00
session_options.h Disable per-session thread pool for web (#18480) 2024-01-10 18:45:49 -08:00
session_state.cc Add prepacked weights container to subgraphs (#17671) 2023-09-26 12:01:41 -07:00
session_state.h Fix compilation with newer flatbuffers (#17164) 2023-08-29 10:28:26 -07:00
session_state_utils.cc Fix a bug in SaveInputOutputNamesToNodeMapping function (#18456) 2023-11-15 14:51:42 -08:00
session_state_utils.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
simple_tensor_allocator.cc ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
simple_tensor_allocator.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
sparse_tensor.cc Throw if unique_ptr or array allocation fails due to SafeInt overflow (#18941) 2024-01-03 07:57:51 +10:00
sparse_utils.cc Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
sparse_utils.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
stream_execution_context.cc ORT ETW dynamic logging that improves ORT diagnosability & performance (#18882) 2024-01-11 12:43:27 -08:00
stream_execution_context.h Enable -Wshorten-64-to-32 warning if available. (#16524) 2023-07-07 08:11:44 -07:00
tensor.cc Handle string initializers in constant folding (#17422) 2023-09-27 21:15:58 +10:00
tensor_allocator.cc ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
tensor_allocator.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
tensor_allocator_with_mem_pattern.h ExecutionProvider API refactor - replace OrtMemoryInfo with OrtDevice (#15618) 2023-05-01 10:06:00 -07:00
tensor_external_data_info.cc
tensor_external_data_info.h
tensor_shape.cc Fix typo in the TensorShape (#17813) 2023-12-01 16:48:55 -08:00
tensor_type_and_shape.cc Custom op shape inference API (#17737) 2023-10-13 12:57:42 -07:00
tensor_type_and_shape.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
tensorprotoutils.cc [js] enable external data loading for ort-web (#19087) 2024-01-12 19:24:24 -08:00
tensorprotoutils.h Handle string initializers in constant folding (#17422) 2023-09-27 21:15:58 +10:00
TensorSeq.h Enable Opset11 Sequence Ops on DirectML, and make the CPU implementations agnostic to backend EP (#14442) 2023-02-21 18:08:28 -08:00
transform_layout_functions.h Re-organize the transpose optimization and layout transformation files. (#16246) 2023-07-07 08:24:47 +10:00
transpose_helper.cc Parallelize Transpose (#16854) 2023-08-07 14:25:53 -07:00
transpose_helper.h Parallelize Transpose (#16854) 2023-08-07 14:25:53 -07:00
tunable.h Improve tunable verbose log (#17328) 2023-10-31 13:10:21 +08:00
tuning_context.h remove gsl to make status.h independent from gsl (#17402) 2023-09-13 21:47:43 -07:00
tuning_context_impl.h Run clang-format in CI (#15524) 2023-04-18 09:26:58 -07:00
tuning_results.h Add TuningContext for TunableOp (#14557) 2023-02-10 14:27:43 +08:00
utils.cc [ORTModule] ATen Efficient Attention and Triton Flash Attention (#17959) 2023-10-27 10:29:27 +08:00
utils.h [ORTModule] ATen Efficient Attention and Triton Flash Attention (#17959) 2023-10-27 10:29:27 +08:00