mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-24 19:43:35 +00:00
87b14ac7e4
### Release backward inputs per static graph ref count
For the output buffer marked as external output:
1. Remove the additional ref count we used for avoiding reusing buffer.
Instead, when we find reuse input/output buffer, we will make sure the
reused buffer not not generated by nodes that has external outputs.
2. Remove the ref count of pybind feed inputs, which exists all the time
until the run_backward completed. Instead, passing a mutuble feeds, and
we clean the feeds vector once that is copied into session states and
not needed any more before run the graph sequencentially.
#### Before the change:
One of the backward inputs is 3.9GB, it lives until the backward ends.
#### With the change:
The 3.9GB is released when the last node depending on that tensor
completed.
Be noted: the peak did not change though, we have more work to do to
reduce on the peak.
#### Others
It is found there are few tests that were updated to use incorrect
expected values in previous code refactoring
|
||
|---|---|---|
| .. | ||
| alloc_kind.h | ||
| allocator.h | ||
| buffer_deleter.h | ||
| customregistry.h | ||
| data_types.h | ||
| data_types_internal.h | ||
| endian.h | ||
| execution_provider.h | ||
| float8.h | ||
| float16.h | ||
| framework_common.h | ||
| framework_provider_common.h | ||
| func_api.h | ||
| int4.h | ||
| kernel_def_builder.h | ||
| kernel_registry.h | ||
| op_kernel.h | ||
| op_kernel_context.h | ||
| op_kernel_info.h | ||
| op_node_proto_helper.h | ||
| ort_value.h | ||
| ortdevice.h | ||
| ortmemoryinfo.h | ||
| provider_options.h | ||
| provider_options_utils.h | ||
| provider_shutdown.h | ||
| run_options.h | ||
| sparse_tensor.h | ||
| stream_handles.h | ||
| tensor.h | ||
| tensor_shape.h | ||
| to_tensor_proto_element_type.h | ||