mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-27 22:45:57 +00:00
a55b2688b6
### Description
<!-- Describe your changes. -->
The model with big Constants tensors size: Estimate size of the RWKV
model: ONNX graph (8MB), initializer tensors(200MB), constants (~5.7GB).
The `onnx.save_model` will got error due to the Constants is not output
in external data. Only the initializer tensors are output as external
data. In this change, expose parameter to support the constants in
external data. Model owner can customize the output behavior and still
keep the default behavior.
Quantize the model and output it to local, got issue due to output size
exceed 2GB even set `use_external_data_format=True`. The
`use_external_data_format` flag only outputs initializer tensors to
external data.
Use the falg `convert_attribute` flag to output all tensors to external
data.
```
def convert_model_to_external_data(
model: ModelProto,
all_tensors_to_one_file: bool = True,
location: Optional[str] = None,
size_threshold: int = 1024,
include_attribute: bool = False,
) -> None:
tensors = _get_initializer_tensors(model)
if include_attribute:
tensors = _get_all_tensors(model)
...
```
The `onnx.external_data_helper.convert_model_to_external_data` support
output the attribute to external with flag `include_attribute=True`.
However, this parameter is hide by the
`onnxruntime\quantization\onnx_model.py` and the constants(`5.7GB)
within the model will got protobuf 2GB limitation issue with default
parameters.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Fix https://github.com/microsoft/onnxruntime/issues/17944
---------
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
|
||
|---|---|---|
| .. | ||
| contrib_ops | ||
| core | ||
| python | ||
| test | ||
| tool/etw | ||
| wasm | ||
| __init__.py | ||
| ReformatSource.ps1 | ||
| ReformatSourcePython.bat | ||
| VSCodeCoverage.runsettings | ||