32558134a9
### Description
Modifications to support 2GB+ checkpoint & Upgrading Flatbuffers
### Motivation and Context
This PR includes changes that will make ort handle 2GB+ checkpoints.
To do that we need to upgrade flatbuffers to 23.5.9 -
https://github.com/google/flatbuffers/pull/7945
- Modified the commitHash and the hash for the new version
- Removed the patch for rust generator's unused variable warning as it
is no longer producing this - [Check it out
here](d121e09d89/src/idl_gen_rust.cpp)
- Updated the VerifyField calls with alignment values that were
introduced in the new version.
---------
Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com>
3.9 KiB
ORT Flatbuffer Schemas
This directory contains the ORT file format schema and the generated C++ header file for the ORT file format. This directory also contains the on-device training checkpoint schema and the corresponding auto generated C++ header file for the checkpoint format.
The ORT file format schema and the on-device training checkpoint schema uses the FlatBuffers serialization library.
Please do not directly modify the generated C++ header file for the ORT file format or for the training checkpoint file format, or the generated Python binding files.
The flatbuffers compiler (flatc) is built as part of an ONNX Runtime build. It is located in the _deps/flatbuffers-build/ subdirectory of the build output directory.
e.g.
- Windows Debug build
- \build\Windows\Debug_deps\flatbuffers-build\Debug\flatc.exe
- Linux Debug build
- /build/Linux/Debug/_deps/flatbuffers-build/flatc
It is possible to use another flatc as well, e.g., from a separate installation. Note that ONNX Runtime uses FlatBuffers 23.5.26.
To update the flatbuffers schemas and generated files:
-
Modify the ORT file format schema or training checkpoint schema.
-
Run compile_schema.py to generate the C++ and Python bindings.
python onnxruntime/core/flatbuffers/schema/compile_schema.py --flatc <path to flatc> -
Update the version history and record the changes. Changes made to the ORT file format schema warrants not only updating the ort format version, but also the checkpoint version since the checkpoint schema depends on the ort format schema.
ORT FB format version history
In ort_format_version.h, see IsOrtModelVersionSupported() for the supported versions and
kOrtModelVersion for the current version.
Version 1
History begins.
Initial support for FlatBuffers that includes Model support. Graph support including Attributes, Tensors, Tensor Sequences, Maps and Sequences. Constant initializers are also supported. Constant nodes are converted to constant initializers in the ORT format.
Version 2
Support for sparse initializers. Sparse intializers are stored within ORT FlatBuffers format, which includes sparse initializers converted from a Constant node attribute.
Version 3
Support for storing graph_doc_string field in Model (ORT FlatBuffers format).
Version 4
Update kernel def hashing to not depend on ordering of type constraint types (NOT BACKWARDS COMPATIBLE).
Version 5
Remove kernel def hashes and add KernelTypeStrResolver info to replace them (LIMITED BACKWARDS COMPATIBILITY). The change to the ORT format itself is not backwards compatibility-breaking, but ORT provides limited backwards compatibility for processing older models with missing KernelTypeStrResolver info. The motivation for this update is to support additional execution providers with statically registered kernels. The original approach of using kernel def hashes was not so extensible as it required the execution provider providing hashes to be enabled at model conversion time.
Version 6
Support for float 8 types. See Float stored in 8 bits for further details about their format and usage.
Checkpoint format version history
In checkpoint_version.h, see IsCheckpointVersionSupported() for the supported versions and
kCheckpointVersion for the current version.
Version 1
Initial support for the On-Device Training Checkpoint format. The format includes support for the ModuleState (stores the module parameters), OptimizerGroups (stores the optimizer states), and PropertyBag (stores custom user properties with support for int64, float and strings).