saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-06-06 00:03:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Relax check for optimized model saving (#2291)

Browse source 
So user may save model with layout optimization.
This commit is contained in:
KeDengMS 2019-10-30 21:48:49 -07:00 committed by GitHub
parent ecfbb1bb99
commit ff64d1f55b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 8 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/ONNX_Runtime_Graph_Optimizations.md
View file

@ -55,7 +55,7 @@ All optimizations can be performed either online or offline. In online mode, whe
**Notes**:
* When running in offline mode, make sure to use the exact same options (e.g., execution providers, optimization level) and hardware as the target machine that the model inference will run on (e.g., you cannot run a model pre-optimized for a GPU execution provider on a machine that is equipped only with CPU).
* When layout optimizations are enabled, the offline mode cannot be used.
* When layout optimizations are enabled, the offline mode can only be used on compatible hardware to the environment when the offline model is saved. For example, if model has layout optimized for AVX2, the offline model would require CPUs that support AVX2.
## Usage

12
onnxruntime/core/session/inference_session.cc
View file

@ -608,12 +608,14 @@ common::Status InferenceSession::Initialize() {
ORT_RETURN_IF_ERROR_SESSIONID_(graph.Resolve());
if (!session_options_.optimized_model_filepath.empty()) {
if (session_options_.graph_optimization_level < TransformerLevel::Level3) {
// Serialize optimized ONNX model.
ORT_RETURN_IF_ERROR_SESSIONID_(Model::Save(*model_, session_options_.optimized_model_filepath));
} else {
// Serialize optimized ONNX model.
ORT_RETURN_IF_ERROR_SESSIONID_(Model::Save(*model_, session_options_.optimized_model_filepath));
if (session_options_.graph_optimization_level >= TransformerLevel::Level3) {
LOGS(*session_logger_, WARNING) << "Serializing Optimized ONNX model with Graph Optimization"
" level greater than 2 is not supported.";
" level greater than ORT_ENABLE_EXTENDED. The generated"
" model may contain hardware and execution provider specific"
" optimizations, and should only be used in the same environment"
" the model was optimized for.";
}
}

9
onnxruntime/test/framework/inference_session_test.cc
View file

@ -415,15 +415,6 @@ TEST(InferenceSessionTests, TestModelSerialization) {
InferenceSession session_object_emptyValidation{so_opt, &DefaultLoggingManager()};
ASSERT_TRUE(session_object_emptyValidation.Load(test_model).IsOK());
ASSERT_TRUE(session_object_emptyValidation.Initialize().IsOK());
// Assert that level 3 optimization doesn't result in serialized model.
so_opt.optimized_model_filepath = ToWideString("ShouldNotSerialize");
so_opt.graph_optimization_level = TransformerLevel::Level3;
InferenceSession session_object_Level3Test{so_opt, &DefaultLoggingManager()};
ASSERT_TRUE(session_object_Level3Test.Load(test_model).IsOK());
ASSERT_TRUE(session_object_Level3Test.Initialize().IsOK());
std::ifstream model_fs_Level3(so_opt.optimized_model_filepath, ios::in | ios::binary);
ASSERT_TRUE(model_fs_Level3.fail());
}
#ifdef ORT_RUN_EXTERNAL_ONNX_TESTS