saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-29 20:14:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Default kOrtSessionOptionsDisableQuantQDQ to 1 when the DML EP is registered (#15725)

Browse source 
This addresses a performance regression in some INT8 models with the
DirectML EP by defaulting OrtSessionOptionsDisableQuantQDQ to 1 when the
EP is registered.

This regression occured due to the introduction of the QDQ propagation
transformer, which is based on this session option. That transformer
maximizes the number of nodes which are executed as quantized by
logically propagating quantize operators upstream and dequantize
operators downstream. However, it does this simply by inserting QDQ
pairs, with an expectation that something will recognize sequences of
DQ->Op->Q. This logic and related L2 transformers are not currently
enabled for the DirectML EP.

This change also removes a noisy warning when the session option for
memory pattern is overriden as the DirectML EP is registered.
This commit is contained in:
Jeff Bloomfield 2023-05-01 08:26:03 -07:00 committed by GitHub
parent 10dff4f665
commit 3df3a85114
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 23 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h
View file

@ -44,7 +44,7 @@ static const char* const kOrtSessionOptionsConfigSetDenormalAsZero = "session.se
// It controls to run quantization model in QDQ (QuantizelinearDeQuantizelinear) format or not.
// "0": enable. ORT does fusion logic for QDQ format.
// "1": disable. ORT doesn't do fusion logic for QDQ format.
// Its default value is "0"
// Its default value is "0" unless the DirectML execution provider is registered, in which case it defaults to "1".
static const char* const kOrtSessionOptionsDisableQuantQDQ = "session.disable_quant_qdq";
// It controls whether to enable Double QDQ remover and Identical Children Consolidation

24
onnxruntime/core/session/inference_session.cc
View file

@ -492,15 +492,35 @@ common::Status InferenceSession::RegisterExecutionProvider(const std::shared_ptr
if (provider_type == onnxruntime::kDmlExecutionProvider) {
// DML's memory is not byte addressable and hence mem pattern doesn't work.
if (session_options_.enable_mem_pattern) {
LOGS(*session_logger_, WARNING)
LOGS(*session_logger_, INFO)
<< "Having memory pattern enabled is not supported while using the DML Execution Provider. "
<< "So disabling it for this session since it uses the DML Execution Provider.";
session_options_.enable_mem_pattern = false;
}
// Default this option to true when the DML EP is registered.
// This should be removed if QDQ is supported for DML through QDQSelectorActionTransformer and the DML EP does not
// rely on the constant folding pass for DequantizeLinear.
optional<std::string> disable_quant_qdq = session_options_.config_options.GetConfigEntry(kOrtSessionOptionsDisableQuantQDQ);
if (disable_quant_qdq == std::nullopt) {
LOGS(*session_logger_, INFO)
<< "QDQ quantization is not supported while using the DML Execution Provider. "
<< "So disabling it for this session since it uses the DML Execution Provider.";
auto st = session_options_.config_options.AddConfigEntry(kOrtSessionOptionsDisableQuantQDQ, "1");
if (!st.IsOK()) {
return st;
}
} else if (*disable_quant_qdq != "1") {
LOGS(*session_logger_, WARNING)
<< "QDQ quantization is not supported while using the DML Execution Provider. "
<< "It is enabled within session options which may result in lower performance.";
}
// Parallel execution mode does not support DML EP
if (session_options_.execution_mode != ExecutionMode::ORT_SEQUENTIAL) {
LOGS(*session_logger_, WARNING)
LOGS(*session_logger_, INFO)
<< "Parallel execution mode does not support the DML Execution Provider. "
<< "So making the execution mode sequential for this session since it uses the DML Execution Provider.";