onnxruntime/onnxruntime/core/session/ort_env.cc
ivberg 74028e4bdc
Fully dynamic ETW controlled logging for ORT and QNN logs (#20537)
### Description
Windows - Fully dynamic ETW controlled logging for ORT and QNN logs

The logging support is documented here 
-
https://onnxruntime.ai/docs/performance/tune-performance/logging_tracing.html#tracing---windows
-
https://onnxruntime.ai/docs/performance/tune-performance/profiling-tools.html#tracelogging-etw-windows-profiling

Also add support for logging ORT SessionCreation on ETW CaptureState

### Motivation and Context
The previous ETW support only worked if you enabled ETW before the
session started. There can commonly be long-lived AI inference processes
that need to be traced & debugged. This enables logging fully on the
fly.

Without this support a dev would have to end up killing a process or
stopping a service in order to get tracing. We had to do this for a
recent issue with QNN, and it was a bit painful to get the logs and it
ruined the repro.

### Testing
I tested with the following cases
- Leaving default ORT run
- Enabling ETW prior to start and leaving running for entire session +
inferences, then stopping
- Starting ORT session + inf, then enabling and stopping ETW
  - Start ORT session /w long running Inferences 
- wpr -start
[ort.wprp](e6228575e4/ort.wprp (L4))
-start
[etw_provider.wprp](e6228575e4/onnxruntime/test/platform/windows/logging/etw_provider.wprp)
  - Wait a few seconds
  - wpr -stop ort.etl
  - Inferences are still running
- Verify ONNXRuntimeLogEvent provider events are present and new
SessionCreation_CaptureState event under Microsoft.ML.ONNXRuntime
provider

Related:
#18882
#19428
2024-06-06 21:11:14 -07:00

112 lines
4 KiB
C++

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
// this file contains implementations of the C API
#include <cassert>
#include "ort_env.h"
#include "core/session/ort_apis.h"
#include "core/session/environment.h"
#include "core/session/allocator_adapters.h"
#include "core/session/user_logging_sink.h"
#include "core/common/logging/logging.h"
#include "core/framework/provider_shutdown.h"
#include "core/platform/logging/make_platform_default_log_sink.h"
using namespace onnxruntime;
using namespace onnxruntime::logging;
std::unique_ptr<OrtEnv> OrtEnv::p_instance_;
int OrtEnv::ref_count_ = 0;
onnxruntime::OrtMutex OrtEnv::m_;
OrtEnv::OrtEnv(std::unique_ptr<onnxruntime::Environment> value1)
: value_(std::move(value1)) {
}
OrtEnv::~OrtEnv() {
// We don't support any shared providers in the minimal build yet
#if !defined(ORT_MINIMAL_BUILD)
UnloadSharedProviders();
#endif
}
OrtEnv* OrtEnv::GetInstance(const OrtEnv::LoggingManagerConstructionInfo& lm_info,
onnxruntime::common::Status& status,
const OrtThreadingOptions* tp_options) {
std::lock_guard<onnxruntime::OrtMutex> lock(m_);
if (!p_instance_) {
std::unique_ptr<LoggingManager> lmgr;
std::string name = lm_info.logid;
std::unique_ptr<ISink> sink = nullptr;
if (lm_info.logging_function) {
sink = std::make_unique<UserLoggingSink>(lm_info.logging_function, lm_info.logger_param);
} else {
sink = MakePlatformDefaultLogSink();
}
auto etwOverrideSeverity = logging::OverrideLevelWithEtw(static_cast<Severity>(lm_info.default_warning_level));
sink = EnhanceSinkWithEtw(std::move(sink), static_cast<Severity>(lm_info.default_warning_level),
etwOverrideSeverity);
lmgr = std::make_unique<LoggingManager>(std::move(sink),
std::min(static_cast<Severity>(lm_info.default_warning_level), etwOverrideSeverity),
false,
LoggingManager::InstanceType::Default,
&name);
std::unique_ptr<onnxruntime::Environment> env;
if (!tp_options) {
status = onnxruntime::Environment::Create(std::move(lmgr), env);
} else {
status = onnxruntime::Environment::Create(std::move(lmgr), env, tp_options, true);
}
if (!status.IsOK()) {
return nullptr;
}
p_instance_ = std::make_unique<OrtEnv>(std::move(env));
}
++ref_count_;
return p_instance_.get();
}
void OrtEnv::Release(OrtEnv* env_ptr) {
if (!env_ptr) {
return;
}
std::lock_guard<onnxruntime::OrtMutex> lock(m_);
ORT_ENFORCE(env_ptr == p_instance_.get()); // sanity check
--ref_count_;
if (ref_count_ == 0) {
p_instance_.reset();
}
}
onnxruntime::logging::LoggingManager* OrtEnv::GetLoggingManager() const {
return value_->GetLoggingManager();
}
void OrtEnv::SetLoggingManager(std::unique_ptr<onnxruntime::logging::LoggingManager> logging_manager) {
value_->SetLoggingManager(std::move(logging_manager));
}
onnxruntime::common::Status OrtEnv::RegisterAllocator(AllocatorPtr allocator) {
auto status = value_->RegisterAllocator(allocator);
return status;
}
onnxruntime::common::Status OrtEnv::CreateAndRegisterAllocator(const OrtMemoryInfo& mem_info,
const OrtArenaCfg* arena_cfg) {
auto status = value_->CreateAndRegisterAllocator(mem_info, arena_cfg);
return status;
}
onnxruntime::common::Status OrtEnv::UnregisterAllocator(const OrtMemoryInfo& mem_info) {
return value_->UnregisterAllocator(mem_info);
}
onnxruntime::common::Status OrtEnv::CreateAndRegisterAllocatorV2(const std::string& provider_type, const OrtMemoryInfo& mem_info, const std::unordered_map<std::string, std::string>& options, const OrtArenaCfg* arena_cfg) {
return value_->CreateAndRegisterAllocatorV2(provider_type, mem_info, options, arena_cfg);
}