saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-23 19:32:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Always call cudaSetDevice at the beginning of session::Run (#3475)

Browse source 
This is required for running multithreaded with multi-GPUs. Without it, when running in a work thread it would default to GPU 0, while CUDAExecutionProvider is assigned on other GPUs. That might cause CUDA crash when some CUDA resources is from GPU 0, while being used in GPU N>0.
This commit is contained in:
KeDengMS 2020-04-09 18:54:58 -07:00 committed by GitHub
parent a443b1b6b9
commit 739c9d4875
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
onnxruntime/core/providers/cuda/cuda_execution_provider.cc
View file

@ -198,6 +198,8 @@ void CUDAExecutionProvider::AddDeferredReleaseCPUPtr(void* p) {
}
Status CUDAExecutionProvider::OnRunStart() {
// always set CUDA device when session::Run() in case it runs in a worker thread
CUDA_RETURN_IF_ERROR(cudaSetDevice(GetDeviceId()));
auto cpu_alloc = GetAllocator(0, OrtMemTypeCPU);
// check if cudaEvents has passed for deferred release
// note that we need to take a mutex in case of multi-threaded Run()