saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-21 21:52:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/include
History
Abhishek Udupa d460c01b8c
Fix skew between GPU/CPU timestamps in ORT profiler (#14004) 
### Description
This PR fixes the skew between GPU/CPU timestamps with a more reliable
algorithm.

### Motivation and Context
An earlier implementation attempted to guess the right correction to
apply, but this led to misleading profile outputs. This PR fixes this
problem by utilizing a more reliable technique to normalize GPU
timestamps. Attached are sample profile outputs and visualization
screen-grabs from a run of a transformer-based model before and after
the fix.

Before Fix:

![profile_visualization_cuda_without_fix](https://user-images.githubusercontent.com/17418420/208197234-7390d8e3-4354-4e67-93cf-958c319146ee.png)

After Fix:

![profile_visualization_cuda_with_fix](https://user-images.githubusercontent.com/17418420/208197230-3e108b82-8dfa-476b-9277-7895639a3785.png)

Profiler outputs that are rendered in the visualizations above:

[sample_outputs.zip](https://github.com/microsoft/onnxruntime/files/10249689/sample_outputs.zip)

Co-authored-by: Abhishek Udupa <abhishek.udupa@microsoft.com>
2023-01-05 11:07:26 -08:00
..
onnxruntime/core Fix skew between GPU/CPU timestamps in ORT profiler (#14004) 2023-01-05 11:07:26 -08:00