mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-12 17:57:38 +00:00
only Flush once for the same stream in copyInputAcrossDevice() (#17303)
### Description <!-- Describe your changes. --> In CopyInputAcrossDevice() function, we assign each feed a stream to copy across device, once the copy is done, each stream will trigger the Flush() function which is undesired. Same stream should be only flushed once ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> This change is to address a perf issue of TLNGv4 inference which contains subgraph with many input feeds.
This commit is contained in:
parent
70e8c23944
commit
64f06d0b4a
1 changed files with 2 additions and 2 deletions
|
|
@ -478,9 +478,9 @@ static common::Status CopyInputsAcrossDevices(const SessionState& session_state,
|
|||
// TODO: this sync is because the graph inputs can be consumed by multiple stream,
|
||||
// but we can only place the MemCpyAsync on one of the stream. Ideally we should make
|
||||
// other stream wait on the event of the memory copy stream, instead of host sync stream.
|
||||
std::unordered_set<Stream*> visited;
|
||||
for (auto* stream : feed_streams) {
|
||||
if (stream)
|
||||
stream->Flush();
|
||||
if (stream && visited.insert(stream).second) stream->Flush();
|
||||
}
|
||||
return Status::OK();
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue