mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-06-09 00:30:53 +00:00
fc7185f060
* Various optimizations to reduce the setup and execution cost.
Cache information about the feeds and fetches, and any device copies required to execute the graph so we minimize checking for later calls to ExecuteGraph using the same input/output.
- enable use of caching in Loop and Scan
- make use of caching optional for InferenceSession::Run
- handle calls to Run with different feeds and fetches to support scenarios where there may be a truncated sequence in some calls
Take the feed names and MLValue instances as vectors so the order is deterministic.
Add unit tests
Update onnxruntime_perf_test to enable caching.
* Couple of tweaks.
Fix shared library unit test failure.
Attempt to workaround MacOS build failure due to VC++ bug around including reaching scope values in a lambda automatically.
* Rework order of init in Run so we get nice error messages about invalid feed/output names.
* Refine logic around copying MLValue using execution provider so common code can be used. Simplify the logic due to this change.
Split the paths for executing with/without cached info so we can be more const correct with how FeedsFetchesManager is passed in. This makes it clearer when a shared instance can be used due to it being const.
Cache the FeedsFetchesManager instances in the control flow nodes. They can be re-used across calls to Compute.
* Removed unused local variable to fix some builds.
* Fix build issue by cleaning up some more unused params.
* Check names when using cache entry from SessionState. Add unit test.
15 lines
420 B
C++
15 lines
420 B
C++
#include "dummy_provider.h"
|
|
|
|
#include "core/framework/kernel_registry.h"
|
|
#include "core/framework/compute_capability.h"
|
|
|
|
namespace onnxruntime {
|
|
namespace test {
|
|
|
|
std::shared_ptr<KernelRegistry> DummyExecutionProvider::GetKernelRegistry() const {
|
|
static std::shared_ptr<KernelRegistry> kernel_registry = std::make_shared<KernelRegistry>();
|
|
return kernel_registry;
|
|
}
|
|
|
|
} // namespace test
|
|
} // namespace onnxruntime
|