mirror of
https://github.com/saymrwulf/pytorch.git
synced 2026-05-14 20:57:59 +00:00
d522b3ca58
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/23257 Overal context: open-source BlackBoxPredictor as the entry point for inference in Caffe2 (thread safe abstraction for Caffe2 inference). This should be used in ThroughputBenchmark for the purpose of framework comparison This specific diff: There should be no harm in moving transformation code to OSS. On the advantages side we will be able to compare production Caffe2 setup with PyTorch in the most fair way via ThroughputBenchmark. This approach avoid any complicated transformation regirstries. Building those proper would be significant engineering effort as well as production risk. In the past we had SEVs related to transforms being turned off due to various refactors. Given that we don't plan to build any other significant investments into transformation logic except existing ones (like TVM and Glow), and those also relate to open-source technologies, I came up to the conclusion of moving to OSS the whole thing. Reviewed By: zrphercule Differential Revision: D16428124 fbshipit-source-id: b35deada5c015cd97b91ae12a7ea4aac53bd14b8
24 lines
789 B
C++
24 lines
789 B
C++
#pragma once
|
|
|
|
#include "caffe2/core/workspace.h"
|
|
|
|
namespace caffe2 {
|
|
|
|
/**
|
|
* This struct stores information about the inference graph which defines
|
|
* underlying math of BlackBoxPredictor. Other parts of it such as various
|
|
* threading optimizations don't belong here.
|
|
*/
|
|
struct InferenceGraph {
|
|
std::unique_ptr<NetDef> predict_init_net_def;
|
|
// shared_ptr allows to share NetDef with its operators on each of the threads
|
|
// without memory replication. Note that predict_init_net_def_ could be stored
|
|
// by value as its operators are discarded immidiatly after use (via
|
|
// RunNetOnce)
|
|
std::shared_ptr<NetDef> predict_net_def;
|
|
|
|
std::vector<std::string> input_names;
|
|
std::vector<std::string> output_names;
|
|
std::vector<std::string> parameter_names;
|
|
};
|
|
} // namespace caffe2
|