pytorch/test/cpp/jit/test_backend_lib.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

94 lines
3 KiB
C++
Raw Normal View History

#include <torch/csrc/jit/backends/backend.h>
[Pytorch backend delegation] Preprocess to accept (#58873) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/58873 BackenDebugInforRecorder Prior to this PR: In order to generate debug handles corresponding to the graph being lowered, backend's preprocess will call generate_debug_handles and will get map of Node*-to-debug_handles. In order to facilitate this, to_backend will own BackendDebugInfoRecorder and initialize thread local pointer to it. generate_debug_handle function will query thread local pointer to see if there is a valid BackendDebugInforRecorder for the context. If there is it will generate debug handles. After this PR: Signature of preprocess is changed such that backends have to register preprocess that accepts instance of BackendDebugInfoRecorder by reference. generate_debug_handles is no more a free function but becomes part of the API of BackendDebugInfoRecorder. Now backend's preprocess function will call generate_debug_handles on BackendDebugInfoRecorder instead of free function. Reason for this change: With RAII that initializes thread local pointer, results in a lose contract with backends, which may result in backends not storing debug information. Making it part of API results in backends having to be aware of BackendDebugInfoRecorder and explicitly chosing not to generate/store debug information if they chose to do so. Test Plan: backend tests Imported from OSS Reviewed By: jbschlosser, raziel Differential Revision: D28648613 fbshipit-source-id: c9b7e7bf0f78e87023ea7bc08612cf893b08cb98
2021-06-11 17:14:56 +00:00
#include <torch/csrc/jit/backends/backend_debug_handler.h>
#include <torch/csrc/jit/backends/backend_preprocess.h>
namespace torch {
namespace jit {
// This test JIT backend is intended to do the minimal amount of work
// necessary to test that the JIT backend registration endpoints and
// code generation are working correctly. It is not intended to
// produce numerically correct results.
Adds a bool is_available() method to the backend contract (#53068) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/53068 Adds a ```bool is_available()``` method to the backend contract: it returns ```true``` if ```compile()``` and ```execute()``` can be called; ```false``` otherwise. It is used to implement the following changes in the ```LoweredModule```: * ```compile()``` in ```__setstate__``` will run if ```is_available()```, else ```__setstate__``` throws an exception (“Backend not available.”). * ```compile()``` at ```LoweredModule``` creation will run if ```is_available()```, else a WARNING will be thrown. * ```execute()``` will only be executed if ```is_available()``` returns true; else throws an exception (“Backend not available.”). The goal of these changes is to ensure we have a well defined behaviour for the different combinations of backend availability on-host and on-target. More specifically, backends may have different capabilities to compile and/or execute the Module, depending whether this happens on-host (i.e. where the program is being written) or on-target (where the program is being executed). First of all, we know that "preprocess" always takes place, and that only happens on-host at creation time. So, we can assume that any compilation is needed/possible on-host then all of it could be pushed here. Overall, we want to ensure the following: **On host** | compile | execute | Outcome | | -- | -- | -- | | No | No | On module creation, LoweredModule is generated, with a warning (since compilation and execution can still take place on-target). On module load, throws an exception (since execution is not possible). | | No | Yes | This configuration should not be possible. This assumes the full compiler is not available, even if some work was done in preprocess the program cannot be finalized for execution. | | Yes | No | In this case, the expectation would be for is_available() to return false, and compilation logic to move into preprocess. | | Yes | Yes | All good. This is the only case that is_available() should return true. | **On target** | compile | execute | Outcome | | -- | -- | -- | | No | No | Loading the LoweredModule throws an exception. Since execution is not possible. | | No | Yes | Basically this is another instance of Yes/Yes: compilation per se may not be possible on device, which means compile() can be called without issue but it is a no-op, and thus is_available should return true. Consequently, loading the LoweredModule: Succeeds, if the preprocessed module is ready for execution. Fails with exception otherwise. | | Yes | No | This configuration should not be possible. Just putting here for completeness. | | Yes | Yes | All good. This, along with No/Yes case (because compilation is assumed to have happened on-host, so it's just another instance of Yes/Yes), are the cases where is_available() should return true. | **Refactoring existing code** This change also updates other backends (Glow) code, to implement the is_available() method to have the same behaviour as before this change (i.e. always available). This should not cause backward incompatibilities with already saved models since we're adding a new method to the PyTorchBackendInterface. Models saved with the old interface that didn't have is_available() will still find the other 2 methods in the bound object (i.e. compile and execute), and the saved LoweredModule logic will be the old one. **Future** We plan to use is_available() to implement support for fallback to the PyTorch interpreter. ghstack-source-id: 123498571 Test Plan: Added C++ (test_backend.cpp) and Python (test_backends.py) tests to validate the exceptions. Reviewed By: jackm321, spaugh, iseeyuan Differential Revision: D26615833 fbshipit-source-id: 562e8b11db25784348b5f86bbc4179aedf15e0d3
2021-03-10 08:21:34 +00:00
template <bool isAvailable>
class TestBackend : public PyTorchBackendInterface {
public:
// Constructor.
// NOLINTNEXTLINE(modernize-use-equals-default)
explicit TestBackend() {}
virtual ~TestBackend() override = default;
Adds a bool is_available() method to the backend contract (#53068) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/53068 Adds a ```bool is_available()``` method to the backend contract: it returns ```true``` if ```compile()``` and ```execute()``` can be called; ```false``` otherwise. It is used to implement the following changes in the ```LoweredModule```: * ```compile()``` in ```__setstate__``` will run if ```is_available()```, else ```__setstate__``` throws an exception (“Backend not available.”). * ```compile()``` at ```LoweredModule``` creation will run if ```is_available()```, else a WARNING will be thrown. * ```execute()``` will only be executed if ```is_available()``` returns true; else throws an exception (“Backend not available.”). The goal of these changes is to ensure we have a well defined behaviour for the different combinations of backend availability on-host and on-target. More specifically, backends may have different capabilities to compile and/or execute the Module, depending whether this happens on-host (i.e. where the program is being written) or on-target (where the program is being executed). First of all, we know that "preprocess" always takes place, and that only happens on-host at creation time. So, we can assume that any compilation is needed/possible on-host then all of it could be pushed here. Overall, we want to ensure the following: **On host** | compile | execute | Outcome | | -- | -- | -- | | No | No | On module creation, LoweredModule is generated, with a warning (since compilation and execution can still take place on-target). On module load, throws an exception (since execution is not possible). | | No | Yes | This configuration should not be possible. This assumes the full compiler is not available, even if some work was done in preprocess the program cannot be finalized for execution. | | Yes | No | In this case, the expectation would be for is_available() to return false, and compilation logic to move into preprocess. | | Yes | Yes | All good. This is the only case that is_available() should return true. | **On target** | compile | execute | Outcome | | -- | -- | -- | | No | No | Loading the LoweredModule throws an exception. Since execution is not possible. | | No | Yes | Basically this is another instance of Yes/Yes: compilation per se may not be possible on device, which means compile() can be called without issue but it is a no-op, and thus is_available should return true. Consequently, loading the LoweredModule: Succeeds, if the preprocessed module is ready for execution. Fails with exception otherwise. | | Yes | No | This configuration should not be possible. Just putting here for completeness. | | Yes | Yes | All good. This, along with No/Yes case (because compilation is assumed to have happened on-host, so it's just another instance of Yes/Yes), are the cases where is_available() should return true. | **Refactoring existing code** This change also updates other backends (Glow) code, to implement the is_available() method to have the same behaviour as before this change (i.e. always available). This should not cause backward incompatibilities with already saved models since we're adding a new method to the PyTorchBackendInterface. Models saved with the old interface that didn't have is_available() will still find the other 2 methods in the bound object (i.e. compile and execute), and the saved LoweredModule logic will be the old one. **Future** We plan to use is_available() to implement support for fallback to the PyTorch interpreter. ghstack-source-id: 123498571 Test Plan: Added C++ (test_backend.cpp) and Python (test_backends.py) tests to validate the exceptions. Reviewed By: jackm321, spaugh, iseeyuan Differential Revision: D26615833 fbshipit-source-id: 562e8b11db25784348b5f86bbc4179aedf15e0d3
2021-03-10 08:21:34 +00:00
bool is_available() override {
return isAvailable;
}
c10::impl::GenericDict compile(
c10::IValue processed,
c10::impl::GenericDict method_compile_spec) override {
auto spec =
c10::impl::toTypedDict<std::string, at::IValue>(method_compile_spec);
// Return the same string as a value for every key in method_compile_spec.
auto handles = c10::Dict<std::string, std::string>();
for (const auto& it : spec) {
handles.insert(it.key(), it.key());
}
return c10::impl::toGenericDict(handles);
}
c10::impl::GenericList execute(
c10::IValue handle,
c10::impl::GenericList inputs) override {
TORCH_INTERNAL_ASSERT(handle.isString());
TORCH_INTERNAL_ASSERT(inputs.size() > 0);
c10::List<at::Tensor> output_list;
// Implement simple accumulator and negative accumulator (?) ops. Return one
// or both of them depending on the handle to make sure multiple outputs are
// handled.
c10::IValue value = inputs[0];
at::Tensor accum = value.toTensor();
accum = accum.clone();
at::Tensor sub_accum = value.toTensor();
sub_accum = sub_accum.clone();
for (size_t i = 1, e = inputs.size(); i < e; ++i) {
value = inputs[i];
accum.add_(value.toTensor(), 1.0);
sub_accum.sub_(value.toTensor(), 1.0);
}
if (handle.toStringRef() == "accum") {
output_list.emplace_back(accum);
} else if (handle.toStringRef() == "sub_accum") {
output_list.emplace_back(sub_accum);
} else if (handle.toStringRef() == "forward") {
output_list.emplace_back(accum);
output_list.emplace_back(sub_accum);
}
return c10::impl::toList(output_list);
}
};
Add a demo backend with compiler (#52603) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/52603 This PR introduced a backend with minimum compilation capability to the to_<backend> flow. The targets are: - Demonstrate the end-to-end flow with adding a backend -> compilation -> runtime - How the backend compilation errors be surfaced to the user, with the original model's source code information. (C++ only in this PR. Python APIs will be demonstrated in a following PR.) Changes: - Compilation 1. A backend with minimum compilation features, "backend_with_compiler_demo" is added. 2. The compilation happens AOT in the ```pre_process``` function registered to this backend. 3. Compiled results are stored in a string blob for each method. They are serialized to the lowered module with ```__get_state__``` function. 4. Error message with model source code is thrown, for features not handled by the backend compiler. - Runtime 1. The compiled blob is loaded in ```__set_state__``` method. 2. The ```compile``` function of the backend pass through the AOT compiled blob. (TODO: parsing the blob to the format that the backend can understand can happen here.) 3. The ```execute``` function of the backend executes the specified method (handle). Test Plan: - ```BackendTest.TestCompiler```: the C++ end-to-end demonstration on a supported model. After compilation and running, the lowered model produces the same result as the original torchscript model. - ```BackendTest.TestCompilerNotSupport```: Demonstrate the error message from the AOT compilation for a feature not supported from the input module. The error message looks like: ``` "The node of aten::mul is not supported in this compiler. Source code: File "<string>", line 3 def forward(self, x, h): return x * h ~~~~~ <--- HERE ``` Reviewed By: raziel Differential Revision: D26593968 Pulled By: iseeyuan fbshipit-source-id: 8f264f60a0470e9f07e36fdeccbf17da6c1d7cd7
2021-02-26 19:51:29 +00:00
namespace {
c10::IValue preprocess(
const Module& mod,
[Pytorch backend delegation] Preprocess to accept (#58873) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/58873 BackenDebugInforRecorder Prior to this PR: In order to generate debug handles corresponding to the graph being lowered, backend's preprocess will call generate_debug_handles and will get map of Node*-to-debug_handles. In order to facilitate this, to_backend will own BackendDebugInfoRecorder and initialize thread local pointer to it. generate_debug_handle function will query thread local pointer to see if there is a valid BackendDebugInforRecorder for the context. If there is it will generate debug handles. After this PR: Signature of preprocess is changed such that backends have to register preprocess that accepts instance of BackendDebugInfoRecorder by reference. generate_debug_handles is no more a free function but becomes part of the API of BackendDebugInfoRecorder. Now backend's preprocess function will call generate_debug_handles on BackendDebugInfoRecorder instead of free function. Reason for this change: With RAII that initializes thread local pointer, results in a lose contract with backends, which may result in backends not storing debug information. Making it part of API results in backends having to be aware of BackendDebugInfoRecorder and explicitly chosing not to generate/store debug information if they chose to do so. Test Plan: backend tests Imported from OSS Reviewed By: jbschlosser, raziel Differential Revision: D28648613 fbshipit-source-id: c9b7e7bf0f78e87023ea7bc08612cf893b08cb98
2021-06-11 17:14:56 +00:00
const c10::Dict<IValue, IValue>& method_compile_spec,
const BackendDebugHandleGenerator& generate_debug_handles) {
return mod._ivalue();
}
constexpr auto backend_name = "test_backend";
Adds a bool is_available() method to the backend contract (#53068) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/53068 Adds a ```bool is_available()``` method to the backend contract: it returns ```true``` if ```compile()``` and ```execute()``` can be called; ```false``` otherwise. It is used to implement the following changes in the ```LoweredModule```: * ```compile()``` in ```__setstate__``` will run if ```is_available()```, else ```__setstate__``` throws an exception (“Backend not available.”). * ```compile()``` at ```LoweredModule``` creation will run if ```is_available()```, else a WARNING will be thrown. * ```execute()``` will only be executed if ```is_available()``` returns true; else throws an exception (“Backend not available.”). The goal of these changes is to ensure we have a well defined behaviour for the different combinations of backend availability on-host and on-target. More specifically, backends may have different capabilities to compile and/or execute the Module, depending whether this happens on-host (i.e. where the program is being written) or on-target (where the program is being executed). First of all, we know that "preprocess" always takes place, and that only happens on-host at creation time. So, we can assume that any compilation is needed/possible on-host then all of it could be pushed here. Overall, we want to ensure the following: **On host** | compile | execute | Outcome | | -- | -- | -- | | No | No | On module creation, LoweredModule is generated, with a warning (since compilation and execution can still take place on-target). On module load, throws an exception (since execution is not possible). | | No | Yes | This configuration should not be possible. This assumes the full compiler is not available, even if some work was done in preprocess the program cannot be finalized for execution. | | Yes | No | In this case, the expectation would be for is_available() to return false, and compilation logic to move into preprocess. | | Yes | Yes | All good. This is the only case that is_available() should return true. | **On target** | compile | execute | Outcome | | -- | -- | -- | | No | No | Loading the LoweredModule throws an exception. Since execution is not possible. | | No | Yes | Basically this is another instance of Yes/Yes: compilation per se may not be possible on device, which means compile() can be called without issue but it is a no-op, and thus is_available should return true. Consequently, loading the LoweredModule: Succeeds, if the preprocessed module is ready for execution. Fails with exception otherwise. | | Yes | No | This configuration should not be possible. Just putting here for completeness. | | Yes | Yes | All good. This, along with No/Yes case (because compilation is assumed to have happened on-host, so it's just another instance of Yes/Yes), are the cases where is_available() should return true. | **Refactoring existing code** This change also updates other backends (Glow) code, to implement the is_available() method to have the same behaviour as before this change (i.e. always available). This should not cause backward incompatibilities with already saved models since we're adding a new method to the PyTorchBackendInterface. Models saved with the old interface that didn't have is_available() will still find the other 2 methods in the bound object (i.e. compile and execute), and the saved LoweredModule logic will be the old one. **Future** We plan to use is_available() to implement support for fallback to the PyTorch interpreter. ghstack-source-id: 123498571 Test Plan: Added C++ (test_backend.cpp) and Python (test_backends.py) tests to validate the exceptions. Reviewed By: jackm321, spaugh, iseeyuan Differential Revision: D26615833 fbshipit-source-id: 562e8b11db25784348b5f86bbc4179aedf15e0d3
2021-03-10 08:21:34 +00:00
static auto cls_available =
torch::jit::backend<TestBackend<true>>(backend_name);
static auto pre_reg = backend_preprocess_register(backend_name, preprocess);
constexpr auto backend_unavailable_name = "test_backend_unavailable";
static auto cls_unavailable =
torch::jit::backend<TestBackend<false>>(backend_unavailable_name);
static auto pre_reg_unavailable =
backend_preprocess_register(backend_unavailable_name, preprocess);
} // namespace
} // namespace jit
} // namespace torch