ad182479b0
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/69251 This adds some actual documentation for deploy, which is probably useful since we told everyone it was experimentally available so they will probably be looking at what the heck it is. It also wires up various compoenents of the OSS build to actually work when used from an external project. Differential Revision: D32783312 D32783312 Test Plan: Imported from OSS Reviewed By: wconstab Pulled By: suo fbshipit-source-id: c5c0a1e3f80fa273b5a70c13ba81733cb8d2c8f8 |
||
|---|---|---|
| .. | ||
| example | ||
| interpreter | ||
| unity | ||
| .gitignore | ||
| CMakeLists.txt | ||
| deploy.cpp | ||
| deploy.h | ||
| elf_file.cpp | ||
| elf_file.h | ||
| environment.h | ||
| interactive_embedded_interpreter.cpp | ||
| loader.cpp | ||
| loader.h | ||
| mem_file.h | ||
| noop_environment.h | ||
| path_environment.cpp | ||
| path_environment.h | ||
| README.md | ||
| remove_dt_needed.cpp | ||
| test_deploy.cpp | ||
| test_deploy_from_python.py | ||
| test_deploy_gpu.cpp | ||
| test_deploy_lib.cpp | ||
| test_deploy_missing_interpreter.cpp | ||
| test_deploy_python.py | ||
| test_deploy_python_ext.cpp | ||
Torch Deploy
This is an experimental feature to embed multiple python interpreters inside the torch library, providing a solution to the 'GIL problem' for multithreading with the convenience of python and eager or torchscripted pytorch programs.
libinterpreter
This is an internal library used behind the scenes to enable multiple python interpreters in a single deploy runtime. libinterpreter.so is DLOPENed multiple times by the deploy library. Each copy of libinterpreter exposes a simple interpreter interface but hides its python and other internal symbols, preventing the different python instances from seeing each other.
CPython build
Torch Deploy builds CPython from source as part of the embedded python interpreter. CPython has a flexible build system that builds successfully with or without a variety of dependencies installed - if missing, the resulting CPython build simply omits optional functionality, meaning some stdlib modules/libs are not present.
Currently, the torch deploy build setup assumes the full CPython build is present. This matters because there is a hardcoded list of python stdlib modules that are explicitly loaded from the embedded binary at runtime.
rebuilding CPython after installing missing dependencies
Becuase CPython builds successfully when optional dependencies are missing, the cmake wrapper currently doesn't know if you need to rebuild CPython after adding missing dependencies (or whether dependencies were missing in the first place).
To be safe, install the complete list of dependencies for CPython for your platform, before trying to build torch with USE_DEPLOY=1.
If you already built CPython without all the dependencies and want to fix it, just blow away the CPython folder under torch/csrc/deploy/third_party, install the missing system dependencies, and re-attempt the pytorch build command.