pytorch/caffe2/python
Dmytro Dzhulgakov dec116e96f PyTorch/Caffe2 tensor interop in Python (#17190)
Summary:
Because of two separate python extensions with different pybind
instances I have to go through void* conversion. Since it's hidden from
user, it's fine.

New APIs added on C2 side:
- workspace.FetchTorch('blob')
- workspace.Workspace.current.blobs['blob'].to_torch()
- workspace.FeedBlob('blob', pytorch_tensor)

Works on CPU an GPU.

The only glitches are with resizing because of variable/tensor split.
But data sharing works properly.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/17190

Reviewed By: ezyang

Differential Revision: D14163882

Pulled By: dzhulgakov

fbshipit-source-id: d18e5b8fcae026f393c842a1149e972515732de2
2019-03-04 11:34:01 -08:00
..
docs Fix several DeprecationWarning: invalid escape sequence (#15733) 2019-01-05 08:53:35 -08:00
examples Make timeout in resnet50_trainer configurable (#17058) 2019-02-13 17:03:48 -08:00
helpers
ideep Implementation convolutionTranspose operator for mkl-dnn (#12866) 2019-02-20 17:26:10 -08:00
layers add dropout during eval (#17549) 2019-02-28 23:21:29 -08:00
mint re-enable copy of python files, but be careful that the copy is only … (#14982) 2018-12-11 16:54:08 -08:00
mkl Add __init__.py so files get picked up on install (#14898) 2018-12-07 13:40:23 -08:00
modeling diagnose option: get_entry to print a whole row (#11308) 2018-09-06 21:26:30 -07:00
models Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
onnx Automatic update of fbcode/onnx to e18bb41d255a23daf368ffd62a2645db55db4c72 (#17460) 2019-02-25 11:09:08 -08:00
operator_test Enable boolean_mask, adadelta, adagrad fp16 on ROCm (#17235) 2019-02-27 10:07:36 -08:00
predictor Enhance cpu support on gloo based multi-nodes mode. (#11330) 2019-01-15 11:47:10 -08:00
rnn Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
serialized_test Export PyTorch erf to ONNX Erf and add Caffe2 Erf operator 2019-01-17 09:18:08 -08:00
test Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
trt Caffe 2: Reshape Op upgrade (#15380) 2019-01-13 22:49:40 -08:00
__init__.py Revert #17191 and #17215 that no longer apply on Windows (#17567) 2019-03-01 10:37:27 -08:00
_import_c_extension.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
allcompare_test.py
attention.py
benchmark_generator.py
binarysize.py
brew.py
brew_test.py
build.py
cached_reader.py
caffe_translator.py Fix several ResourceWarning: unclosed file (#15746) 2019-01-09 15:36:53 -08:00
caffe_translator_test.py Fix several ResourceWarning: unclosed file (#15746) 2019-01-09 15:36:53 -08:00
checkpoint.py Create class constant for string literal 'blob_names' 2018-08-24 22:11:43 -07:00
checkpoint_test.py Revert D9566744: [New Checkpoint] Kill the dummy TaskOutput when task.get_step() (#11164) 2018-08-31 22:25:57 -07:00
CMakeLists.txt Fix CMakeLists.txt for Int8 python bindings (#15047) 2018-12-11 10:48:47 -08:00
cnn.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
compatibility.py migrating deprecated calls without abc module for containers (#11515) 2018-09-13 15:09:22 -07:00
context.py
context_test.py
control.py
control_ops_grad.py DeviceScope support for CUDA and testing (#15357) 2019-01-30 18:42:12 -08:00
control_ops_grad_test.py fix auto grad summing for IfOp where intermediate output needs renaming (#14772) 2018-12-09 08:26:46 -08:00
control_ops_util.py
control_test.py
convert.py New serialization format (#12384) 2018-10-16 16:36:58 -07:00
convert_test.py New serialization format (#12384) 2018-10-16 16:36:58 -07:00
convnet_benchmarks.py
convnet_benchmarks_test.py Skip convnets benchmark in rocm CI (#17331) 2019-02-20 21:12:24 -08:00
core.py Extend Net.RunAllOnGPU() to support RecurrentNetwork op (#15713) 2019-02-08 15:48:42 -08:00
core_gradients_test.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
core_test.py Extend Net.RunAllOnGPU() to support RecurrentNetwork op (#15713) 2019-02-08 15:48:42 -08:00
crf.py Productionize CRF layer in PyText (#10362) 2018-08-22 00:25:26 -07:00
crf_predict.py Move crf in caffe2 from fb to oss (#12200) 2018-10-01 18:31:41 -07:00
crf_viterbi_test.py Move crf in caffe2 from fb to oss (#12200) 2018-10-01 18:31:41 -07:00
data_parallel_model.py Support multi-device configuration for MKL-DNN (#12856) 2019-02-20 16:57:43 -08:00
data_parallel_model_test.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
data_workers.py Fixed log message (#10874) 2018-09-05 09:55:52 -07:00
data_workers_test.py
dataio.py Fixing stop condition on composite reader (#9888) 2018-08-20 03:02:20 -07:00
dataio_test.py
dataset.py
db_file_reader.py
db_test.py
device_checker.py
dlpack.h Upgrade DLPack 2018-11-12 15:59:46 -08:00
dyndep.py
embedding_generation_benchmark.py
experiment_util.py
extension_loader.py Completely remove build_aten and use_aten (#10469) 2018-08-20 20:26:42 -07:00
functional.py Caffe2 Functional enforcing inplace output (#10797) 2018-08-23 22:42:47 -07:00
functional_test.py
fused_8bit_rowwise_conversion_ops_test.py
gradient_check_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
gradient_checker.py make the variable declaration closer to usage 2018-10-12 12:07:08 -07:00
gru_cell.py
hip_test_util.py Make CUDNN an alias of MIOPEN for HIP ops (#12278) 2018-10-24 17:07:31 -07:00
hsm_util.py
hypothesis_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
hypothesis_test_util.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
ideep_test_util.py
layer_model_helper.py parallize the dense part in event models 2018-08-22 22:40:07 -07:00
layer_model_instantiator.py
layer_parameter_sharing_test.py Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
layer_test_util.py
layers_test.py add dropout during eval (#17549) 2019-02-28 23:21:29 -08:00
lengths_reducer_fused_8bit_rowwise_ops_test.py make the threshold for acurracy more precise (#17194) 2019-02-20 13:14:11 -08:00
lengths_reducer_rowwise_8bit_ops_test.py
lstm_benchmark.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
memonger.py
memonger_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
mkl_test_util.py
model_device_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
model_helper.py Add error type to raise statement 2018-12-11 17:41:44 -08:00
model_helper_test.py keep net type info when generating model complete net (#11032) 2018-09-04 21:10:06 -07:00
modifier_context.py
mpi_python.cc
muji.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
muji_test.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
net_builder.py
net_builder_test.py
net_drawer.py Allow customization of blob node in net_drawer (#16915) 2019-02-12 15:02:50 -08:00
net_printer.py Rename cuda_gpu_id to device_id in DeviceOption (#12456) 2018-10-09 15:54:04 -07:00
net_printer_test.py
nomnigraph.py createUniqueDataNode 2018-10-31 11:16:38 -07:00
nomnigraph_test.py nomnigraph - support subgraph visualization (#13795) 2018-11-16 08:19:20 -08:00
nomnigraph_transformations.py Add transpose network pass (#13437) 2018-11-01 14:27:07 -07:00
nomnigraph_transformations_test.py Add transpose network pass (#13437) 2018-11-01 14:27:07 -07:00
normalizer.py Enable alternative LayerNorm impl in FisherGan (#12178) 2018-10-11 17:36:11 -07:00
normalizer_context.py
normalizer_test.py
numa_benchmark.py Revert D13205604: Move numa.{h, cc} to c10/util 2018-12-07 10:01:25 -08:00
numa_test.py Move numa.{h, cc} to c10/util (#15024) 2018-12-12 12:21:10 -08:00
observer_test.py
optimizer.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
optimizer_context.py
optimizer_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
optimizer_test_util.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
parallel_workers.py
parallel_workers_test.py
parallelize_bmuf_distributed_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
pipeline.py SNNTest with Data Preproc Service (#11707) 2018-09-17 21:25:49 -07:00
pipeline_test.py
predictor_constants.py
pybind_state.cc PyTorch/Caffe2 tensor interop in Python (#17190) 2019-03-04 11:34:01 -08:00
pybind_state.h Remove caffe2::Tensor copy constructor (#15416) 2019-01-18 00:31:56 -08:00
pybind_state_dlpack.cc Upgrade DLPack 2018-11-12 15:59:46 -08:00
pybind_state_dlpack.h Remove PythonOp non-CPU path and PytorchOp (#15417) 2019-01-02 16:36:37 -08:00
pybind_state_gpu.cc Fix build with cuda but no cudnn in caffe2 (#16701) 2019-02-03 22:14:51 -08:00
pybind_state_hip.cc Remaining Tensor API fixes - dims() -> sizes() (#15743) 2019-01-15 18:42:02 -08:00
pybind_state_ideep.cc Apply modernize-use-override - 2/2 2019-02-13 21:01:28 -08:00
pybind_state_int8.cc Renaming meta() to dtype() - 2/2 (#13334) 2018-10-30 18:24:30 -07:00
pybind_state_nomni.cc nomnigraph - support subgraph visualization (#13795) 2018-11-16 08:19:20 -08:00
pybind_state_registry.cc Move registry fully to c10 (#12077) 2018-09-27 03:09:54 -07:00
pybind_state_registry.h Move registry fully to c10 (#12077) 2018-09-27 03:09:54 -07:00
python_op_test.py Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
queue_util.py
record_queue.py
recurrent.py
regularizer.py
regularizer_context.py
regularizer_test.py
rnn_cell.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
schema.py Make the exception raised from "numpy.dtype(numpy.void, (INT,))" less cryptic (#16809) 2019-02-08 16:46:50 -08:00
schema_test.py Make the exception raised from "numpy.dtype(numpy.void, (INT,))" less cryptic (#16809) 2019-02-08 16:46:50 -08:00
scope.py Add EmptyNameScope to allow you jump out from current scope. (#14631) 2018-12-12 01:39:50 -08:00
scope_test.py Add EmptyNameScope to allow you jump out from current scope. (#14631) 2018-12-12 01:39:50 -08:00
session.py
session_test.py
sparse_to_dense_mask_test.py
sparse_to_dense_test.py
task.py A trivial typo fix in caffe2.python (#15907) 2019-01-17 04:57:34 -08:00
task_test.py caffe2/python/task: added __repr__ methods to all task definitions (#15250) 2018-12-17 16:02:16 -08:00
test_util.py Start unittesting our main observer (#15191) 2018-12-14 16:24:38 -08:00
text_file_reader.py
timeout_guard.py
toy_regression_test.py Enable junk fill for the default CPU allocator (#13377) 2018-11-08 00:02:37 -08:00
transformations.py
transformations_test.py Relax check on outputs (#15458) 2018-12-21 14:19:37 -08:00
tt_core.py
tt_core_test.py
utils.py Fix several ResourceWarning: unclosed file (#15746) 2019-01-09 15:36:53 -08:00
utils_test.py Convert Arguments to dictionary (#13436) 2018-11-01 14:27:05 -07:00
visualize.py
workspace.py PyTorch/Caffe2 tensor interop in Python (#17190) 2019-03-04 11:34:01 -08:00
workspace_test.py PyTorch/Caffe2 tensor interop in Python (#17190) 2019-03-04 11:34:01 -08:00