pytorch/caffe2/python
Kevin Matzen 4b1ebd2f65 Fast path for serializing large floating-point tensors to protobuf
Summary: Our existing serialization routines take a significant amount of time for large numpy arrays in order to verify the type of each element in the array as well as converting each element to a canonical type.  For large floating-point tensors, such as model parameters, this checking and converting takes a significant amount of time.  Adding a fast track path for just float32 arrays as this is the most common use case to worry about.

Reviewed By: akyrola

Differential Revision: D5389953

fbshipit-source-id: 26f44cb2426ea3efb849e7707b27d5485f69956c
2017-07-10 17:52:22 -07:00
..
docs Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
examples Fixed typo 2017-06-23 14:02:40 -07:00
helpers Conv frontend: checking engine and use_cudnn 2017-06-27 09:47:48 -07:00
layers Implementing Arc Cosine Layer 2017-07-10 10:10:36 -07:00
mint
mkl
modeling
models fast simple-net memonger for C++ 2017-07-06 15:17:07 -07:00
operator_test minor leaky relu bug fixes 2017-07-10 16:04:45 -07:00
predictor Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
rnn
_import_c_extension.py
attention.py Unrolled test for AttentionCell 2017-06-25 17:21:24 -07:00
brew.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
brew_test.py
caffe_translator.py Read pretrained weights using binary mode in caffe_translator.py 2017-07-08 10:17:57 -07:00
caffe_translator_test.py
checkpoint.py
checkpoint_test.py Allow tasks/execution_steps to be cloned at runtime 2017-06-20 22:32:07 -07:00
CMakeLists.txt
cnn.py
context.py
context_test.py
control.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
control_test.py
convnet_benchmarks.py brew API in convnet benchmark 2017-07-05 10:34:48 -07:00
convnet_benchmarks_test.py
core.py Fix communication_schema decoding 2017-07-02 13:04:20 -07:00
core_gradients_test.py add debug information when there is blob version mismatch 2017-06-30 16:22:46 -07:00
core_test.py single trainer hybrid device 2017-06-27 22:06:30 -07:00
crf.py
data_parallel_model.py fix a bug in the report function of Data_Parallel 2017-07-07 13:03:46 -07:00
data_parallel_model_test.py Add synchronization barrier API to data parallel model 2017-07-06 09:21:19 -07:00
data_workers.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
data_workers_test.py fix a rare race condition by initializing scratch blobs beforehand 2017-06-26 10:18:18 -07:00
dataio.py
dataio_test.py Allow tasks/execution_steps to be cloned at runtime 2017-06-20 22:32:07 -07:00
dataset.py
db_test.py
device_checker.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
dyndep.py
empty.so
experiment_util.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
extension_loader.py
gradient_check_test.py Cos, Sin, and Abs operators 2017-07-03 22:18:32 -07:00
gradient_checker.py
hsm_util.py
hypothesis_test.py Cos, Sin, and Abs operators 2017-07-03 22:18:32 -07:00
hypothesis_test_util.py Add min_satisfying_examples 2017-06-29 12:48:01 -07:00
layer_model_helper.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
layer_model_instantiator.py
layer_test_util.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
layers_test.py Implementing Arc Cosine Layer 2017-07-10 10:10:36 -07:00
load_save_test.py
lstm_benchmark.py Added flags to lstm, convnet and sparse_nn_benchmarks to print out operators 2017-06-30 23:47:04 -07:00
memonger.py fast simple-net memonger for C++ 2017-07-06 15:17:07 -07:00
memonger_test.py fast simple-net memonger for C++ 2017-07-06 15:17:07 -07:00
mkl_test_util.py
model_device_test.py Deprecate CNNModelHelper in caffe2/python/model_device_test.py 2017-06-22 15:37:17 -07:00
model_helper.py make GetComputedParams work like GetParams 2017-07-10 12:30:44 -07:00
mpi_python.cc
muji.py
muji_test.py
net_builder.py Allow tasks/execution_steps to be cloned at runtime 2017-06-20 22:32:07 -07:00
net_builder_test.py Allow tasks/execution_steps to be cloned at runtime 2017-06-20 22:32:07 -07:00
net_drawer.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
net_printer.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
net_printer_test.py Allow tasks/execution_steps to be cloned at runtime 2017-06-20 22:32:07 -07:00
optimizer.py Adding dedup aggregator options to sgd optimizer 2017-07-04 02:10:18 -07:00
optimizer_test.py
optimizer_test_util.py
parallelize_gpu_bmuf_distributed_test.py Add distributed BMUF implementation. 2017-06-21 16:18:11 -07:00
pipeline.py Enable runtime cloning of tasks. 2017-06-21 03:18:20 -07:00
predictor_constants.py
pybind_state.cc fast simple-net memonger for C++ 2017-07-06 15:17:07 -07:00
pybind_state.h fast simple-net memonger for C++ 2017-07-06 15:17:07 -07:00
pybind_state_gpu.cc
pybind_state_mkl.cc
python_op_test.py Fix some typos 2017-06-28 13:50:48 -07:00
queue_util.py
record_queue.py
recurrent.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
rnn_cell.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
schema.py IndexHash 2017-07-07 23:06:11 -07:00
schema_test.py Add __sub__ function for schema.Struct 2017-06-28 11:24:01 -07:00
scope.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
scope_test.py
session.py Allow tasks/execution_steps to be cloned at runtime 2017-06-20 22:32:07 -07:00
session_test.py
sparse_to_dense_mask_test.py
task.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
test_util.py
text_file_reader.py
timeout_guard.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
toy_regression_test.py
tt_core.py
tt_core_test.py
utils.py Fast path for serializing large floating-point tensors to protobuf 2017-07-10 17:52:22 -07:00
visualize.py Python 3 compatible integer division 2017-07-06 11:47:12 -07:00
workspace.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
workspace_test.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00