saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-15 21:00:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/caffe2/python
History
Dmytro Dzhulgakov a7df0e6724 Clone model net to avoid hard-coded inputs 
Summary:
Previously DPER was quite broken - we couldn't change loaders on the fly because serialized model had blob names hard-coded, e.g. "nn_loader/dense". In fact, the tests worked only by accident as both trainer and evaluator used the same loader type.

This diff does the following:
1) when writing out model, remap input blobs to be 'inputs/<field_name>'
2) when loading eval model, remap them back to the current loader

This diff uses Net.input_schema() for convenience, in particular the schema format is implicitly serialized in input blobs names. From our discussion with Andrey this type of hardcoding is actually acceptible since the schema of HiveReader on python side is inferred via the same string-parsing procedure

It also modifies model saving a bit so that we don't pollute global namespace with shape_provider net.

Overall code in mlp.py is pretty terrible. But I'd leave refactoring to xianjiec as a part of Layers migration.

Reviewed By: xianjiec

Differential Revision: D4218902

fbshipit-source-id: 6cd19f0343ec1be6ddaa3581512e61879957749e
2016-11-29 15:18:38 -08:00
..
examples remove unused function from resnet50_trainer 2016-11-29 15:18:37 -08:00
layers implement sparse nn using layers 2016-11-29 15:18:38 -08:00
mint more build updates: 2016-08-02 23:28:23 -07:00
models fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
operator_test Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
_import_c_extension.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
BREW more build fixing 2016-09-07 23:30:35 -07:00
caffe_translator.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
caffe_translator_test.py protected legacy_pad_, replace DeleteDropout with is_test=True 2016-07-29 11:44:55 -07:00
cnn.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
context.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
control.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
control_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
convnet_benchmarks.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
convnet_benchmarks_test.py chunky sync - build scripts to be written 2016-07-21 10:16:42 -07:00
core.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
core_gradients_test.py chunky sync 2016-09-06 15:55:19 -07:00
core_test.py Proper error message if passing NoneType value for kwargs 2016-11-29 15:18:36 -08:00
data_parallel_model.py use Pieter-MPI and fb.distributed 2016-11-29 15:18:36 -08:00
data_parallel_model_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
dataio.py fbsync at f5a877 2016-11-18 15:41:06 -08:00
dataio_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
dataset.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
db_test.py Fix db_test under tsan 2016-11-29 15:18:37 -08:00
device_checker.py chunky sync 2016-09-06 15:55:19 -07:00
dyndep.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
experiment_util.py use Pieter-MPI and fb.distributed 2016-11-29 15:18:36 -08:00
extension_loader.py fbsync 2016-10-07 13:08:53 -07:00
gradient_check_test.py Fix few more operators to handle empty batches correctly. 2016-11-29 15:18:37 -08:00
gradient_checker.py fbsync 2016-10-07 13:08:53 -07:00
hsm_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
hsm_util.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
hypothesis_test.py position weighted embedding 2016-11-29 15:18:35 -08:00
hypothesis_test_util.py fbsync at f5a877 2016-11-18 15:41:06 -08:00
layer_model_helper.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
layer_model_instantiator.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
load_save_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
memonger.py add untracked files 2016-07-21 11:26:41 -07:00
memonger_test.py add untracked files 2016-07-21 11:26:41 -07:00
model_device_test.py sync 2016-07-28 15:06:43 -07:00
model_helper.py add missed comma 2016-11-29 15:18:37 -08:00
muji.py fbsync 2016-10-07 13:08:53 -07:00
muji_test.py chunky sync - build scripts to be written 2016-07-21 10:16:42 -07:00
net_builder.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
net_builder_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
net_drawer.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
pipeline.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
pybind_state.cc Expose MKLMemory to the Python Feed and Fetch interface, and misc changes 2016-11-29 15:18:36 -08:00
pybind_state.h fbsync at f5a877 2016-11-18 15:41:06 -08:00
pybind_state_gpu.cc fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
pybind_state_mkl.cc Expose MKLMemory to the Python Feed and Fetch interface, and misc changes 2016-11-29 15:18:36 -08:00
python_op_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
queue_util.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
record_queue.py chunky sync 2016-09-06 15:55:19 -07:00
schema.py fbsync at f5a877 2016-11-18 15:41:06 -08:00
schema_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
scope.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
scope_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
session.py fbsync at f5a877 2016-11-18 15:41:06 -08:00
session_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
snapshot.py fbsync at f5a877 2016-11-18 15:41:06 -08:00
snapshot_test.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
sparse_to_dense_mask_test.py Fix few more operators to handle empty batches correctly. 2016-11-29 15:18:37 -08:00
task.py fbsync at f5a877 2016-11-18 15:41:06 -08:00
test_util.py chunky sync 2016-09-06 15:55:19 -07:00
text_file_reader.py fix race condition in text_file_reader.py 2016-11-29 15:18:36 -08:00
timeout_guard.py fbsync. TODO: check if build files need update. 2016-11-15 00:00:46 -08:00
toy_regression_test.py sync 2016-08-10 11:02:15 -07:00
tt_core.py sync 2016-08-10 11:02:15 -07:00
tt_core_test.py sync 2016-08-10 11:02:15 -07:00
utils.py Proper error message if passing NoneType value for kwargs 2016-11-29 15:18:36 -08:00
visualize.py chunky sync 2016-05-13 14:43:48 -07:00
workspace.py Clone model net to avoid hard-coded inputs 2016-11-29 15:18:38 -08:00
workspace_test.py check that numpy arrays are float32 when CUDA is used 2016-11-29 15:18:37 -08:00