saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-15 21:00:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/caffe2/python
History
Ansha Yu 98aebed88e Refactor tests part 1 (#11350) 
Summary:
Followup to [the serialized test framework](https://github.com/pytorch/pytorch/pull/10594)

Round 1 for refactoring tests, starting alphabetically. I added some functionality, so I wanted to send out some of these initial changes sooner.

I'm skipping all tests that don't explicitly call assertReferenceChecks. Some tests directly call np.allclose, and others are simply TestCase (rather than HypothesisTestCase).

1. Start alphabetically producing serialized outputs for test functions, annotating those we want to include with `serialized_test_util.given`. So far I've only added one test per operator, but this already does seem to add quite a few tests.
2. Add functionality to allow us to generate outputs using pytest by adding pytest argument options. This allows us to skip adding a `__main__` function to quite a few tests.
3. Catch any exceptions generating the gradient operator and skip serializing/reading it, since certain operators don't have gradients.
4. Add functionality to better handle jagged array inputs, which numpy doesn't handle very well. We simply explicitly do the conversion to dtype=object.
5. Make only one file per test function, rather than 4, to reduce the number of files in the github repo.

I also noticed that there is some hypothesis handling that makes `serialized_test_util.given` not compatible with adding more hypothesis decorators on top. For example, there are tests that do
```
settings(...)
given(...)
def test_my_stuff(...)
```
But there is a hypothesis handler that explicitly checks that `given` is called below `settings`, so we cannot refactor this to `serialized_test_util.given`. I've just avoided decorating these kinds of tests for now, I hope that's alright.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11350

Reviewed By: houseroad

Differential Revision: D9693857

Pulled By: ajyu

fbshipit-source-id: a9b4279afbe51c90cf2025c5ac6b2db2111f4af7
2018-09-18 10:42:10 -07:00
..
docs Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
examples Add resnext model to OSS (#11468) 2018-09-12 15:59:20 -07:00
helpers [Caffe2] Add full impl of GroupNorm (#7058) 2018-04-29 11:26:40 -07:00
ideep Enable Detectron model inference for CPU and MKL-DNN paths (#10157) 2018-08-29 15:11:01 -07:00
layers Support FP16 sparse lookup (#11674) 2018-09-14 02:40:08 -07:00
mint Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
mkl enable_mkl support for resnet18+lstm model 2018-09-12 18:56:46 -07:00
modeling diagnose option: get_entry to print a whole row (#11308) 2018-09-06 21:26:30 -07:00
models Add resnext model to OSS (#11468) 2018-09-12 15:59:20 -07:00
onnx migrating deprecated calls without abc module for containers (#11515) 2018-09-13 15:09:22 -07:00
operator_test Refactor tests part 1 (#11350) 2018-09-18 10:42:10 -07:00
predictor Export uint8 tensors as byte string in mobile_exporter and add GivenTensorByteStringToUInt8FillOp (#10385) 2018-08-15 14:26:50 -07:00
rnn Fix a typo 2018-03-30 21:00:44 -07:00
serialized_test Refactor tests part 1 (#11350) 2018-09-18 10:42:10 -07:00
test Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
trt ONNXIFI transform (#9569) 2018-07-20 15:09:59 -07:00
__init__.py caffe2::DeviceType -> at::DeviceType (#11254) 2018-09-05 16:28:09 -07:00
_import_c_extension.py Completely remove build_aten and use_aten (#10469) 2018-08-20 20:26:42 -07:00
allcompare_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
attention.py [Caffe2] Update elementwise ops to support numpy style boradcast (#8070) 2018-06-05 15:49:16 -07:00
benchmark_generator.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
binarysize.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
brew.py [Caffe2] Changes done inside Facebook (#6378) 2018-04-10 21:11:43 -07:00
brew_test.py Move tanh function to math (#9328) 2018-07-11 13:59:50 -07:00
build.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
cached_reader.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
caffe_translator.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
caffe_translator_test.py Remove caffe1 specific proto (#10380) 2018-08-10 11:10:26 -07:00
checkpoint.py Create class constant for string literal 'blob_names' 2018-08-24 22:11:43 -07:00
checkpoint_test.py Revert D9566744: [New Checkpoint] Kill the dummy TaskOutput when task.get_step() (#11164) 2018-08-31 22:25:57 -07:00
CMakeLists.txt Add nomnigraph bindings 2018-08-28 12:40:16 -07:00
cnn.py [Caffe2] Changes done inside Facebook (#6378) 2018-04-10 21:11:43 -07:00
compatibility.py migrating deprecated calls without abc module for containers (#11515) 2018-09-13 15:09:22 -07:00
context.py Resolve name conflict of ContextManager (#7244) 2018-06-22 00:41:51 -04:00
context_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
control.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
control_ops_grad.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
control_ops_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
control_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
convnet_benchmarks.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
convnet_benchmarks_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
core.py parallize the dense part in event models 2018-08-22 22:40:07 -07:00
core_gradients_test.py [Caffe2] Update elementwise ops to support numpy style boradcast (#8070) 2018-06-05 15:49:16 -07:00
core_test.py Revert D9566744: [New Checkpoint] Kill the dummy TaskOutput when task.get_step() (#11164) 2018-08-31 22:25:57 -07:00
crf.py Productionize CRF layer in PyText (#10362) 2018-08-22 00:25:26 -07:00
data_parallel_model.py Fix typo in data_parallel_model 2018-09-04 13:15:31 -07:00
data_parallel_model_test.py Add unit test for Parallel Spatial Batch Normalization (#11098) 2018-09-06 14:26:56 -07:00
data_workers.py Fixed log message (#10874) 2018-09-05 09:55:52 -07:00
data_workers_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
dataio.py Fixing stop condition on composite reader (#9888) 2018-08-20 03:02:20 -07:00
dataio_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
dataset.py Update from facebook (#7855) 2018-05-29 11:38:02 -07:00
db_file_reader.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
db_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
device_checker.py Update from facebook (#7451) 2018-05-10 23:14:27 -07:00
dlpack.h
dyndep.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
embedding_generation_benchmark.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
experiment_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
extension_loader.py Completely remove build_aten and use_aten (#10469) 2018-08-20 20:26:42 -07:00
functional.py Caffe2 Functional enforcing inplace output (#10797) 2018-08-23 22:42:47 -07:00
functional_test.py Add support for specifying device_option in Functional (#9619) 2018-07-24 14:41:59 -07:00
fused_8bit_rowwise_conversion_ops_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
gradient_check_test.py [Caffe2] Fix gradient_check on in-place ops (#8828) 2018-06-25 15:25:56 -07:00
gradient_checker.py framework for committed serialized tests (#10594) 2018-08-30 22:41:46 -07:00
gru_cell.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
hsm_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
hypothesis_test.py 64B align for avx512 (#11748) 2018-09-17 14:08:31 -07:00
hypothesis_test_util.py framework for committed serialized tests (#10594) 2018-08-30 22:41:46 -07:00
ideep_test_util.py [feature request] [Caffe2] Enable MKLDNN support for inference (#6699) 2018-04-22 21:58:14 -07:00
layer_model_helper.py parallize the dense part in event models 2018-08-22 22:40:07 -07:00
layer_model_instantiator.py [caffe2] Fbcode to GitHub sync (#6208) 2018-04-02 16:35:27 -07:00
layer_parameter_sharing_test.py [layer model] remove duplicated init ops 2018-03-27 18:10:39 -07:00
layer_test_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
layers_test.py fbshipit-source-id: ba600fcd2b5cefc7621357bdeb05e24cea02e5af 2018-06-27 04:50:56 -07:00
lengths_reducer_fused_8bit_rowwise_ops_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
lengths_reducer_rowwise_8bit_ops_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
lstm_benchmark.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
memonger.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
memonger_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
mkl_test_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
model_device_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
model_helper.py fix empty net type (#11286) 2018-09-06 11:10:01 -07:00
model_helper_test.py keep net type info when generating model complete net (#11032) 2018-09-04 21:10:06 -07:00
modifier_context.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
mpi_python.cc Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
muji.py [Caffe2] Support non peer access in muji and fix bug when reduced_affix is empty (#6896) 2018-06-05 03:14:43 +08:00
muji_test.py [Caffe2] Support non peer access in muji and fix bug when reduced_affix is empty (#6896) 2018-06-05 03:14:43 +08:00
net_builder.py Update from Facebook (#6692) 2018-04-17 23:36:40 -07:00
net_builder_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
net_drawer.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
net_printer.py [DT] [38/n] Rename add_stop_signal to add_stop_condition (#6825) 2018-04-23 10:39:37 -07:00
net_printer_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
nomnigraph.py Expose annotations (#11649) 2018-09-17 16:39:37 -07:00
nomnigraph_test.py Expose annotations (#11649) 2018-09-17 16:39:37 -07:00
normalizer.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
normalizer_context.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
normalizer_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
numa_benchmark.py Update from facebook (#6871) 2018-04-23 15:01:56 -07:00
numa_test.py [core] NUMA-aware pinned allocator 2018-03-06 00:33:11 -08:00
observer_test.py
optimizer.py Support lr adaption for SparseAdam and RowWiseSparseAdam (#11162) 2018-09-17 10:29:03 -07:00
optimizer_context.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
optimizer_test.py Add Adadelta optimizer to caffe2 (#9088) 2018-07-24 20:09:21 -07:00
optimizer_test_util.py Implementation of Wngrad optimizer caffe2 python wrapper and unit test on least square regression (#9001) 2018-07-13 18:54:52 -07:00
parallel_workers.py Update from facebook (#7696) 2018-05-19 23:10:48 -07:00
parallel_workers_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
parallelize_bmuf_distributed_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
pipeline.py SNNTest with Data Preproc Service (#11707) 2018-09-17 21:25:49 -07:00
pipeline_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
predictor_constants.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
pybind_state.cc caffe2::DeviceType -> at::DeviceType (#11254) 2018-09-05 16:28:09 -07:00
pybind_state.h caffe2::DeviceType -> at::DeviceType (#11254) 2018-09-05 16:28:09 -07:00
pybind_state_dlpack.cc caffe2::DeviceType -> at::DeviceType (#11254) 2018-09-05 16:28:09 -07:00
pybind_state_dlpack.h caffe2::StorageImpl use at::DataPtr (#11282) 2018-09-12 09:39:23 -07:00
pybind_state_gpu.cc guard spurious cudnn.h include (#11562) 2018-09-11 21:09:54 -07:00
pybind_state_hip.cc Windows DLL build with Caffe2 code (#11266) 2018-09-06 15:12:20 -07:00
pybind_state_ideep.cc caffe2::DeviceType -> at::DeviceType (#11254) 2018-09-05 16:28:09 -07:00
pybind_state_int8.cc Remove template parameter from Tensor (#9939) 2018-07-27 10:56:39 -07:00
pybind_state_mkl.cc fix typo in error message (#9827) 2018-08-21 18:41:29 -07:00
pybind_state_nomni.cc Expose annotations (#11649) 2018-09-17 16:39:37 -07:00
pybind_state_registry.cc Add registry to pybind_state (#10759) 2018-08-24 17:25:02 -07:00
pybind_state_registry.h Add registry to pybind_state (#10759) 2018-08-24 17:25:02 -07:00
python_op_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
queue_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
record_queue.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
recurrent.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
regularizer.py Add GroupL1Norm regularizer (#9115) 2018-07-06 13:26:09 -07:00
regularizer_context.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
regularizer_test.py Add GroupL1Norm regularizer (#9115) 2018-07-06 13:26:09 -07:00
rnn_cell.py Fix typo in comment (#7183) 2018-05-02 11:58:30 -07:00
schema.py Add util function from core type to dtype (#10716) 2018-08-21 10:55:19 -07:00
schema_test.py Add util function from core type to dtype (#10716) 2018-08-21 10:55:19 -07:00
scope.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
scope_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
session.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
session_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
sparse_to_dense_mask_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
sparse_to_dense_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
task.py Revert D9566744: [New Checkpoint] Kill the dummy TaskOutput when task.get_step() (#11164) 2018-08-31 22:25:57 -07:00
test_util.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
text_file_reader.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
timeout_guard.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
toy_regression_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
transformations.py Enable Conv fusion optimizations in optimizeForIdeep (#9255) 2018-07-16 21:28:50 -07:00
transformations_test.py Clean up converter and accept less-valid networks 2018-09-17 17:31:41 -07:00
tt_core.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
tt_core_test.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
utils.py migrating deprecated calls without abc module for containers (#11515) 2018-09-13 15:09:22 -07:00
visualize.py Remove Apache headers from source. 2018-03-27 13:10:18 -07:00
workspace.py Add Workspace methods ws.feed_blob(name, arr) ws.remove_blob(name) (#10929) 2018-08-28 17:54:34 -07:00
workspace_test.py Add workspace.RunPlanInBackground (#9637) 2018-07-20 14:56:12 -07:00