pytorch/test
Jane Xu bcbe07200c Improve logic for S3 stats gathering. Uses automatic SLOW_TESTS. (#53549)
Summary:
This PR:
1. refactors the logic for S3 stats gathering.
2. Renames SLOW_TESTS to TARGET_DET_LIST to disambiguate and remove confusion with slowTest
2. detects slow tests (tests with time > 5min) to add to the TARGET_DET_LIST based on results in S3 from the previous nightly.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/53549

Test Plan:
Set CIRCLE_JOB to your favorite CI job (like `pytorch_linux_bionic_py3_8_gcc9_coverage_test1`).
Run `python test/run_test.py --determine-from=<your fave pytorch files>`
e.g., `python test/run_test.py --determine-from=test/run_test.py`

Reviewed By: mrshenli

Differential Revision: D26904478

Pulled By: janeyx99

fbshipit-source-id: 9576b34f4fee09291d60e36ff2631753a3925094
2021-03-10 09:37:06 -08:00
..
backward_compatibility [JIT] Fix backward compatibility test broken by #53410 (#53683) 2021-03-09 22:00:15 -08:00
benchmark_utils Quality of life improvements to Timer (#53294) 2021-03-09 08:15:30 -08:00
bottleneck_test
cpp Adds a bool is_available() method to the backend contract (#53068) 2021-03-10 00:24:16 -08:00
cpp_api_parity Huber loss (#50553) 2021-03-02 17:30:45 -08:00
cpp_extensions Register DefaultBackend implementations for functional/inplace structured operators (#53037) 2021-03-02 14:13:08 -08:00
custom_backend Adds a bool is_available() method to the backend contract (#53068) 2021-03-10 00:24:16 -08:00
custom_operator
distributed [RELAND] Deduplicate shared params before constructing Reducer in DDP (#53279) 2021-03-10 07:56:20 -08:00
distributions Add sample validation for LKJCholesky.log_prob (#52763) 2021-02-25 16:12:29 -08:00
error_messages
expect Trace linear as aten::linear (#51897) 2021-02-19 10:20:42 -08:00
fx [fx] Add DCE pass (#52658) 2021-03-08 19:54:56 -08:00
jit Adds a bool is_available() method to the backend contract (#53068) 2021-03-10 00:24:16 -08:00
jit_hooks
mobile [pytorch] reintroduce static dispatch (#51957) 2021-02-19 11:41:39 -08:00
namespace_b
onnx Forbid trailing whitespace (#53406) 2021-03-05 17:22:55 -08:00
optim
package_a [fx/package] make GraphModules packageable (#51976) 2021-02-23 22:43:00 -08:00
quantization [quant][fix] MHA tensor assignment fix (#53031) 2021-03-03 14:49:19 -08:00
scripts
test_img
type_hint_tests
HowToWriteTestsUsingFileCheck.md
linear.py Move conv to mkldnn (#51483) 2021-03-01 21:19:27 -08:00
module_a.py
run_test.py Improve logic for S3 stats gathering. Uses automatic SLOW_TESTS. (#53549) 2021-03-10 09:37:06 -08:00
simulate_nccl_errors.py
test_autograd.py Fix gradfn attr bindings when saved variable is of an output (#53205) 2021-03-04 16:59:42 -08:00
test_binary_ufuncs.py Modify tests to use assertWarnsOnceRegex instead of maybeWarnsRegex (#52387) 2021-03-08 03:32:14 -08:00
test_bundled_images.py
test_bundled_inputs.py [Pytorch Mobile] Improve Bundled Inputs Error Checking (#52386) 2021-02-24 13:55:45 -08:00
test_complex.py
test_cpp_api_parity.py
test_cpp_extensions_aot.py Marginally improve pytest collection for top-level test files (#53617) 2021-03-10 08:56:39 -08:00
test_cpp_extensions_jit.py
test_cuda.py Prefix assert_async with underscore, fix some bugs in assert_async CUDA testing (#53276) 2021-03-05 17:36:01 -08:00
test_cuda_primary_ctx.py
test_dataloader.py enable coverage test for dataloader on Windows (#52550) 2021-03-02 18:40:47 -08:00
test_datapipe.py [DataLoader] ConcatDataPipe (#53301) 2021-03-06 07:32:02 -08:00
test_dataset.py [DataLoader] Rename Functional DataSet to DataPipe (#51488) 2021-02-03 07:01:09 -08:00
test_determination.py Improve logic for S3 stats gathering. Uses automatic SLOW_TESTS. (#53549) 2021-03-10 09:37:06 -08:00
test_dispatch.py
test_expecttest.py
test_foreach.py Forbid trailing whitespace (#53406) 2021-03-05 17:22:55 -08:00
test_function_schema.py
test_functional_autograd_benchmark.py
test_futures.py [JIT/Futures] support set_exception api (#50983) 2021-02-04 20:22:19 -08:00
test_fx.py [GraphModule] Improve buffer registration during init (#53444) 2021-03-09 21:05:01 -08:00
test_fx_experimental.py [GraphModule] Improve buffer registration during init (#53444) 2021-03-09 21:05:01 -08:00
test_indexing.py
test_jit.py Fix pylint error torch.tensor is not callable (#53424) 2021-03-09 11:32:53 -08:00
test_jit_cuda_fuser.py [ROCm] disable tests for ROCm 4.0.1 (#51510) 2021-02-02 14:39:08 -08:00
test_jit_disabled.py
test_jit_fuser.py
test_jit_fuser_legacy.py
test_jit_fuser_te.py add a full pipeline test for a TypeCheck (#52933) 2021-03-01 10:58:08 -08:00
test_jit_legacy.py
test_jit_profiling.py
test_jit_py3.py
test_jit_simple.py
test_jit_string.py [JIT] Allow implicit boolean conversion of containers (#51683) 2021-02-05 00:34:35 -08:00
test_kernel_launch_checks.py
test_license.py concantenate LICENSE files when building a wheel (#51634) 2021-02-08 08:28:46 -08:00
test_linalg.py Modify tests to use assertWarnsOnceRegex instead of maybeWarnsRegex (#52387) 2021-03-08 03:32:14 -08:00
test_logging.py
test_metal.py
test_mkldnn.py Dont error on 0-dim in convolution (#51922) 2021-03-01 21:22:30 -08:00
test_mobile_optimizer.py [Reland] Update XNNPACK (#52691) 2021-02-24 06:40:38 -08:00
test_multiprocessing.py
test_multiprocessing_spawn.py
test_namedtensor.py Enhance Tensor.unflatten to support -1 as the inferred size (#51955) 2021-02-18 08:37:41 -08:00
test_namedtuple_return_api.py Revert D26723384: [pytorch][PR] Implements torch.linalg.lstsq 2021-03-03 15:24:25 -08:00
test_native_functions.py De-prioritise Dimname and DimnameList in python overload resolution (#51350) 2021-03-02 23:07:08 -08:00
test_nn.py move rnn cell size check to cpp (#51964) 2021-03-09 15:02:20 -08:00
test_numba_integration.py
test_numpy_interop.py Mypy fixes for pytorch master (#52090) 2021-02-17 10:39:51 -08:00
test_op_aliases.py OpInfo: Added clamp and trunc tests with aliases (#51167) 2021-02-10 05:36:18 -08:00
test_openmp.py Marginally improve pytest collection for top-level test files (#53617) 2021-03-10 08:56:39 -08:00
test_ops.py Expands OpInfo out= testing (#53259) 2021-03-09 08:19:26 -08:00
test_optim.py [ROCm] Enable test in test_linalg.py, test_optim.py and test_vmap.py … (#52818) 2021-03-04 02:29:45 -08:00
test_overrides.py Marginally improve pytest collection for top-level test files (#53617) 2021-03-10 08:56:39 -08:00
test_package.py [package] Add deny method to PackageExporter (#53233) 2021-03-04 20:37:41 -08:00
test_profiler.py Skip tests on ROCm (#53339) 2021-03-04 21:55:34 -08:00
test_pruning_op.py
test_public_bindings.py [JIT] Add torch._C._jit submodule (#52910) 2021-02-26 16:05:05 -08:00
test_pytree.py
test_quantization.py NS for FX: add test for a simple sparsenn model (#52092) 2021-02-18 08:17:57 -08:00
test_reductions.py Enabled amin & amax for float16 & bfloat16 (#52579) 2021-03-04 07:03:03 -08:00
test_serialization.py
test_set_default_mobile_cpu_allocator.py [Pytorch] Add python binding to use mobile cpu allocator. (#52323) 2021-02-17 08:42:23 -08:00
test_shape_ops.py Removes nonzero method warning (#51618) 2021-02-04 17:43:43 -08:00
test_show_pickle.py
test_sort_and_select.py Stable sort for the CPU take 2. (#51790) 2021-02-19 09:28:57 -08:00
test_sparse.py Modify tests to use assertWarnsOnceRegex instead of maybeWarnsRegex (#52387) 2021-03-08 03:32:14 -08:00
test_spectral_ops.py Modify tests to use assertWarnsOnceRegex instead of maybeWarnsRegex (#52387) 2021-03-08 03:32:14 -08:00
test_static_runtime.py [static runtime] Move all heavy constructor logic into InferenceModule (renamed to StaticModule) (#51564) 2021-03-05 10:15:26 -08:00
test_tensor_creation_ops.py Modify tests to use assertWarnsOnceRegex instead of maybeWarnsRegex (#52387) 2021-03-08 03:32:14 -08:00
test_tensorboard.py
test_tensorexpr.py Add NNC support for aten::hardtanh (a hot operation in mobilenet v2/v3) (#52394) 2021-02-18 22:56:03 -08:00
test_tensorexpr_pybind.py [TensorExpr] Reland: "PyBind: bind ExternalCalls." (#53063) 2021-03-01 20:44:10 -08:00
test_testing.py Store test file in S3 as well for every TestSuite (#52869) 2021-03-02 07:36:00 -08:00
test_throughput_benchmark.py
test_torch.py Filter 0's returned by exponential distribution (#53480) 2021-03-10 00:35:31 -08:00
test_type_hints.py Mypy fixes for pytorch master (#52090) 2021-02-17 10:39:51 -08:00
test_type_info.py
test_type_promotion.py
test_unary_ufuncs.py Revert D26811466: [pytorch][PR] [reland] Add OpInfo for bitwise_not and make ROCM and CUDA OpInfo tests consistent 2021-03-08 15:47:47 -08:00
test_utils.py Store test file in S3 as well for every TestSuite (#52869) 2021-03-02 07:36:00 -08:00
test_view_ops.py
test_vmap.py [ROCm] Enable test in test_linalg.py, test_optim.py and test_vmap.py … (#52818) 2021-03-04 02:29:45 -08:00
test_vulkan.py
test_xnnpack_integration.py