pytorch/benchmarks
Horace He 4cca08368b Adds per-op microbenchmarks for NNC (#50845)
Summary:
Runs through vast majority of primitive ops that exist in NNC and benchmarks them against PyTorch ops on CPU. Dumps out a plot like this.

![nnc](https://user-images.githubusercontent.com/6355099/105247994-a854d380-5b43-11eb-9ac9-1ee779e5ab54.png)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/50845

Reviewed By: ngimel

Differential Revision: D25989080

Pulled By: Chillee

fbshipit-source-id: 6d6a39eb06b3de9a999993224d5e718537c0c8c4
2021-01-21 13:21:01 -08:00
..
cpp/tensorexpr Add CPP Full Reduction Benchmarks. (#50193) 2021-01-21 10:00:50 -08:00
distributed Pytorch Distributed RPC Reinforcement Learning Benchmark (Throughput and Latency) (#46901) 2021-01-11 19:02:36 -08:00
fastrnns Back out "Revert D25717510: Clean up some type annotations in benchmarks/fastrnns" (#50556) 2021-01-14 15:15:03 -08:00
framework_overhead_benchmark Remove py2 compatible future imports (#44735) 2020-09-16 12:55:57 -07:00
functional_autograd_benchmark Enable distribution validation if __debug__ (#48743) 2021-01-05 13:59:10 -08:00
operator_benchmark Cat benchmark: use mobile feed tensor shapes and torch.cat out-variant (#50778) 2021-01-19 22:50:28 -08:00
overrides_benchmark Add __torch_function__ for methods (#37091) 2020-08-05 20:44:13 -07:00
profiler_benchmark Use libkineto in profiler (#46470) 2020-11-25 04:32:16 -08:00
record_function_benchmark Fix D23995953 import. 2020-09-29 19:30:23 -07:00
serialization
sparse Sparse-sparse matrix multiplication (CPU/CUDA) (#39526) 2020-12-21 11:53:55 -08:00
static_runtime [static runtime] add static subgraph fusion pass (#49185) 2020-12-10 14:03:11 -08:00
tensorexpr Adds per-op microbenchmarks for NNC (#50845) 2021-01-21 13:21:01 -08:00
compare-fastrnn-results.py Benchmarks: add scripts for FastRNNs results comparison. (#44134) 2020-09-03 13:44:42 -07:00
compare.sh Benchmarks: add scripts for FastRNNs results comparison. (#44134) 2020-09-03 13:44:42 -07:00
README.md
upload_scribe.py Benchmarks: make fuser and executor configurable from command line. (#44291) 2020-09-09 11:59:35 -07:00

PyTorch Benchmarks

NOTE: This folder is currently work in progress.

This folder contains scripts that produce reproducible timings of various PyTorch features.

It also provides mechanisms to compare PyTorch with other frameworks.

Setup environment

Make sure you're on a machine with CUDA, torchvision, and pytorch installed. Install in the following order:

# Install torchvision. It comes with the pytorch stable release binary
conda install pytorch torchvision -c pytorch

# Install the latest pytorch master from source.
# It should supersede the installation from the release binary.
cd $PYTORCH_HOME
python setup.py build develop

# Check the pytorch installation version
python -c "import torch; print(torch.__version__)"

Benchmark List

Please refer to each subfolder to discover each benchmark suite