pytorch/tools
PaliC d1de5f5f0d Change daily aggregates upload job to use sum and occurence counter instead of averages (#98359)
We used to keep track of the average of stats, however, when we munge the data to find interesting insights this makes things difficult (ie. finding total test time for an oncall). The pin is updated such that we keep track of the sum instead as well as an "occurrences" field such that the average can be rederived from sum/occurrences.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/98359
Approved by: https://github.com/huydhn
2023-04-05 16:31:58 +00:00
..
amd_build Bump black version to 23.1.0 (#96578) 2023-03-15 06:27:59 +00:00
autograd Add itemsize and nbytes properties to Tensor (#98322) 2023-04-05 12:11:55 +00:00
bazel_tools
build_defs
code_analyzer [BE] Enable flake8-comprehension rule C417 (#97880) 2023-03-30 14:34:24 +00:00
code_coverage Fix typos under tools directory (#97779) 2023-03-30 08:21:35 +00:00
config
coverage_plugins_package
dynamo Fix for verify_dynamo on ROCm (#97013) 2023-03-21 03:19:31 +00:00
gdb Fix typos under tools directory (#97779) 2023-03-30 08:21:35 +00:00
iwyu
jit [BE] Prefer dash over underscore in command-line options (#94505) 2023-02-09 20:16:49 +00:00
linter Fix typos under tools directory (#97779) 2023-03-30 08:21:35 +00:00
lite_interpreter [BE] Enable flake8-comprehension rule C417 (#97880) 2023-03-30 14:34:24 +00:00
lldb Add helpful pretty pretting summaries to torch for lldb debugging (#97101) 2023-03-20 01:27:44 +00:00
onnx Bump black version to 23.1.0 (#96578) 2023-03-15 06:27:59 +00:00
pyi Allow new_full's fill_value argument type to be complex (#91345) 2023-03-21 12:34:00 +00:00
rules
rules_cc [bazel] enable sccache+nvcc in CI (#95528) 2023-02-28 03:51:11 +00:00
setup_helpers Fix typos under tools directory (#97779) 2023-03-30 08:21:35 +00:00
shared Cleaning up some logic in tools/shared/cwrap_common.py (#94475) 2023-02-10 17:49:11 +00:00
stats Change daily aggregates upload job to use sum and occurence counter instead of averages (#98359) 2023-04-05 16:31:58 +00:00
test Retry at test file level (#97506) 2023-03-31 18:36:53 +00:00
testing Retry at test file level (#97506) 2023-03-31 18:36:53 +00:00
__init__.py
bazel.bzl
BUCK.bzl Revert "Move functional collectives to the right namespace (#97793)" 2023-03-31 16:02:07 +00:00
BUCK.oss
build_libtorch.py
build_pytorch_libs.py
download_mnist.py
extract_scripts.py [BE] Import Literal, Protocol, and Final from standard library typing as of Python 3.8+ (#94490) 2023-02-09 19:17:49 +00:00
gen_flatbuffers.sh
gen_vulkan_spv.py
generate_torch_version.py [BE] Prefer dash over underscore in command-line options (#94505) 2023-02-09 20:16:49 +00:00
generated_dirs.txt
git_add_generated_dirs.sh
git_reset_generated_dirs.sh
nightly.py [BE] Modernize PyTorch even more for 3.8 with pyupgrade (#94520) 2023-02-10 18:02:50 +00:00
nvcc_fix_deps.py
pytorch.version
README.md Remove fast_nvcc entry in README.md (#97624) 2023-03-28 16:23:09 +00:00
render_junit.py
substitute.py [BE] Prefer dash over underscore in command-line options (#94505) 2023-02-09 20:16:49 +00:00
update_masked_docs.py Bump black version to 23.1.0 (#96578) 2023-03-15 06:27:59 +00:00
vscode_settings.py [1/3] Recognize .py.in and .pyi.in files as Python in VS Code (#95200) 2023-03-01 19:16:56 +00:00

This folder contains a number of scripts which are used as part of the PyTorch build process. This directory also doubles as a Python module hierarchy (thus the __init__.py).

Overview

Modern infrastructure:

  • autograd - Code generation for autograd. This includes definitions of all our derivatives.
  • jit - Code generation for JIT
  • shared - Generic infrastructure that scripts in tools may find useful.
    • module_loader.py - Makes it easier to import arbitrary Python files in a script, without having to add them to the PYTHONPATH first.

Build system pieces:

  • setup_helpers - Helper code for searching for third-party dependencies on the user system.
  • build_pytorch_libs.py - cross-platform script that builds all of the constituent libraries of PyTorch, but not the PyTorch Python extension itself.
  • build_libtorch.py - Script for building libtorch, a standalone C++ library without Python support. This build script is tested in CI.

Developer tools which you might find useful:

Important if you want to run on AMD GPU:

  • amd_build - HIPify scripts, for transpiling CUDA into AMD HIP. Right now, PyTorch and Caffe2 share logic for how to do this transpilation, but have separate entry-points for transpiling either PyTorch or Caffe2 code.
    • build_amd.py - Top-level entry point for HIPifying our codebase.

Tools which are only situationally useful: