Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/69041
`TH_CONCAT_{N}` is still being used by THP so I've moved that into
it's own header but all the compiled code is gone.
Test Plan: Imported from OSS
Reviewed By: anjali411
Differential Revision: D32872477
Pulled By: ngimel
fbshipit-source-id: 06c82d8f96dbcee0715be407c61dfc7d7e8be47a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62550
I noticed that running the build twice in a row resulted in ~80 CUDA files being
rebuilt. Running `ninja -d explain` shows
```
ninja explain: TH/generic/THStorage.h is dirty
ninja explain: TH/generic/THStorageCopy.h is dirty
ninja explain: THC/generic/THCStorage.h is dirty
ninja explain: THC/generic/THCStorageCopy.h is dirty
ninja explain: TH/generic/THTensor.h is dirty
ninja explain: THC/generic/THCTensor.h is dirty
ninja explain: THC/generic/THCTensorCopy.h is dirty
ninja explain: THC/generic/THCTensorMath.h is dirty
ninja explain: THC/generic/THCTensorMathMagma.h is dirty
ninja explain: THC/generic/THCTensorMathPairwise.h is dirty
ninja explain: THC/generic/THCTensorScatterGather.h is dirty
```
considering `ninja` is working relative to the `build` folder, these files don't
actually exist. I traced this back to the output of `nvcc -MD` containing
paths relative to the include directory, instead of being absolute.
This adds a little script to launch the compiler then resolve any relative paths
in the `.d` file before `ninja` looks at it. To use it, I run the build with
```
export CMAKE_CUDA_COMPILER_LAUNCHER="python;`pwd`/tools/nvcc_fix_deps.py;ccache"
```
There are some possible pit-falls here. The same relative path might work for
two include directories, and the compiler could pick a different one. Or,
the compiler might have additional implicit include directories that are needed
to resolve the path. However, this has worked perfectly in my testing and it's
completely opt-in so should be fine.
Test Plan: Imported from OSS
Reviewed By: ejguan
Differential Revision: D31503351
Pulled By: malfet
fbshipit-source-id: b184c4526679d976b93829b5715cafcb1c7db2ae
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64598
This adds a filter option rather than an all-or-nothing so it's easier to iterate on a specific job.
```bash
python tools/testing/explicit_ci_jobs.py --filter-gha '*generated-linux-*gcc5.4*'
```
See #64600 for an example usage
NB: If you regenerate the worfklows you will need to re-run that command to re-delete everything.
Test Plan: Imported from OSS
Reviewed By: janeyx99
Differential Revision: D30788850
Pulled By: driazati
fbshipit-source-id: a32c266bbd876c396665bceef9a0a961b4586564
Summary:
This PR is created to replace https://github.com/pytorch/pytorch/pull/53180 PR stack, which has all the review discussions. Reason for needing a replacement is due to a messy Sandcastle issue.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64234
Reviewed By: gmagogsfm
Differential Revision: D30656444
Pulled By: ansley
fbshipit-source-id: 77536c8bcc88162e2c72636026ca3c16891d669a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63385
Correcting a mistake for the pytorch uninstall, and
adding an extra note for Darwin.
Test Plan: Imported from OSS
Reviewed By: janeyx99, heitorschueroff
Differential Revision: D30530234
fbshipit-source-id: e0f88a1725eeadabfb4b28c1da11e369ee878ab4
Summary:
1. In fact, Visual Studio isn't supported as CMAKE generator
2. I was asked many times why there's error as 'Could NOT find OpenMP'
3. Add Newly added Best Practices link in contributing.md
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63843
Reviewed By: seemethere, heitorschueroff
Differential Revision: D30514095
Pulled By: janeyx99
fbshipit-source-id: 76715a1d8c049122546e5a7778cafe54e4dfd5d6
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62549
When building CUDA files with native CMake support, it will respect the
`CMAKE_CUDA_COMPILER_LAUNCHER` setting. So, there's no need for symlinks.
Test Plan: Imported from OSS
Reviewed By: bdhirsh
Differential Revision: D30498488
Pulled By: malfet
fbshipit-source-id: 71c2ae9d4570cfac2a64d777bc95cda3764332a0
Summary:
This option was added in https://github.com/pytorch/pytorch/issues/61940 and fits with this section's theme of improving build times.
I've also changed it to a `cmake_dependent_option` instead of `FATAL_ERROR`ing for older CMake versions.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62827
Reviewed By: astaff
Differential Revision: D30342102
Pulled By: malfet
fbshipit-source-id: 3095b44b7085aee8a884ec95cba9f8998d4442e7
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63228
It is a quick summary and links to a page on the Developer Wiki that has
more detail.
Test Plan: Imported from OSS
Reviewed By: SplitInfinity
Differential Revision: D30347109
Pulled By: zou3519
fbshipit-source-id: a6242986d275e5279ca3f61ade2294a132d268c4
Summary:
This PR enables installing our custom MacOS clang-tidy binaries. It also updates related documentation.
The binaries are produced by [this CI job](https://github.com/pytorch/test-infra/blob/master/.github/workflows/clang-tidy-macos.yml), and are published to S3.
This PR does not handle versioning of the downloaded binaries as this is being worked on separately. See https://github.com/pytorch/test-infra/issues/73
Pull Request resolved: https://github.com/pytorch/pytorch/pull/62214
Test Plan:
On a MacOS machine, run
```bash
python3 -m tools.linter.install.clang_tidy
.clang-tidy-bin/clang-tidy --checks="*" --list-checks | grep "misc-max-tokens"
```
Reviewed By: janeyx99, mruberry
Differential Revision: D29917728
Pulled By: 1ntEgr8
fbshipit-source-id: 98d0d8b7a57bdebf0ebcdc83228ef391e8c6629e
Summary:
Now expecttest is an independent library but `CONTRIBUTING.md` and `requirements.txt` do not mention the need of the library.
Related: https://github.com/pytorch/pytorch/pull/60658
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61163
Reviewed By: heitorschueroff
Differential Revision: D29660296
Pulled By: ezyang
fbshipit-source-id: e2e86d42526c83bec7cdf7221e19fe83d9686103
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61119
This change spilts the clang-tidy CI job into smaller steps and uses a
refactored version of the clang_tidy.py script.
The new folder structure is as follows:
```
tools/linter/clang_tidy
|_ __main__py
|_ requirements.txt
|_ run.py
|_ setup.sh
```
`__main__.py`
This script will run `tools/linter/clang_tidy/setup.sh` if a `build`
directory doesn't exist, mimicing what used to be done as a separate
step in the CI job.
After that, it will invoke `clang-tidy` with default arguments being
declared in the script itself (as opposed to declaring them in
lint.yml).
The reasoning behind this approach is two-fold:
- Make it easier to run `clang-tidy` locally using this script
- De-duplicate the option passing
`requirements.txt`
Contains a list of additional python dependencies needed by the
`clang-tidy` script.
`setup.sh`
If a build directory doesn't exist, this command will run the necessary
codegen and build commands for running `clang-tidy`
Example usage:
```
python3 tools/linter/clang_tidy --parallel
```
Notice that we don't have to put the `.py` at the end of `clang_tidy`.
Test Plan:
Run the following command:
```
python3 tools/linter/clang_tidy --paths torch/csrc/fx --parallel
```
Reviewed By: walterddr, janeyx99
Differential Revision: D29568582
Pulled By: 1ntEgr8
fbshipit-source-id: cd6d11c5cb8ba9f1344a87c35647a1cd8dd45b04
Summary:
People don't need to self host these anymore since we do it automatically in PRs
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61080
Reviewed By: VitalyFedyunin, janeyx99
Differential Revision: D29506465
Pulled By: driazati
fbshipit-source-id: 45875cb229f8cc565a9a1405f52cef198ee0e687
Summary:
Changes including:
- introduced `linter/`, `testing/`, `stats/` folders in `tools/`
- move appropriate scripts into these folders
- change grepped references in the pytorch/pytorch repo
Next step
- introduce `build/` folder for build scripts
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60473
Test Plan:
- CI (this is important b/c pytorch/test-infra also rely on some script reference.
- tools/tests/
Reviewed By: albanD
Differential Revision: D29352716
Pulled By: walterddr
fbshipit-source-id: bad40b5ce130b35dfd9e59b8af34f9025f3285fd
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58001
Adds a script so that devs can generate a commit (at the base of a stack) that removes all CI jobs but the set that they care about. See CONTRIBUTING.md changes for usage
Test Plan: Imported from OSS
Reviewed By: mruberry
Differential Revision: D28359910
Pulled By: driazati
fbshipit-source-id: 2741570f2bab2c28f4a9d7aef727b1b2399d0ce1
Summary:
Why:
To keep VS version always updated in README
1. update VS version link in CI. It's more convenient for my PR robot to update the version in README once the VS in CI is updated. and permlink isn't stable.
2. Move `building on legacy code` to development tips. The table is big and it looks the REAMD not updated at the first sight.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56686
Reviewed By: janeyx99
Differential Revision: D28272060
Pulled By: samestep
fbshipit-source-id: 4bb879ea2914cc8bcd68343a9ed230418e1f9268
Summary:
This queries the local git repo for changed files (any changed files, not just committed ones) and sends them to mypy/flake8 instead of the default (which is the whole repo, defined the .flake8 and mypy.ini files). This brings a good speedup (from 15 seconds with no cache to < 1 second from my local testing on this PR).
```bash
make quicklint -j 6
```
It should be noted that the results of this aren’t exactly what’s in the CI, since mypy and flake8 ignore the `include` and `exclude` parts of their config when an explicit list of files is passed in.
](https://our.intern.facebook.com/intern/diff/27901577/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56559
Pulled By: driazati
Reviewed By: malfet
Differential Revision: D27901577
fbshipit-source-id: 99f351cdfe5aba007948aea2b8a78f683c5d8583
Summary:
This pulls out shell scripts from an action and runs them locally as a first pass at https://github.com/pytorch/pytorch/issues/55847. A helper script extracts specific steps in some order and runs them:
```bash
$ time -p make lint -j 5 # run lint with 5 CPUs
python scripts/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'flake8-py3' \
--step 'Run flake8'
python scripts/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'mypy' \
--step 'Run mypy'
python scripts/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'quick-checks' \
--step 'Ensure no trailing spaces' \
--step 'Ensure no tabs' \
--step 'Ensure no non-breaking spaces' \
--step 'Ensure canonical include' \
--step 'Ensure no unqualified noqa' \
--step 'Ensure no direct cub include' \
--step 'Ensure correct trailing newlines'
python scripts/actions_local_runner.py \
--file .github/workflows/lint.yml \
--job 'cmakelint' \
--step 'Run cmakelint'
quick-checks: Ensure no direct cub include
quick-checks: Ensure canonical include
quick-checks: Ensure no unqualified noqa
quick-checks: Ensure no non-breaking spaces
quick-checks: Ensure no tabs
quick-checks: Ensure correct trailing newlines
cmakelint: Run cmakelint
quick-checks: Ensure no trailing spaces
mypy: Run mypy
Success: no issues found in 1316 source files
Success: no issues found in 56 source files
flake8-py3: Run flake8
./test.py:1:1: F401 'torch' imported but unused
real 13.89
user 199.63
sys 6.08
```
Mypy/flake8 are by far the slowest, but that's mostly just because they're wasting a bunch of work linting the entire repo.
In followup, we could/should:
* Improve ergonomics (i.e. no output unless there are errors)
* Speed up lint by only linting files changes between origin and HEAD
* Add clang-tidy
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56439
Reviewed By: samestep
Differential Revision: D27888027
Pulled By: driazati
fbshipit-source-id: d6f2a59a45e9d725566688bdac8e909210175996
Summary:
This PR includes:
- A formatting change to make katex installation instructions more visible for Facebook employees.
- A short tip about how to start a lightweight HTTP server on a remote machine to browse the doc build artifacts.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56018
Reviewed By: H-Huang
Differential Revision: D27765157
Pulled By: cbalioglu
fbshipit-source-id: 67663de0ba7b742e0deb5358d1e45eea9edd840f
Summary:
PRs https://github.com/pytorch/pytorch/issues/53652 and https://github.com/pytorch/pytorch/issues/54693 attempted to increase the consistency of our choice of commit (head vs merge) for CI on PRs, and have so far been unsuccessful. This PR takes a less ambitious approach to the problem by clarifying the choice in one specific way (see the following paragraph) and documenting it in `CONTRIBUTING.md`.
In addition to documentation, this PR also removes the current behavior of our GHA jobs that checkout the PR tip instead of the merge commit. At first glance, this behavior seems to increase consistency (by eliminating the special-case for `ghstack` PRs), but in reality, it actually just means that for non-`ghstack` PRs, the question "Which commit is used in CI?" has *two* answers instead of just one; see the description of https://github.com/pytorch/pytorch/issues/53652 for more details.
Once merged, this PR will unblock other PRs that add modify our GHA workflows in breaking ways, such as https://github.com/pytorch/pytorch/issues/54737.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54967
Test Plan: None.
Reviewed By: walterddr, seemethere
Differential Revision: D27435913
Pulled By: samestep
fbshipit-source-id: 405fb419cf015cf88107d5eb2498cfb5bcb7ce33
Summary:
This is something which I wrote because it was useful during my debugging sessions, but I think it might be generally useful to other people as well so I took the liberty of proposing an official `pytorch-gdb` extension.
`pytorch-gdb` is a gdb script written in python. Currently, it contains only one command: `torch-tensor-repr`, which prints a human-readable repr of an `at::Tensor` object. Example:
```
Breakpoint 1, at::native::neg (self=...) at [...]/pytorch/aten/src/ATen/native/UnaryOps.cpp:520
520 Tensor neg(const Tensor& self) { return unary_op_impl(self, at::neg_out); }
(gdb) # the default repr of 'self' is not very useful
(gdb) p self
$1 = (const at::Tensor &) 0x7ffff72ed780: {impl_ = {target_ = 0x5555559df6e0}}
(gdb) torch-tensor-repr self
Python-level repr of self:
tensor([1., 2., 3., 4.], dtype=torch.float64)
```
The idea is that by having an official place where to put these things, `pytorch-gdb` will slowly grow other useful features and make the pytorch debugging experience nicer and faster.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54339
Reviewed By: bdhirsh
Differential Revision: D27253674
Pulled By: ezyang
fbshipit-source-id: dba219e126cc2fe66b2d26740f3a8e3b886e56f5
Summary:
Context: https://github.com/pytorch/pytorch/pull/53299#discussion_r587882857
These are the only hand-written parts of this diff:
- the addition to `.github/workflows/lint.yml`
- the file endings changed in these four files (to appease FB-internal land-blocking lints):
- `GLOSSARY.md`
- `aten/src/ATen/core/op_registration/README.md`
- `scripts/README.md`
- `torch/csrc/jit/codegen/fuser/README.md`
The rest was generated by running this command (on macOS):
```
git grep -I -l ' $' -- . ':(exclude)**/contrib/**' ':(exclude)third_party' | xargs gsed -i 's/ *$//'
```
I looked over the auto-generated changes and didn't see anything that looked problematic.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/53406
Test Plan:
This run (after adding the lint but before removing existing trailing spaces) failed:
- https://github.com/pytorch/pytorch/runs/2043032377
This run (on the tip of this PR) succeeded:
- https://github.com/pytorch/pytorch/runs/2043296348
Reviewed By: walterddr, seemethere
Differential Revision: D26856620
Pulled By: samestep
fbshipit-source-id: 3f0de7f7c2e4b0f1c089eac9b5085a58dd7e0d97
Summary:
Adds a link to existing instructions in CONTRIBUTING.md, so those instructions are more visible to contributors.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50841
Reviewed By: samestep
Differential Revision: D25983089
Pulled By: janeyx99
fbshipit-source-id: 0b777ec760765153c607515ab09441dd0cfddf3c
Summary:
Addresses one of the documentation points in https://github.com/pytorch/pytorch/issues/50513 by making it easier to find our `mypy` wiki page. Also updates the `CONTRIBUTING.md` table of contents and removes some trailing whitespace.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50540
Reviewed By: janeyx99
Differential Revision: D25912366
Pulled By: samestep
fbshipit-source-id: b305f974700a9d9ebedc0c2cb75c92e72d84882a
Summary:
This PR adds instructions on what to do if one committed into a PR branch w/o having a pre-commit hook enabled and having CI report flake8 errors.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49345
Reviewed By: cpuhrsch
Differential Revision: D25683167
Pulled By: soumith
fbshipit-source-id: 3c45c866e1636c116d2cacec438d62c860e6b854
Summary:
Since caffe2 and torch have been consolidated, CAFFE2_API should be merged with TORCH_API. Addresses a TODO.
Manually edited some references of the removed `CAFFE2_API`:
* `CONTRIBUTING.md`
* `caffe2/proto/CMakeLists.txt`
* `cmake/ProtoBuf.cmake`
* `c10/macros/Export.h`
* `torch/csrc/WindowsTorchApiMacro.h`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49496
Reviewed By: malfet, samestep
Differential Revision: D25600726
Pulled By: janeyx99
fbshipit-source-id: 7e068d959e397ac183c097d7e9a9afeca5ddd782
Summary:
In the case people are confused how to make sure ccache is working, I added another sentence in the documentation for how to check that the symlinks are correctly set up in addition to waiting for 2 clean builds of PyTorch.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49337
Reviewed By: walterddr
Differential Revision: D25535659
Pulled By: janeyx99
fbshipit-source-id: 435696255f517c074dd0d9f96534d22b60f795b2
Summary:
This PR moves the list of Flake8 requirements/versions out of `.github/workflows/lint.yml` and into its own file `requirements-flake8.txt`. After (if) this PR is merged, I'll modify the Flake8 installation instructions on [the "Lint as you type" wiki page](https://github.com/pytorch/pytorch/wiki/Lint-as-you-type) (and its internal counterpart) to just say to install from that new file, rather than linking to the GitHub Actions YAML file and/or giving a command with a set of packages to install that keeps becoming out-of-date.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49032
Test Plan:
Either look at CI, or run locally using [act](https://github.com/nektos/act):
```sh
act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04 -j flake8-py3
```
Reviewed By: janeyx99
Differential Revision: D25404037
Pulled By: samestep
fbshipit-source-id: ba4d1e17172a7808435df06cba8298b2b91bb27c
Summary:
This PR tries to make building the docs less confusing for new contributors:
- `npm` is discouraged on devservers for Facebook employees, so I added another way to install `katex`
- the path to `check-doxygen.sh` was wrong, so I fixed it
- while generating the CPP docs, it created two new folders that weren't ignored by Git, so I added those to `.gitignore`
- I wasn't able to get the SSH tunnel to work, so I added instructions to use `scp` as an alternative
I'm not entirely sure how the `docs/cpp/source/{html,latex}/` directories were created since I haven't been able to reproduce them.
I also think that it would be better to use the SSH tunnel since `scp` is so much slower, but I just wasn't able to figure it out; I followed the instructions from `CONTRIBUTING.md` and then ran a [Python `http.server`](https://docs.python.org/3/library/http.server.html) on my devserver:
```bash
python -m http.server 8000 --bind 127.0.0.1 --directory build/html
```
but my browser failed to connect and my (local) terminal printed error messages (presumably from the SSH command).
If anyone knows how to properly set up the SSH tunnel and HTTP server, I add those more detailed instructions to `CONTRIBUTING.md` and remove the `scp` instructions from this PR.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/47539
Reviewed By: malfet
Differential Revision: D24806833
Pulled By: samestep
fbshipit-source-id: 456691018a76efadde28fa5eb783b0895582e72d
Summary:
When attempting to install PyTorch locally on my Macbook, I had some difficulty running the setup steps and understanding what I was really doing. I've added some clarifications and summarized some debugging steps about `python setup.py develop` to lower the barrier of entrance for new contributors.
I'm seeking a lot of review here since I am not sure if what I wrote is entirely the most useful or accurate. Thank you!
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45903
Reviewed By: albanD
Differential Revision: D24140343
Pulled By: janeyx99
fbshipit-source-id: a5e40d1bc804945ae7db2b95ab18cf7fe169e68a
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45610
Also add to the usual documentation places that this option exists.
Test Plan: Imported from OSS
Reviewed By: gmagogsfm
Differential Revision: D24058199
Pulled By: suo
fbshipit-source-id: 81574fbd042f47587e2c7820c726fac0f68af2a7
Summary:
Fixes https://github.com/pytorch/pytorch/issues/40829
This is cross-platform but I have only tried it on linux, personally. Also, I am not fully certain of the usage pattern, so if there are any additional features / adjustments / tests that you want me to add, please just let me know!
CC ezyang rgommers
Pull Request resolved: https://github.com/pytorch/pytorch/pull/42635
Reviewed By: zhangguanheng66
Differential Revision: D23078663
Pulled By: ezyang
fbshipit-source-id: 5c8c8abebd1d462409c22dc4301afcd8080922bb
Summary:
A minor spell check!
I have gone through a dozen of .md files to fix the typos.
zou3519 take a look!
Pull Request resolved: https://github.com/pytorch/pytorch/pull/41599
Reviewed By: ezyang
Differential Revision: D22601629
Pulled By: zou3519
fbshipit-source-id: 68d8f77ad18edc1e77874f778b7dadee04b393ef
Summary:
Spotted a broken link, and while I was at it, fixed a few little language and formatting nits.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/41066
Reviewed By: mruberry
Differential Revision: D22415371
Pulled By: dongreenberg
fbshipit-source-id: 7d11c13235b28a01886063c11a4c5ccb333c0c02
Summary:
I think it would be nice to have these extra README links here so they're easier to find. There are even more READMEs throughout the source tree that I didn't include, but most of them seem to have pretty minimal information.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/38049
Differential Revision: D21470749
Pulled By: ezyang
fbshipit-source-id: aa164a3776ab90f2453634082eeae20c0dd002ce
Summary:
Some links in the TOC of CONTRIBUTING.md is broken since GitHub removes the invalid characters (e.g., `+` in C++) in the anchor link, while the existing TOC uses `-` for replacement.
This PR uses `-` instead of `*` and `+` for the bullet lists to make it consistent with README.md.
b889e0da8a/README.md (L11-L18)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/37131
Differential Revision: D21231299
Pulled By: zou3519
fbshipit-source-id: 8e7bb61550827ce97378d3428542e43612bac8e1
Summary:
Finding out how to ssh into a CircleCI job to debug a failure is a challenge because, as far as I know, there isn't any concise documentation about it. I figured it might be nice to include this in CONTRIBUTING.md.
Maybe there are some other tips about non-CircleCI jobs that could be added in the future as well.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/36507
Differential Revision: D21006526
Pulled By: ezyang
fbshipit-source-id: 0a544ecf37bf9550e9b2f07595332dc5f394bb9e
Summary:
- replace the old build variables NO_CUDA and NO_DISTRIBUTED in CONTRIBUTING.md with the new USE_CUDA and USE_DISTRIBUTED versions.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34831
Differential Revision: D20512659
Pulled By: colesbury
fbshipit-source-id: 2d6cb6fd35886eec0b4b1c94f568b5137407c551
Summary:
This PR move glu to Aten(CPU).
Test script:
```
import torch
import torch.nn.functional as F
import time
torch.manual_seed(0)
def _time():
if torch.cuda.is_available():
torch.cuda.synchronize()
return time.time()
device = "cpu"
#warm up
for n in [10, 100, 1000, 10000]:
input = torch.randn(128, n, requires_grad=True, device=device)
grad_output = torch.ones(128, n // 2, device=device)
for i in range(1000):
output = F.glu(input)
output.backward(grad_output)
for n in [10, 100, 1000, 10000]:
fwd_t = 0
bwd_t = 0
input = torch.randn(128, n, requires_grad=True, device=device)
grad_output = torch.ones(128, n // 2, device=device)
for i in range(10000):
t1 = _time()
output = F.glu(input)
t2 = _time()
output.backward(grad_output)
t3 = _time()
fwd_t = fwd_t + (t2 -t1)
bwd_t = bwd_t + (t3 - t2)
fwd_avg = fwd_t / 10000 * 1000
bwd_avg = bwd_t / 10000 * 1000
print("input size(128, %d) forward time is %.2f (ms); backwad avg time is %.2f (ms)."
% (n, fwd_avg, bwd_avg))
```
Test device: **skx-8180.**
Before:
```
input size(128, 10) forward time is 0.04 (ms); backwad avg time is 0.08 (ms).
input size(128, 100) forward time is 0.06 (ms); backwad avg time is 0.14 (ms).
input size(128, 1000) forward time is 0.11 (ms); backwad avg time is 0.31 (ms).
input size(128, 10000) forward time is 1.52 (ms); backwad avg time is 2.04 (ms).
```
After:
```
input size(128, 10) forward time is 0.02 (ms); backwad avg time is 0.05 (ms).
input size(128, 100) forward time is 0.04 (ms); backwad avg time is 0.09 (ms).
input size(128, 1000) forward time is 0.07 (ms); backwad avg time is 0.17 (ms).
input size(128, 10000) forward time is 0.13 (ms); backwad avg time is 1.03 (ms).
```
Fix https://github.com/pytorch/pytorch/issues/24707, https://github.com/pytorch/pytorch/issues/24708.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/33179
Differential Revision: D19839835
Pulled By: VitalyFedyunin
fbshipit-source-id: e4d3438556a1068da2c4a7e573d6bbf8d2a6e2b9
Summary:
The extra dashes are breaking the link here
Pull Request resolved: https://github.com/pytorch/pytorch/pull/31760
Differential Revision: D19697301
Pulled By: ezyang
fbshipit-source-id: 65de026b9016dc8689c9dac9efb8aafd00b535cd
Summary:
Stacked PRs
* **#31908 - Remove C++ docs contributing page**
* #31905 - Add doc previewing instructions
We should have 1 source of truth for contribution instructions (CONTRIBUTING.md).
This PR moves the instructions from the C++ doc pages there instead of having its
own separate page.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/31908
Pulled By: driazati
Differential Revision: D19296366
fbshipit-source-id: c1daf004259342bd09e09dea3b80e34db47066ec
Summary:
Stacked PRs
* #31908 - Remove C++ docs contributing page
* **#31905 - Add doc previewing instructions**
This adds some instructions on how to get started with Github pages you can show reviewers your documentation changes. Hopefully we can delete this eventually and build docs automatically on relevant PRs in CI.
](https://our.intern.facebook.com/intern/diff/19296364/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/31905
Pulled By: driazati
Differential Revision: D19296364
fbshipit-source-id: df47fa1a8d7be029c3efcf6521298583ad9f7a95
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30530
Switch some mentions of "C++11" in the docs to "C++14"
ghstack-source-id: 95812049
Test Plan: testinprod
Differential Revision: D18733733
fbshipit-source-id: b9d0490eb3f72bad974d134bbe9eb563f6bc8775
Summary:
This TOC is manually generated but `CONTRIBUTING.md` seems like its
stable enough for that to be okay
Pull Request resolved: https://github.com/pytorch/pytorch/pull/29671
Pulled By: driazati
Differential Revision: D18771604
fbshipit-source-id: 0d6c9c6cf1083d3be413219d3cead79c2fe5050b
Summary:
This adds developer documentation for profiling PyTorch using py-spy. In my work on `__torch_function__` I found its ability to profile native code and dump flame graphs extremely useful. I'm not aware of another Python sampling profiler with similar functionality.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30166
Differential Revision: D18625133
Pulled By: ezyang
fbshipit-source-id: cf1b851564a07c9f12fcf1338ac4527f4a3c61c0
Summary:
katex is a deprecated package in Ubuntu and has been removed in recent
releases of Debian. Use npm instead.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/27758
Differential Revision: D17891039
Pulled By: ezyang
fbshipit-source-id: 53de6e14b2638298e5b61996dcd7ba8de02420a3
Summary:
I was unable to use the existing instructions since I don't have sudo privileges on my GPU development machine and couldn't easily install `ccache` or the build dependencies for `ccache`.
However, I was able to get it working by installing `ccache` with `conda` and then creating symlinks to shadow my compilers as in the build-from-source installation instructions. I figure this might be generally useful as others might not have sudo privileges on their pytorch development machine.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/27481
Differential Revision: D17831556
Pulled By: ezyang
fbshipit-source-id: c5373d8739ad910015e677e7ad48bd91b770f842
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/25375
Either MSVC or the Windows headers have a PURE macro defined and will replace
any occurrences of the PURE token in code with an empty string. Replace
AliasAnalysisKind::PURE with AliasAnalysisKind::PURE_FUNCTION.
Note: this is bc breaking.
ghstack-source-id: 89202222
Test Plan: unit tests
Differential Revision: D17107743
fbshipit-source-id: 899a20651ba32d50691956b5424b351586c21cec
Summary:
Some other environment variables can be added to speed things up for development.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22736
Differential Revision: D16200904
Pulled By: soumith
fbshipit-source-id: 797ef91a863a244a6c96e0adf64d9f9b4c9a9582
Summary:
Currently specifying different build options in respect to the "USE_"
series is in quite a disarray. There are a lot of build options that
accept three variants: USE_OPTION, WITH_OPTION, and NO_OPTION. Some
build options only accept USE_ and NO_ variant. Some accept only USE_.
This inconsistency is quite confusing and hard to maintain.
To resolve this inconsistency, we can either let all these build options
support all three variants, or we only support the USE_ variant.
This commit makes a step to the latter choice, i.e., deprecates and sets
a date for removing the NO_ and WITH_ variants and keeps only the
USE_ variant. This is likely better than the former solution because:
- NO_ and WITH_ variants are not documented.
- CMakeLists.txt only has the USE_ variants for relevant build options
defined. It would be a surprise that when user pass these variables to
CMake during rebuild and find them ineffective.
- Multiple variants are difficult to maintain.
- The behavior is confusing if more than one variant is passed. For
example, what to be expected if one sets "NO_CUDA=1 USE_CUDA=1"?
The downside is that this will break backward compatibility for existing
build scripts in the future (if they used the undocumented build
options).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22474
Differential Revision: D16149396
Pulled By: ezyang
fbshipit-source-id: 7145b88ad195db2051772b9665dd708dfcf50b7d
Summary:
- PyCQA/flake8-bugbear#53 has been fixed (but not yet closed on their side) and a new version of flake8-bugbear has been released on Mar 28, 2019. Switch CI to use the latest stable version.
- Fix the new B011 errors that flake8-bugbear catches in the current codebase.
---
B011: Do not call assert False since python -O removes these calls. Instead callers should raise AssertionError().
Pull Request resolved: https://github.com/pytorch/pytorch/pull/21944
Differential Revision: D15974842
Pulled By: soumith
fbshipit-source-id: de5c2c07015f7f1c50cb3904c651914b8c83bf5c
Summary:
I found this significantly speeds up incremental builds.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/21334
Differential Revision: D15632994
Pulled By: suo
fbshipit-source-id: bb4af90f4400bffa90d168d82ff30fece5e3835c
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18628
ghimport-source-id: d94b81a6f303883d97beaae25344fd591e13ce52
Stack from [ghstack](https://github.com/ezyang/ghstack):
* #18629 Provide flake8 install instructions.
* **#18628 Delete duplicated technical content from contribution_guide.rst**
There's useful guide in contributing_guide.rst, but the
technical bits were straight up copy-pasted from CONTRIBUTING.md,
and I don't think it makes sense to break the CONTRIBUTING.md
link. Instead, I deleted the duplicate bits and added a cross
reference to the rst document.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Differential Revision: D14701003
fbshipit-source-id: 3bbb102fae225cbda27628a59138bba769bfa288
Summary:
Added full instructions for how to use the `ccache` package. Thanks.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18495
Differential Revision: D14635351
Pulled By: ezyang
fbshipit-source-id: 158e1052bae580e95f73644252fdbddcc0213128
Summary:
" ProTip! Great commit summaries contain fewer than 50 characters. Place extra information in the extended description."
lol
Pull Request resolved: https://github.com/pytorch/pytorch/pull/17450
Differential Revision: D14206500
Pulled By: suo
fbshipit-source-id: af7ffe299f8c8f04fa8e720847a1f6d576ebafc1
