Pritam Damania 05d18ffaf5 Distributed Autograd: Allow multiple backward passes to accumulate gradients. (#32506) ... Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/32506 In this PR, we've introduced a `retain_graph` parameter to distributed autograd similar to `torch.autograd.backward`. In terms of design, this parameter is sent over RPC to all nodes and is used to create the GraphTask on the local nodes. This enables us to run `dist_autograd.backward()` multiple times in the same context. The use case currently for this is to benchmark only the backward pass for distributed autograd. We'd like to measure the QPS for the backward pass and as a result, running a single forward pass and multiple backward passes in a loop is one way to benchmark backward pass performance. ghstack-source-id: 97868900 Test Plan: waitforbuildbot Differential Revision: D19521288 fbshipit-source-id: 7ad8521059fd400d7b5a6ab77ce56e1927ced90a		2020-02-06 23:27:21 -08:00
..
api	Fix torch::allclose to handle std::numeric_limits<T>::lowest() for integral types (#32978)	2020-02-04 19:06:52 -08:00
common
dist_autograd	Distributed Autograd: Allow multiple backward passes to accumulate gradients. (#32506)	2020-02-06 23:27:21 -08:00
jit	[JIT] Resolve custom classes in source importer (#32977)	2020-02-06 10:45:40 -08:00
rpc	Remove dead includes in caffe2/test	2020-01-21 11:30:34 -08:00
__init__.py