pytorch/c10/core/GradMode.cpp

#include <c10/core/GradMode.h>

namespace c10 {

bool GradMode::is_enabled() {
  return AutogradState::get_tls_state().get_grad_mode();
}

void GradMode::set_enabled(bool enabled) {
  AutogradState::get_tls_state().set_grad_mode(enabled);
}
} // namespace c10
Move grad_mode.h/cpp to c10. (#56204) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/56204 Test Plan: Imported from OSS Reviewed By: bhosmer Differential Revision: D27807139 Pulled By: ailzhang fbshipit-source-id: 2b693eb0a1034138d8bd68836528078ea5f38145 2021-04-16 18:46:00 +00:00			`#include <c10/core/GradMode.h>`
Move GradMode / AutoGradMode / NoGradGuard to ATen core (#18573) Summary: After the Variable/Tensor merge, code paths in ATen need to be able to check whether a tensor requires gradient, and throw errors in places where a `requires_grad=true` tensor is not allowed (such as https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/Utils.h#L76-L78 and https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/SparseTensorImpl.cpp#L86). Since the `GradMode` thread-local variable controls whether a tensor should accumulate gradients, we need to be able to check this variable from ATen when we determine whether a tensor requires gradient, hence the PR to move `GradMode` / `AutoGradMode` / `NoGradGuard` to ATen. Note that we intentionally don't merge `at::GradMode` and `at::NonVariableTypeMode`, with the following reasoning: Semantically, `at::GradMode` and `at::NonVariableTypeMode` actually mean different things: `at::GradMode` controls whether a tensor should accumulate gradients, and `at::NonVariableTypeMode` controls whether a Variable should be treated as a non-Variable tensor in type dispatches. There are places whether we don't want the tensor to accumulate gradients, but still want the Variable to be treated as a Variable. Here is one example: ```python # torch/tensor.py with torch.no_grad(): ... new_tensor = self.new() # `at::GradMode` is false at this point ... ``` ```cpp // tools/autograd/templates/python_variable_methods.cpp static PyObject * THPVariable_new(PyObject* self, PyObject* args, PyObject* kwargs) { ... // if we merge `at::GradMode` and `at::NonVariableTypeMode`, since `at::GradMode` is false and `self_.type()` checks `at::GradMode` to decide whether to return non-Variable type, it will return a non-Variable type here, which is not what we want (and throws a "Tensor that was converted to Variable was not actually a Variable" error) return THPVariable_Wrap(torch::utils::legacy_tensor_new(self_.type(), args, kwargs)); ... } ``` For the above reason, we cannot merge `at::GradMode` and `at::NonVariableTypeMode`, as they have different purposes. Pull Request resolved: https://github.com/pytorch/pytorch/pull/18573 Differential Revision: D16134413 Pulled By: yf225 fbshipit-source-id: 6140347e78bc54206506499c264818eb693cdb8a 2019-07-06 06:36:30 +00:00
Move grad_mode.h/cpp to c10. (#56204) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/56204 Test Plan: Imported from OSS Reviewed By: bhosmer Differential Revision: D27807139 Pulled By: ailzhang fbshipit-source-id: 2b693eb0a1034138d8bd68836528078ea5f38145 2021-04-16 18:46:00 +00:00			`namespace c10 {`
Move GradMode / AutoGradMode / NoGradGuard to ATen core (#18573) Summary: After the Variable/Tensor merge, code paths in ATen need to be able to check whether a tensor requires gradient, and throw errors in places where a `requires_grad=true` tensor is not allowed (such as https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/Utils.h#L76-L78 and https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/SparseTensorImpl.cpp#L86). Since the `GradMode` thread-local variable controls whether a tensor should accumulate gradients, we need to be able to check this variable from ATen when we determine whether a tensor requires gradient, hence the PR to move `GradMode` / `AutoGradMode` / `NoGradGuard` to ATen. Note that we intentionally don't merge `at::GradMode` and `at::NonVariableTypeMode`, with the following reasoning: Semantically, `at::GradMode` and `at::NonVariableTypeMode` actually mean different things: `at::GradMode` controls whether a tensor should accumulate gradients, and `at::NonVariableTypeMode` controls whether a Variable should be treated as a non-Variable tensor in type dispatches. There are places whether we don't want the tensor to accumulate gradients, but still want the Variable to be treated as a Variable. Here is one example: ```python # torch/tensor.py with torch.no_grad(): ... new_tensor = self.new() # `at::GradMode` is false at this point ... ``` ```cpp // tools/autograd/templates/python_variable_methods.cpp static PyObject * THPVariable_new(PyObject* self, PyObject* args, PyObject* kwargs) { ... // if we merge `at::GradMode` and `at::NonVariableTypeMode`, since `at::GradMode` is false and `self_.type()` checks `at::GradMode` to decide whether to return non-Variable type, it will return a non-Variable type here, which is not what we want (and throws a "Tensor that was converted to Variable was not actually a Variable" error) return THPVariable_Wrap(torch::utils::legacy_tensor_new(self_.type(), args, kwargs)); ... } ``` For the above reason, we cannot merge `at::GradMode` and `at::NonVariableTypeMode`, as they have different purposes. Pull Request resolved: https://github.com/pytorch/pytorch/pull/18573 Differential Revision: D16134413 Pulled By: yf225 fbshipit-source-id: 6140347e78bc54206506499c264818eb693cdb8a 2019-07-06 06:36:30 +00:00
			`bool GradMode::is_enabled() {`
Add a common autograd TLS state (#63860) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/63860 Test Plan: Imported from OSS Reviewed By: heitorschueroff Differential Revision: D30513253 Pulled By: albanD fbshipit-source-id: 97d76ed54dfbdf4ba3fc7051ce3b9bb636cefb4b 2021-08-24 22:32:42 +00:00			`return AutogradState::get_tls_state().get_grad_mode();`
Move GradMode / AutoGradMode / NoGradGuard to ATen core (#18573) Summary: After the Variable/Tensor merge, code paths in ATen need to be able to check whether a tensor requires gradient, and throw errors in places where a `requires_grad=true` tensor is not allowed (such as https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/Utils.h#L76-L78 and https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/SparseTensorImpl.cpp#L86). Since the `GradMode` thread-local variable controls whether a tensor should accumulate gradients, we need to be able to check this variable from ATen when we determine whether a tensor requires gradient, hence the PR to move `GradMode` / `AutoGradMode` / `NoGradGuard` to ATen. Note that we intentionally don't merge `at::GradMode` and `at::NonVariableTypeMode`, with the following reasoning: Semantically, `at::GradMode` and `at::NonVariableTypeMode` actually mean different things: `at::GradMode` controls whether a tensor should accumulate gradients, and `at::NonVariableTypeMode` controls whether a Variable should be treated as a non-Variable tensor in type dispatches. There are places whether we don't want the tensor to accumulate gradients, but still want the Variable to be treated as a Variable. Here is one example: ```python # torch/tensor.py with torch.no_grad(): ... new_tensor = self.new() # `at::GradMode` is false at this point ... ``` ```cpp // tools/autograd/templates/python_variable_methods.cpp static PyObject * THPVariable_new(PyObject* self, PyObject* args, PyObject* kwargs) { ... // if we merge `at::GradMode` and `at::NonVariableTypeMode`, since `at::GradMode` is false and `self_.type()` checks `at::GradMode` to decide whether to return non-Variable type, it will return a non-Variable type here, which is not what we want (and throws a "Tensor that was converted to Variable was not actually a Variable" error) return THPVariable_Wrap(torch::utils::legacy_tensor_new(self_.type(), args, kwargs)); ... } ``` For the above reason, we cannot merge `at::GradMode` and `at::NonVariableTypeMode`, as they have different purposes. Pull Request resolved: https://github.com/pytorch/pytorch/pull/18573 Differential Revision: D16134413 Pulled By: yf225 fbshipit-source-id: 6140347e78bc54206506499c264818eb693cdb8a 2019-07-06 06:36:30 +00:00			`}`

			`void GradMode::set_enabled(bool enabled) {`
Add a common autograd TLS state (#63860) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/63860 Test Plan: Imported from OSS Reviewed By: heitorschueroff Differential Revision: D30513253 Pulled By: albanD fbshipit-source-id: 97d76ed54dfbdf4ba3fc7051ce3b9bb636cefb4b 2021-08-24 22:32:42 +00:00			`AutogradState::get_tls_state().set_grad_mode(enabled);`
Move GradMode / AutoGradMode / NoGradGuard to ATen core (#18573) Summary: After the Variable/Tensor merge, code paths in ATen need to be able to check whether a tensor requires gradient, and throw errors in places where a `requires_grad=true` tensor is not allowed (such as https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/Utils.h#L76-L78 and https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/SparseTensorImpl.cpp#L86). Since the `GradMode` thread-local variable controls whether a tensor should accumulate gradients, we need to be able to check this variable from ATen when we determine whether a tensor requires gradient, hence the PR to move `GradMode` / `AutoGradMode` / `NoGradGuard` to ATen. Note that we intentionally don't merge `at::GradMode` and `at::NonVariableTypeMode`, with the following reasoning: Semantically, `at::GradMode` and `at::NonVariableTypeMode` actually mean different things: `at::GradMode` controls whether a tensor should accumulate gradients, and `at::NonVariableTypeMode` controls whether a Variable should be treated as a non-Variable tensor in type dispatches. There are places whether we don't want the tensor to accumulate gradients, but still want the Variable to be treated as a Variable. Here is one example: ```python # torch/tensor.py with torch.no_grad(): ... new_tensor = self.new() # `at::GradMode` is false at this point ... ``` ```cpp // tools/autograd/templates/python_variable_methods.cpp static PyObject * THPVariable_new(PyObject* self, PyObject* args, PyObject* kwargs) { ... // if we merge `at::GradMode` and `at::NonVariableTypeMode`, since `at::GradMode` is false and `self_.type()` checks `at::GradMode` to decide whether to return non-Variable type, it will return a non-Variable type here, which is not what we want (and throws a "Tensor that was converted to Variable was not actually a Variable" error) return THPVariable_Wrap(torch::utils::legacy_tensor_new(self_.type(), args, kwargs)); ... } ``` For the above reason, we cannot merge `at::GradMode` and `at::NonVariableTypeMode`, as they have different purposes. Pull Request resolved: https://github.com/pytorch/pytorch/pull/18573 Differential Revision: D16134413 Pulled By: yf225 fbshipit-source-id: 6140347e78bc54206506499c264818eb693cdb8a 2019-07-06 06:36:30 +00:00			`}`
Move grad_mode.h/cpp to c10. (#56204) Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/56204 Test Plan: Imported from OSS Reviewed By: bhosmer Differential Revision: D27807139 Pulled By: ailzhang fbshipit-source-id: 2b693eb0a1034138d8bd68836528078ea5f38145 2021-04-16 18:46:00 +00:00			`} // namespace c10`