Deprecate and set a date for removing NO_* and WITH_* (user) build options (#22474)

Summary:
Currently specifying different build options in respect to the "USE_"
series is in quite a disarray. There are a lot of build options that
accept three variants: USE_OPTION, WITH_OPTION, and NO_OPTION. Some
build options only accept USE_ and NO_ variant. Some accept only USE_.
This inconsistency is quite confusing and hard to maintain.

To resolve this inconsistency, we can either let all these build options
support all three variants, or we only support the USE_ variant.

This commit makes a step to the latter choice, i.e., deprecates and sets
a date for removing the NO_ and WITH_ variants and keeps only the
USE_ variant. This is likely better than the former solution because:

- NO_ and WITH_ variants are not documented.
- CMakeLists.txt only has the USE_ variants for relevant build options
  defined. It would be a surprise that when user pass these variables to
  CMake during rebuild and find them ineffective.
- Multiple variants are difficult to maintain.
- The behavior is confusing if more than one variant is passed. For
  example, what to be expected if one sets "NO_CUDA=1 USE_CUDA=1"?

The downside is that this will break backward compatibility for existing
build scripts in the future (if they used the undocumented build
options).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/22474

Differential Revision: D16149396

Pulled By: ezyang

fbshipit-source-id: 7145b88ad195db2051772b9665dd708dfcf50b7d

.circleci/scripts/binary_populate_env.sh

@@ -75,7 +75,7 @@ export OVERRIDE_PACKAGE_VERSION="$PYTORCH_BUILD_VERSION"
 export TORCH_PACKAGE_NAME='torch-nightly'
 export TORCH_CONDA_BUILD_FOLDER='pytorch-nightly'
 
-export NO_FBGEMM=1
+export USE_FBGEMM=0
 export PIP_UPLOAD_FOLDER="$PIP_UPLOAD_FOLDER"
 export DOCKER_IMAGE="$DOCKER_IMAGE"
 

.jenkins/pytorch/build-asan.sh

@@ -33,7 +33,7 @@ export ASAN_OPTIONS=detect_leaks=0:symbolize=1
 CC="clang" CXX="clang++" LDSHARED="clang --shared" \
   CFLAGS="-fsanitize=address -fsanitize=undefined -fno-sanitize-recover=all -shared-libasan -pthread" \
   CXX_FLAGS="-pthread" \
-  USE_ASAN=1 NO_CUDA=1 USE_MKLDNN=0 \
+  USE_ASAN=1 USE_CUDA=0 USE_MKLDNN=0 \
   python setup.py install
 
 assert_git_not_dirty

.jenkins/pytorch/macos-build.sh

@@ -30,7 +30,7 @@ if [[ "${BUILD_ENVIRONMENT}" == *cuda9.2* ]]; then
   export PATH=/Developer/NVIDIA/CUDA-${CUDA_VERSION}/bin${PATH:+:${PATH}}
   export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-${CUDA_VERSION}/lib${DYLD_LIBRARY_PATH:+:${DYLD_LIBRARY_PATH}}
   export CUDA_HOME=/Developer/NVIDIA/CUDA-${CUDA_VERSION}
-  export NO_CUDA=0
+  export USE_CUDA=1
 
   if [ -z "${IN_CIRCLECI}" ]; then
     # Eigen gives "explicit specialization of class must precede its first use" error

.jenkins/pytorch/win-test-helpers/build_pytorch.bat

@@ -74,8 +74,12 @@ set CMAKE_GENERATOR=Ninja
 
 if not "%USE_CUDA%"=="1" (
   if "%REBUILD%"=="" (
-    set NO_CUDA=1
+    :: Must save and restore the original value of USE_CUDA, otherwise the
+    :: `if not "%USE_CUDA%"=="0"` line can be messed up.
+    set OLD_USE_CUDA=%USE_CUDA%
+    set USE_CUDA=0
     python setup.py install
+    set USE_CUDA=%OLD_USE_CUDA%
   )
   if errorlevel 1 exit /b 1
   if not errorlevel 0 exit /b 1
@@ -99,7 +103,7 @@ if not "%USE_CUDA%"=="0" (
 
   set CUDA_NVCC_EXECUTABLE=%TMP_DIR_WIN%\bin\nvcc
 
-  if "%REBUILD%"=="" set NO_CUDA=0
+  if "%REBUILD%"=="" set USE_CUDA=1
 
   python setup.py install --cmake && sccache --show-stats && (
     if "%BUILD_ENVIRONMENT%"=="" (

CONTRIBUTING.md

@@ -243,15 +243,15 @@ only interested in a specific component.
   Caffe2 operators.
 
 On the initial build, you can also speed things up with the environment
-variables `DEBUG` and `NO_CUDA`.
+variables `DEBUG` and `USE_CUDA`.
 
 - `DEBUG=1` will enable debug builds (-g -O0)
 - `REL_WITH_DEB_INFO=1` will enable debug symbols with optimizations (-g -O3)
-- `NO_CUDA=1` will disable compiling CUDA (in case you are developing on something not CUDA related), to save compile time.
+- `USE_CUDA=0` will disable compiling CUDA (in case you are developing on something not CUDA related), to save compile time.
 
 For example:
 ```bash
-NO_CUDA=1 DEBUG=1 python setup.py develop
+USE_CUDA=0 DEBUG=1 python setup.py develop
 ```
 
 Make sure you continue to pass these flags on subsequent builds.

README.md

@@ -159,7 +159,7 @@ If you want to compile with CUDA support, install
 - [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads) 9 or above
 - [NVIDIA cuDNN](https://developer.nvidia.com/cudnn) v7 or above
 
-If you want to disable CUDA support, export environment variable `NO_CUDA=1`.
+If you want to disable CUDA support, export environment variable `USE_CUDA=0`.
 Other potentially useful environment variables may be found in `setup.py`.
 
 If you are building for NVIDIA's Jetson platforms (Jetson Nano, TX1, TX2, AGX Xavier), Instructions to [are available here](https://devtalk.nvidia.com/default/topic/1049071/jetson-nano/pytorch-for-jetson-nano/)

aten/src/ATen/cudnn/cudnn-wrapper.h

@@ -6,7 +6,7 @@
 #define STRING(x) STRINGIFY(x)
 
 #if CUDNN_MAJOR < 6
-#pragma message ("CuDNN v" STRING(CUDNN_MAJOR) " found, but need at least CuDNN v6. You can get the latest version of CuDNN from https://developer.nvidia.com/cudnn or disable CuDNN with NO_CUDNN=1")
+#pragma message ("CuDNN v" STRING(CUDNN_MAJOR) " found, but need at least CuDNN v6. You can get the latest version of CuDNN from https://developer.nvidia.com/cudnn or disable CuDNN with USE_CUDNN=0")
 #pragma message "We strongly encourage you to move to 6.0 and above."
 #pragma message "This message is intended to annoy you enough to update."
 #endif

tools/setup_helpers/env.py

@@ -32,19 +32,28 @@ def lib_paths_from_base(base_path):
 
 
 def hotpatch_var(var, prefix='USE_'):
+    def print_warning(good_prefix, bad_prefix, var):
+        print(("The use of {bad_prefix}{var} is deprecated and will be removed on Feb 20, 2020."
+               "Please use {good_prefix}{var} instead.").format(
+                   good_prefix=good_prefix, bad_prefix=bad_prefix, var=var))
+
     if check_env_flag('NO_' + var):
+        print_warning(prefix, 'NO_', var)
         os.environ[prefix + var] = '0'
     elif check_negative_env_flag('NO_' + var):
+        print_warning(prefix, 'NO_', var)
         os.environ[prefix + var] = '1'
     elif check_env_flag('WITH_' + var):
+        print_warning(prefix, 'WITH_', var)
         os.environ[prefix + var] = '1'
     elif check_negative_env_flag('WITH_' + var):
+        print_warning(prefix, 'WITH_', var)
         os.environ[prefix + var] = '0'
 
 
 def hotpatch_build_env_vars():
-    # Before we run the setup_helpers, let's look for NO_* and WITH_*
-    # variables and hotpatch environment with the USE_* equivalent
+    # Before we run the setup_helpers, let's look for NO_* and WITH_* variables and hotpatch environment with the USE_*
+    # equivalent The use of NO_* and WITH_* is deprecated and will be removed in Feb 20, 2020.
     use_env_vars = ['CUDA', 'CUDNN', 'FBGEMM', 'MKLDNN', 'NNPACK', 'DISTRIBUTED',
                     'OPENCV', 'TENSORRT', 'QNNPACK', 'FFMPEG', 'SYSTEM_NCCL',
                     'GLOO_IBVERBS']