pytorch/torch
Animesh Jain f44446e851 [dynamo] Turn on inline_inbuilt_nn_modules (#131275)
Known issues that are deliberately kept open and will be fixed later are tracked here - https://github.com/pytorch/pytorch/issues/131696

Training dashboard ([link](https://hud.pytorch.org/benchmark/compilers?dashboard=torchinductor&startTime=Thu%2C%2018%20Jul%202024%2000%3A03%3A50%20GMT&stopTime=Thu%2C%2025%20Jul%202024%2000%3A03%3A50%20GMT&granularity=hour&suite=torchbench&mode=training&dtype=amp&lBranch=gh/anijain2305/435/head&lCommit=408b9358b8fca3a5d08b39741419fe8a596941aa&rBranch=gh/anijain2305/435/base&rCommit=d31f2ae904ba2cf0884bf24413ba2109c3585d51))

![image](https://github.com/user-attachments/assets/08ef081c-37d7-436d-905b-4b9e2b470644)

Inference dashboard ([link](https://hud.pytorch.org/benchmark/compilers?dashboard=torchinductor&startTime=Thu%2C%2018%20Jul%202024%2000%3A03%3A50%20GMT&stopTime=Thu%2C%2025%20Jul%202024%2000%3A03%3A50%20GMT&granularity=hour&suite=torchbench&mode=inference&dtype=bfloat16&lBranch=gh/anijain2305/435/head&lCommit=914244fa2fe0055917e039e35183b21fa90afdc6&rBranch=gh/anijain2305/435/base&rCommit=d31f2ae904ba2cf0884bf24413ba2109c3585d51))
![image](https://github.com/user-attachments/assets/32136eff-a39e-4cde-a438-e51a665bc3c9)

Inference sees a little bit more perf degradation but we are ok with that.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/131275
Approved by: https://github.com/ezyang, https://github.com/jansel
ghstack dependencies: #132053
2024-07-29 20:01:51 +00:00
..
_awaits
_C Fix pyi annotation for ProcessGroupNCCL.Options (#130957) 2024-07-29 17:46:01 +00:00
_C_flatbuffer
_custom_op Tighten torch.library.infer_schema input types (#130705) 2024-07-29 16:01:19 +00:00
_decomp Add decomposition for expand_copy (#130940) 2024-07-29 16:23:56 +00:00
_dispatch
_dynamo [dynamo] Turn on inline_inbuilt_nn_modules (#131275) 2024-07-29 20:01:51 +00:00
_export Revert "[BE] typing for decorators - fx/_compatibility (#131568)" 2024-07-28 03:43:39 +00:00
_functorch Revert "Let dynamo inline functional_call (#128646)" 2024-07-29 16:26:50 +00:00
_higher_order_ops Revert "[BE] typing for decorators - fx/_compatibility (#131568)" 2024-07-28 03:43:39 +00:00
_inductor [inductor] Enable FX graph caching in OSS by default (#125863) 2024-07-29 19:19:54 +00:00
_lazy
_library Tighten torch.library.infer_schema input types (#130705) 2024-07-29 16:01:19 +00:00
_logging [pt2] Increase dynamo/inductor default log level to info (#131311) 2024-07-22 17:33:29 +00:00
_numpy
_prims Fix out_wrapper, _make_copy_from_view to handle all signatures (#130937) 2024-07-21 20:39:24 +00:00
_prims_common [BE] typing for decorators - _prims_common/wrappers (#131567) 2024-07-25 14:35:13 +00:00
_refs Add decomposition for expand_copy (#130940) 2024-07-29 16:23:56 +00:00
_strobelight
_subclasses Revert "[BE] typing for decorators - fx/_compatibility (#131568)" 2024-07-28 03:43:39 +00:00
_vendor
amp
ao Revert "[pt2e][quant] Ensure BN node is erased after convert (#131651)" 2024-07-29 16:42:24 +00:00
autograd [Profiler] exclude gpu_user_annotation when accumulating cuda time total (#130733) 2024-07-22 04:35:21 +00:00
backends Revert "[BE] typing for decorators - _jit_internal (#131573)" 2024-07-28 03:29:32 +00:00
compiler
contrib
cpu
csrc [2/N] Fix clang-tidy warnings in inductor (#132040) 2024-07-29 18:41:24 +00:00
cuda Revert "Add wrappers for synchronous GPUDirect Storage APIs (#130633)" 2024-07-26 18:08:20 +00:00
distributed Revert "support zb1p and zb2p algorithms (#130752)" 2024-07-29 12:40:00 +00:00
distributions
export [pt] immutable accessors in graph signature (#131940) 2024-07-27 05:32:53 +00:00
fft
func
futures
fx Revert "[BE] typing for decorators - fx/_compatibility (#131568)" 2024-07-28 03:43:39 +00:00
jit Revert "[BE] typing for decorators - _jit_internal (#131573)" 2024-07-28 03:29:32 +00:00
legacy
lib
linalg
masked Revert "[BE] typing for decorators - masked/_ops (#131569)" 2024-07-28 03:43:39 +00:00
monitor
mps
mtia Revert "MTIA equivalent of torch.cuda.memory_stats (#131673)" 2024-07-26 00:54:37 +00:00
multiprocessing
nested Revert "[BE] typing for decorators - fx/_compatibility (#131568)" 2024-07-28 03:43:39 +00:00
nn Revert "Let dynamo inline functional_call (#128646)" 2024-07-29 16:26:50 +00:00
onnx Revert "[BE] typing for decorators - fx/_compatibility (#131568)" 2024-07-28 03:43:39 +00:00
optim Revert "Fix public API tests (#131386)" 2024-07-28 03:23:04 +00:00
package
profiler
quantization
signal Revert "[BE] typing for decorators - signal/windows/windows (#131582)" 2024-07-28 03:29:31 +00:00
sparse [BE] mypy: disallow untyped decorators (#131428) 2024-07-23 21:50:55 +00:00
special
testing [CI] add new test config label ci-test-showlocals to control test log verbosity (#131981) 2024-07-29 18:53:14 +00:00
utils Fix error handling in _triton.py (#132006) 2024-07-29 15:02:25 +00:00
xpu
__config__.py
__future__.py
__init__.py Revert "Add wrappers for synchronous GPUDirect Storage APIs (#130633)" 2024-07-26 18:08:20 +00:00
_appdirs.py
_classes.py
_compile.py
_custom_ops.py Tighten torch.library.infer_schema input types (#130705) 2024-07-29 16:01:19 +00:00
_deploy.py
_guards.py
_jit_internal.py Revert "[BE] typing for decorators - _jit_internal (#131573)" 2024-07-28 03:29:32 +00:00
_linalg_utils.py
_lobpcg.py
_lowrank.py
_meta_registrations.py Revert "[BE] typing for decorators - _meta_registrations (#131572)" 2024-07-28 03:29:32 +00:00
_namedtensor_internals.py
_ops.py _get_operation_overload: dont raise exception when overload does not exist (#131554) 2024-07-26 15:38:11 +00:00
_python_dispatcher.py
_size_docs.py
_sources.py
_storage_docs.py
_streambase.py
_tensor.py
_tensor_docs.py [MTIA] Support module.mtia() (#131499) 2024-07-25 04:23:48 +00:00
_tensor_str.py
_torch_docs.py
_utils.py
_utils_internal.py
_VF.py
_vmap_internals.py
_weights_only_unpickler.py Blocklist certain modules for weights_only load (#131259) 2024-07-22 18:23:21 +00:00
abi-check.cpp
CMakeLists.txt Revert "Add wrappers for synchronous GPUDirect Storage APIs (#130633)" 2024-07-26 18:08:20 +00:00
custom_class.h
custom_class_detail.h
extension.h
functional.py
hub.py
library.h [3/N] Fix Wunused-parameter warnings (#131271) 2024-07-20 23:31:03 +00:00
library.py Revert "[BE] typing for decorators - library (#131570)" 2024-07-28 03:43:39 +00:00
overrides.py [MTIA] Support module.mtia() (#131499) 2024-07-25 04:23:48 +00:00
py.typed
quasirandom.py
random.py
README.txt
return_types.py
script.h
serialization.py Updating Types in torch/_dynamo/utils.py (#131001) 2024-07-23 18:25:52 +00:00
storage.py Revert "Fix public API tests (#131386)" 2024-07-28 03:23:04 +00:00
torch_version.py Add mypy typing to torch_version.py (#131447) 2024-07-23 17:31:07 +00:00
types.py FakeTensor cache SymInt support (#127596) 2024-07-21 19:26:38 +00:00
version.py.tpl

Note [TH abstraction violation]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TH/THC provide some hpp headers, which are proper C++ headers rather than
C headers.  These headers serve double duty as *internal implementation
detail* headers, whose contents should largely not be used by external
clients.

Ideally, we would not install these headers at all; instead, you should
use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`)
to manipulate these structs.  However, there are a few places
in torch/csrc where we violate this abstraction.  They are marked with
a pointer to this note.  Each of those sites will have to be refactored
when we refactor the guts of THTensor and related structures.