pytorch/torch
Benjamin Glass 37a1b9efcc [export] Serialize all dataclass fields (#142286)
Reverts a change in #121337. All dataclass members must be serialized, even default-valued members, because downstream code often implicitly assumes their presence.

This PR fixes a segfault when running `test_custom_op_all_inputs` from `test/inductor/test_aot_inductor_custom_ops.py`. This segfault was caused by querying for an "index" field for the `Device` type (see `torch/csrc/inductor/aoti_torch/oss_proxy_executor.cpp:136`), which was previously skipped when serializing if the device index was unspecified. A number of other structs which are deserialized in this file also contain optional fields, and presumably could experience the same bug.

Fixes #138955

Fixes #134793
Pull Request resolved: https://github.com/pytorch/pytorch/pull/142286
Approved by: https://github.com/zhxchen17
ghstack dependencies: #142175
2024-12-17 17:21:27 +00:00
..
_awaits
_C remove allow-untyped-defs for torch/_C/_lazy.pyi (#143370) 2024-12-17 17:18:10 +00:00
_C_flatbuffer
_custom_op
_decomp Back out "Fix undesired specialization on slice after split. (#142372)" (#143356) 2024-12-17 09:17:18 +00:00
_dispatch Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_dynamo Revert "[reland][dynamo][guards] Consider tensors as immutable for dict tag matches (#141085)" 2024-12-17 17:21:14 +00:00
_export [export] Serialize all dataclass fields (#142286) 2024-12-17 17:21:27 +00:00
_functorch Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_higher_order_ops [FlexAttention] Fix broken eager tracing (#143344) 2024-12-17 09:42:36 +00:00
_inductor cpp_builder: handle CUDA lib paths involving "stubs" in more circumstances (#142175) 2024-12-17 17:21:27 +00:00
_lazy
_library Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_logging Add "inductor_pre_grad_graph" logging (#142717) (#143126) 2024-12-13 21:48:25 +00:00
_numpy Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_prims Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_prims_common
_refs Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_strobelight Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_subclasses Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_vendor
accelerator [BE][accelerator] formalize API name {current,set}_device_{idx => index} (#140542) 2024-12-12 10:53:48 +00:00
amp
ao No actual change, just remove variable contain Tensors from global scope (#143225) 2024-12-17 16:14:25 +00:00
autograd Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
backends Revert "[ROCm] CK Flash Attention Backend (#138947)" 2024-12-17 16:46:57 +00:00
compiler Migrate compiler config to Config (#143152) 2024-12-14 07:38:25 +00:00
contrib
cpu
csrc Revert "[reland][dynamo][guards] Consider tensors as immutable for dict tag matches (#141085)" 2024-12-17 17:21:14 +00:00
cuda [torch][cuda] fix race condition in cuda initialization (#143238) 2024-12-14 07:41:24 +00:00
distributed Revert "FileTimerClient: add retry logic on connect (#143318)" 2024-12-17 02:06:52 +00:00
distributions
export fix dynamo nn module stack fqn (#142823) 2024-12-12 07:02:13 +00:00
fft
func
futures
fx Add torch._compile to uninteresting files (#143209) 2024-12-14 05:40:21 +00:00
jit remove allow-untyped-defs for torch/jit/_ir_utils.py (#143366) 2024-12-17 05:15:15 +00:00
legacy
lib
linalg
masked remove allow-untyped-defs for torch/masked/maskedtensor/creation.py (#143321) 2024-12-17 16:44:50 +00:00
monitor
mps
mtia
multiprocessing
nested
nn [FlexAttention] Optimzing learned bias perf to dq calc (#142281) 2024-12-15 21:44:32 +00:00
onnx Fix a misspelling [ONNX] (#143301) 2024-12-16 20:19:41 +00:00
optim
package
profiler [Profiler] Enable Iterative Step without profiler in fbcode (#142077) 2024-12-12 19:00:13 +00:00
quantization
signal
sparse
special
testing Prevent users from seeing hardcoded print stmt when hypothesis is not installed (#142398) 2024-12-17 16:59:05 +00:00
utils remove allow-untyped-defs for torch/utils/benchmark/examples/simple_timeit.py (#143368) 2024-12-17 17:19:11 +00:00
xpu
__config__.py remove allow-untyped-defs for torch/__config__.py (#143320) 2024-12-17 00:16:09 +00:00
__future__.py
__init__.py
_appdirs.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_classes.py
_compile.py
_custom_ops.py
_deploy.py
_environment.py
_guards.py add private config to temporarily preserve old FSDP guard behavior (#142871) 2024-12-13 22:06:48 +00:00
_jit_internal.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_linalg_utils.py
_lobpcg.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_lowrank.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_meta_registrations.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_namedtensor_internals.py
_ops.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_python_dispatcher.py
_size_docs.py
_sources.py
_storage_docs.py
_streambase.py
_tensor.py __cuda_array_interface__: Use "<V2" for bfloat16. (#143042) 2024-12-14 06:27:52 +00:00
_tensor_docs.py
_tensor_str.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
_thread_safe_fork.py
_torch_docs.py Add torch.cat tensors type promotion description (#141339) 2024-12-14 01:36:41 +00:00
_utils.py Reraise worker errors as runtime errors in more cases when the original exception can't be constructed (#140911) 2024-12-14 03:11:36 +00:00
_utils_internal.py Revert "Kill capture_pre_autograd_graph API (#143224)" 2024-12-17 00:47:24 +00:00
_VF.py
_vmap_internals.py
_weights_only_unpickler.py Remove unused Python variables in torch/[_-a]* (#133492) 2024-12-12 17:39:14 +00:00
abi-check.cpp
CMakeLists.txt Hide torch_python symbols (#142214) 2024-12-16 00:59:26 +00:00
custom_class.h
custom_class_detail.h
extension.h
functional.py
hub.py
library.h
library.py make it clearer (in docs) one can double decorate with torch.library.impl_* APIs (#137608) 2024-12-17 15:13:58 +00:00
overrides.py
py.typed
quasirandom.py
random.py
README.txt
return_types.py
script.h
serialization.py
storage.py
torch_version.py
types.py
version.py.tpl

Note [TH abstraction violation]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TH/THC provide some hpp headers, which are proper C++ headers rather than
C headers.  These headers serve double duty as *internal implementation
detail* headers, whose contents should largely not be used by external
clients.

Ideally, we would not install these headers at all; instead, you should
use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`)
to manipulate these structs.  However, there are a few places
in torch/csrc where we violate this abstraction.  They are marked with
a pointer to this note.  Each of those sites will have to be refactored
when we refactor the guts of THTensor and related structures.