saymrwulf/pytorch
1
0
Fork 0
mirror of https://github.com/saymrwulf/pytorch.git synced 2026-05-14 20:57:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
pytorch/torch/csrc/jit
History
Mike Iovine 008469c5e2 [SR] Simplify memory re-use algorithm (#68302) 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/68302

Implement the new memory re-use algorithm. It’s roughly based on the c2 one, but after going through many iterations it may not be a 1:1 port anymore. Also deleted the old liveness analysis.

Test Plan:
## **Re-use metrics**

`inline_cvr` (294738512_58)
**Before**
* `local`
```
Total number of managed tensors: 2660
Total number of managed output tensors: 0
Total number of unmanaged values: 3041
Total memory managed: 4601984 bytes
Total number of reused tensors: 1183
```
* `local_ro`
```
Total number of managed tensors: 1412
Total number of managed output tensors: 0
Total number of unmanaged values: 2677
Total memory managed: 29696 bytes
Total number of reused tensors: 959
```

**After**
* `local`
```
Total number of managed tensors: 2660
Total number of managed output tensors: 0
Total number of unmanaged values: 3041
Total memory managed: 4520000 bytes
Total number of reused tensors: 1198
```
* `local_ro`
```
Total number of managed tensors: 1412
Total number of managed output tensors: 0
Total number of unmanaged values: 2677
Total memory managed: 29120 bytes
Total number of reused tensors: 963
```

Reviewed By: hlu1

Differential Revision: D32370424

fbshipit-source-id: 06a8e0a295ed7a2b4d14071349c1f1e975f746bf
2021-12-07 13:25:42 -08:00
..
api [JIT] optimize_for_inference on methods other than forward (#69367) 2021-12-07 12:36:47 -08:00
backends [Core ML] Avoid recompiling models when the OS version is not changed (#69438) 2021-12-06 00:49:51 -08:00
codegen
cuda
docs
frontend
ir Add TE fuser ops (#68825) 2021-12-01 00:43:42 -08:00
mobile [Pytorch Edge] Add new_empty_strided to tracer (#69492) 2021-12-06 15:28:13 -08:00
operator_upgraders Initialize upgrader and operator version files (#68772) 2021-12-06 16:27:52 -08:00
passes [JIT] Separate GPU implementation of frozen_conv_add_relu_fusion.cpp (#68149) 2021-12-06 21:06:25 -08:00
python [JIT] optimize_for_inference on methods other than forward (#69367) 2021-12-07 12:36:47 -08:00
runtime [SR] Simplify memory re-use algorithm (#68302) 2021-12-07 13:25:42 -08:00
serialization Expose MobileCode to python (#66592) 2021-12-02 13:18:46 -08:00
tensorexpr [TensorExpr] Add lowerings for quantized ops: cat, mul, conv1d, relu. (#69055) 2021-12-02 14:34:21 -08:00
testing Revert D32010095: [pytorch][PR] Add ability for a mobile::Module to save as flatbuffer 2021-12-02 06:41:40 -08:00
JIT-AUTOCAST.md
jit_log.cpp
jit_log.h
jit_opt_limit.cpp
jit_opt_limit.h
OVERVIEW.md
README.md
resource_guard.h

README.md

PyTorch JIT

This folder contains (most of) the C++ code for the PyTorch JIT, a language and compiler stack for executing PyTorch models portably and efficiently. To learn more about the JIT from a user perspective, please consult our reference documentation and tutorials.

A brief summary of the source tree:

  • OVERVIEW.md: High-level technical overview of the JIT.
  • frontend/: Taking PyTorch modules in Python and translating them into the JIT IR.
  • ir/: Core IR abstractions.
  • runtime/: Interpreter, graph execution, and JIT operators.
  • codegen/: Generating efficient, hardware-specific code for JIT subgraphs.
  • serialization/: Saving and loading modules.
  • api/: Any user-facing C++ or Python interfaces.
  • python/: Binding stuff into Python or accessing information from the Python environment.
  • testing/: Utilities and helpers for testing.
  • mobile/: Mobile-specific implementations of runtime components.
  • passes/: IR-to-IR passes, generally for optimization and lowering.
  • generated/: This folder is generated by the PyTorch build, and contains bindings for native PyTorch operators into the JIT.

Refer to each folder for more in-depth documentation.

Other relevant parts of the codebase not contained here:

  • aten/src/ATen/core: contains JIT code re-used by other elements of the runtime system (eager, mobile, etc.)