Shangdi Yu 379b54603a [Inductor] [bc-breaking] Node Level provenance tracking (#144277) ... Summary: - use GraphTransformObserver + replace_node hooks to track node sources when they are replaced - add pre_grad_graph tracking to tlparse - add the node provenance information to post_grad_graph tlparse. This is for the frontend to create a mapping between pre_grad and post_grad graph. See an example frontend (this is just a prototype) here: https://drive.google.com/file/d/1cMHH_0y4FJUSS9tATwGQvA72O0Lth8eh/view?usp=sharing - change "action" of NodeSource from a single action to a list of actions. - It's BC-Breaking because we removed `GraphTransformObserver`'s class methods `on_node_erase` and `on_node_erase` . https://docs.google.com/document/d/1dGh9myqNhywmbfP0Quzx_f04bghDFlj8cawj8MopiO8/edit?tab=t.0 The front-end code that takes in the tlparse result is in https://github.com/yushangdi/compiler_explorer. ghstack-source-id: 260390519 Test Plan: ``` buck2 run mode/dev-nosan fbcode//caffe2/test:fx -- -r test_graph_transform_observer buck run mode/dev-nosan fbcode//caffe2/test:fx -- -r node_source buck run mode/dev-nosan fbcode//caffe2/test:fx -- -r graph_provenance ``` Front-end example screenshots on a real model, 93% coverage rate between pre_grad_graph and post_grad_graph {F1973584210}{F1973584209} ``` buck2 build --show-output mode/opt -c=python.package_style=inplace -c fbcode.enable_gpu_sections=true -c fbcode.platform=platform010 -c fbcode.split-dwarf=true -c fbcode.nvcc_arch=a100,h100 caffe2/torch/fb/model_transform/experimental/benchmark:mts_gpu_benchmark MODEL_ENTITY_ID=644688112 SNAPSHOT_ID=32 MODULE=merge TORCH_COMPILE_DEBUG=1 CUDA_VISIBLE_DEVICES=7 TORCH_LOGS="+inductor,+schedule,output_code,graph_code" TORCHINDUCTOR_MAX_AUTOTUNE=1 TORCHINDUCTOR_UNIQUE_KERNEL_NAMES=1 ../buck-out/v2/gen/fbcode/ec86b05dd59e84db/caffe2/torch/fb/model_transform/experimental/benchmark/__mts_gpu_benchmark__/mts_gpu_benchmark.par --local-model /home/bahuang/models/${MODEL_ENTITY_ID}/${SNAPSHOT_ID}/gpu_lowering/input.predictor.disagg.gpu.merge --lower-backend AOT_INDUCTOR_EP --gpu-trace --aot-inductor-config="{'max_autotune': True}" buck2 run mode/dev-nosan fbcode//caffe2/test/inductor:auto_functionalize ``` Differential Revision: D65006709 Pull Request resolved: https://github.com/pytorch/pytorch/pull/144277 Approved by: https://github.com/desertfire		2025-01-09 22:06:51 +00:00
..
named_tup.py
quantization.py	Fix unused Python variables in test/[e-z]* (#136964)	2024-12-18 23:02:30 +00:00
test_common_passes.py	[BE][Easy][9/19] enforce style for empty lines in import segments in test/[e-h]*/ (#129760)	2024-07-17 14:25:29 +00:00
test_cse_pass.py	Fix unused Python variables in test/[e-z]* (#136964)	2024-12-18 23:02:30 +00:00
test_dce_pass.py	Fix unused Python variables in test/[e-z]* (#136964)	2024-12-18 23:02:30 +00:00
test_future.py
test_fx_const_fold.py	Add None return type to init -- tests (#132352)	2024-08-01 15:44:51 +00:00
test_fx_node_hook.py	[export] Change fx graph _replace_hook to a list of Callable (#142006)	2024-12-05 03:26:48 +00:00
test_fx_param_shape_control_flow.py	[BE][Easy][9/19] enforce style for empty lines in import segments in test/[e-h]*/ (#129760)	2024-07-17 14:25:29 +00:00
test_fx_split.py	[Codemod][AddExplicitStrictExportArg] caffe2/test (#143688)	2024-12-27 07:58:44 +00:00
test_fx_traceback.py	[Inductor] [bc-breaking] Node Level provenance tracking (#144277)	2025-01-09 22:06:51 +00:00
test_fx_xform_observer.py	[Inductor] [bc-breaking] Node Level provenance tracking (#144277)	2025-01-09 22:06:51 +00:00
test_gradual_type.py	Fix unused Python variables in test/[e-z]* (#136964)	2024-12-18 23:02:30 +00:00
test_lazy_graph_module.py
test_matcher_utils.py	Fix unused Python variables in test/[e-z]* (#136964)	2024-12-18 23:02:30 +00:00
test_partitioner_order.py	[4/N] Apply py39 ruff and pyupgrade fixes (#143257)	2025-01-04 10:47:51 +00:00
test_pass_infra.py	[BE][Easy][9/19] enforce style for empty lines in import segments in test/[e-h]*/ (#129760)	2024-07-17 14:25:29 +00:00
test_shape_inference.py	Add None return type to init -- tests (#132352)	2024-08-01 15:44:51 +00:00
test_source_matcher_utils.py	Fix get_source_partitions when weights are tied (#142446)	2025-01-04 04:28:20 +00:00
test_subgraph_rewriter.py	subgraph rewriter supports matched pattern with no users (#143842)	2024-12-27 12:45:39 +00:00
test_z3_gradual_types.py	Fix unused Python variables in test/[e-z]* (#136964)	2024-12-18 23:02:30 +00:00