skip three pyhpc models with dynamic shape test (#120599)

As reported in https://github.com/pytorch/pytorch/issues/119434, `pyhpc_isoneutral_mixing`, `pyhpc_equation_of_state` and `pyhpc_turbulent_kinetic_energy` failed with dynamic shape testing, we propose to skip the dynamic batch size testing of these 3 models in this PR.

* Error msg is
```
  File "/localdisk/leslie/torch_inductor_community/pytorch/benchmarks/dynamo/common.py", line 3879, in run
    assert marked, f"nothing in example_inputs had a dim with {batch_size}"
AssertionError: nothing in example_inputs had a dim with 1048576
```

* Root Cause is
  *  Benchmark code will only annotate the inputs' dim as dynamic when its size equals to batch size c617e7b407/benchmarks/dynamo/common.py (L3867-L3871). If it fails to find any dim equals to batch size, above error throws.
  * However, for these 3 models, none of the inputs' dim will equal to input batch size since the [relationship of dim sizes](26b85eadde/torchbenchmark/models/pyhpc_equation_of_state/__init__.py (L12-L16))
  ```
    shape = (
        math.ceil(2 * size ** (1/3)),
        math.ceil(2 * size ** (1/3)),
        math.ceil(0.25 * size ** (1/3)),
    )
  ```
  * Another thing is `pyhpc_isoneutral_mixing`, `pyhpc_equation_of_state` can pass the dynamic batch size accuracy testing, because the batch size has been set to 4 in accuracy testing (c617e7b407/benchmarks/dynamo/common.py (L3456)) and `math.ceil(2 * size ** (1/3))` happens equaling to 4.

* Since the dim sizes of input has above relationship, running the these models in dynamic shape, we may need to annotate `dim[0](s0) = dim[2](s1) * 8`, per the discussion in https://github.com/pytorch/pytorch/issues/117477#issuecomment-1897108756 @avikchaudhuri, looks like we are not expressible for this case. So, I think we may need to skip the dynamic batch size testing for these 3 models.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/120599
Approved by: https://github.com/jgong5, https://github.com/desertfire
This commit is contained in:
leslie-fang-intel 2024-02-28 10:14:34 +08:00 committed by PyTorch MergeBot
parent 3179107629
commit 950b484356
4 changed files with 15 additions and 0 deletions

View file

@ -262,6 +262,10 @@ pyhpc_isoneutral_mixing,pass,0
pyhpc_turbulent_kinetic_energy,pass,0
pytorch_CycleGAN_and_pix2pix,pass,0

1 name accuracy graph_breaks
262
263
264
265
266
267
268
269
270
271

View file

@ -190,6 +190,10 @@ pyhpc_isoneutral_mixing,pass,0
pyhpc_turbulent_kinetic_energy,pass,0
pytorch_CycleGAN_and_pix2pix,pass,0

1 name accuracy graph_breaks
190
191
192
193
194
195
196
197
198
199

View file

@ -262,6 +262,10 @@ pyhpc_isoneutral_mixing,pass,0
pyhpc_turbulent_kinetic_energy,pass,0
pytorch_CycleGAN_and_pix2pix,pass,0

1 name accuracy graph_breaks
262
263
264
265
266
267
268
269
270
271

View file

@ -135,6 +135,9 @@ CI_SKIP_DYNAMIC_BATCH_ONLY = {
# We should be able to graphbreak there.
"doctr_det_predictor",
"dlrm",
"pyhpc_isoneutral_mixing",
"pyhpc_equation_of_state",
"pyhpc_turbulent_kinetic_energy",
}
# These models currently fail accuracy with eager Adam optimizer