quantization: autogenerate quantization backend configs for documentation (#75126)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75126
Quantization has a high volume of configurations of how to quantize an
op for a reference model representation which is useful for a lowering
step for a backend. An example of this is
```
{'dtype_configs': [{'input_dtype': torch.quint8,
'output_dtype': torch.quint8}],
'observation_type': <ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT: 0>,
'pattern': <class 'torch.nn.modules.conv.ConvTranspose1d'>},
```
These configs are checked into master, and they are created with Python functions.
Therefore, there is no easy way for the user to see what the configs actually
are without running some Python code.
This PR is one approach to document these configs. Here is what this is doing:
1. during documentation build, write a text file of the configs
2. render that text file on a quantization page, with some additional context
In the future, this could be extended to autogenerate better looking tables
such as: op support per backend and dtype, op support per valid quantization settings per backend,
etc.
Test Plan:
```
cd docs
make html
cd html
python -m http.server 8000
// render http://[::]:8000/quantization-backend-configuration.html
// it renders correctly
```
Reviewed By: ejguan
Differential Revision: D35365461
Pulled By: vkuzo
fbshipit-source-id: d60f776ccb57da9db3d09550e4b27bd5e725635a
(cherry picked from commit 14865c0e23bc080120342c8f9278f0fae8eb8fbd)
2022-04-04 22:17:26 +00:00
|
|
|
"""
|
|
|
|
|
This script will generate default values of quantization configs.
|
|
|
|
|
These are for use in the documentation.
|
|
|
|
|
"""
|
|
|
|
|
|
quant doc: improve rendered documentation for backend_config_dict
Summary:
This improves the documentation page for backend_config_dict to render
the configurations in a human readable format, such as
```
{
'pattern': torch.nn.modules.pooling.AdaptiveAvgPool1d,
'dtype_configs': [
{
'input_dtype': torch.quint8,
'output_dtype': torch.quint8,
},
{
'input_dtype': torch.float16,
'weight_dtype': torch.float16,
'bias_dtype': torch.float16,
'output_dtype': torch.float16,
},
],
'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
},
```
The results are also now sorted alphabetically by the normalized name of
the root op in the pattern.
A couple of utility functions are created to help with this. If in the future
we convert backend_config_dict to use typed objects, we can move this logic
to the objects at that time.
Test plan:
```
cd docs
make html
cd build
python -m server.http
// renders correctly, example: https://gist.github.com/vkuzo/76adfc7c89e119c59813a733fa2cd56f
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77535
Approved by: https://github.com/andrewor14
2022-05-17 12:03:57 +00:00
|
|
|
import torch
|
2022-04-18 22:20:09 +00:00
|
|
|
from torch.ao.quantization.backend_config import get_native_backend_config_dict
|
quant doc: improve rendered documentation for backend_config_dict
Summary:
This improves the documentation page for backend_config_dict to render
the configurations in a human readable format, such as
```
{
'pattern': torch.nn.modules.pooling.AdaptiveAvgPool1d,
'dtype_configs': [
{
'input_dtype': torch.quint8,
'output_dtype': torch.quint8,
},
{
'input_dtype': torch.float16,
'weight_dtype': torch.float16,
'bias_dtype': torch.float16,
'output_dtype': torch.float16,
},
],
'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
},
```
The results are also now sorted alphabetically by the normalized name of
the root op in the pattern.
A couple of utility functions are created to help with this. If in the future
we convert backend_config_dict to use typed objects, we can move this logic
to the objects at that time.
Test plan:
```
cd docs
make html
cd build
python -m server.http
// renders correctly, example: https://gist.github.com/vkuzo/76adfc7c89e119c59813a733fa2cd56f
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77535
Approved by: https://github.com/andrewor14
2022-05-17 12:03:57 +00:00
|
|
|
from torch.ao.quantization.backend_config.utils import (
|
|
|
|
|
entry_to_pretty_str,
|
|
|
|
|
remove_boolean_dispatch_from_name,
|
|
|
|
|
)
|
quantization: autogenerate quantization backend configs for documentation (#75126)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75126
Quantization has a high volume of configurations of how to quantize an
op for a reference model representation which is useful for a lowering
step for a backend. An example of this is
```
{'dtype_configs': [{'input_dtype': torch.quint8,
'output_dtype': torch.quint8}],
'observation_type': <ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT: 0>,
'pattern': <class 'torch.nn.modules.conv.ConvTranspose1d'>},
```
These configs are checked into master, and they are created with Python functions.
Therefore, there is no easy way for the user to see what the configs actually
are without running some Python code.
This PR is one approach to document these configs. Here is what this is doing:
1. during documentation build, write a text file of the configs
2. render that text file on a quantization page, with some additional context
In the future, this could be extended to autogenerate better looking tables
such as: op support per backend and dtype, op support per valid quantization settings per backend,
etc.
Test Plan:
```
cd docs
make html
cd html
python -m http.server 8000
// render http://[::]:8000/quantization-backend-configuration.html
// it renders correctly
```
Reviewed By: ejguan
Differential Revision: D35365461
Pulled By: vkuzo
fbshipit-source-id: d60f776ccb57da9db3d09550e4b27bd5e725635a
(cherry picked from commit 14865c0e23bc080120342c8f9278f0fae8eb8fbd)
2022-04-04 22:17:26 +00:00
|
|
|
import os.path
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Create a directory for the images, if it doesn't exist
|
|
|
|
|
QUANTIZATION_BACKEND_CONFIG_IMAGE_PATH = os.path.join(
|
|
|
|
|
os.path.realpath(os.path.join(__file__, "..")),
|
|
|
|
|
"quantization_backend_configs"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
if not os.path.exists(QUANTIZATION_BACKEND_CONFIG_IMAGE_PATH):
|
|
|
|
|
os.mkdir(QUANTIZATION_BACKEND_CONFIG_IMAGE_PATH)
|
|
|
|
|
|
|
|
|
|
output_path = os.path.join(QUANTIZATION_BACKEND_CONFIG_IMAGE_PATH, "default_backend_config.txt")
|
|
|
|
|
|
|
|
|
|
with open(output_path, "w") as f:
|
quant doc: improve rendered documentation for backend_config_dict
Summary:
This improves the documentation page for backend_config_dict to render
the configurations in a human readable format, such as
```
{
'pattern': torch.nn.modules.pooling.AdaptiveAvgPool1d,
'dtype_configs': [
{
'input_dtype': torch.quint8,
'output_dtype': torch.quint8,
},
{
'input_dtype': torch.float16,
'weight_dtype': torch.float16,
'bias_dtype': torch.float16,
'output_dtype': torch.float16,
},
],
'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
},
```
The results are also now sorted alphabetically by the normalized name of
the root op in the pattern.
A couple of utility functions are created to help with this. If in the future
we convert backend_config_dict to use typed objects, we can move this logic
to the objects at that time.
Test plan:
```
cd docs
make html
cd build
python -m server.http
// renders correctly, example: https://gist.github.com/vkuzo/76adfc7c89e119c59813a733fa2cd56f
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77535
Approved by: https://github.com/andrewor14
2022-05-17 12:03:57 +00:00
|
|
|
native_backend_config_dict = get_native_backend_config_dict()
|
|
|
|
|
|
|
|
|
|
configs = native_backend_config_dict['configs']
|
|
|
|
|
|
|
|
|
|
def _sort_key_func(entry):
|
|
|
|
|
pattern = entry['pattern']
|
|
|
|
|
while isinstance(pattern, tuple):
|
|
|
|
|
pattern = pattern[-1]
|
|
|
|
|
|
|
|
|
|
pattern = remove_boolean_dispatch_from_name(pattern)
|
|
|
|
|
if not isinstance(pattern, str):
|
|
|
|
|
# methods are already strings
|
|
|
|
|
pattern = torch.typename(pattern)
|
|
|
|
|
|
|
|
|
|
# we want
|
|
|
|
|
#
|
|
|
|
|
# torch.nn.modules.pooling.AdaptiveAvgPool1d
|
|
|
|
|
#
|
|
|
|
|
# and
|
|
|
|
|
#
|
|
|
|
|
# torch._VariableFunctionsClass.adaptive_avg_pool1d
|
|
|
|
|
#
|
|
|
|
|
# to be next to each other, so convert to all lower case
|
|
|
|
|
# and remove the underscores, and compare the last part
|
|
|
|
|
# of the string
|
|
|
|
|
pattern_str_normalized = pattern.lower().replace('_', '')
|
|
|
|
|
key = pattern_str_normalized.split('.')[-1]
|
|
|
|
|
return key
|
|
|
|
|
|
|
|
|
|
configs.sort(key=_sort_key_func)
|
|
|
|
|
|
|
|
|
|
entries = []
|
|
|
|
|
for entry in configs:
|
|
|
|
|
entries.append(entry_to_pretty_str(entry))
|
|
|
|
|
entries = ",\n".join(entries)
|
|
|
|
|
f.write(entries)
|