jingyanwangms 5dcaf70501 Adding this set_to_none flag to zero_grad to have signature parity with pytorch Adam (#16375) ... ### Description torch.optim Adam zero_grad() signature is zero_grad(set_to_none=True) https://pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam.zero_grad We set this flag in initialization, similar to deepspeed: https://deepspeed.readthedocs.io/en/latest/optimizers.html#deepspeed.ops.adam.FusedAdam Adding this flag to have signature parity with pytorch Adam ### Motivation and Context Easier model integration Co-authored-by: Jingyan Wang <jingywa@microsoft.com@orttrainingdev7.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>		2023-06-19 17:27:41 -07:00
..
__init__.py
_apex_amp_modifier.py	Enable pylint and numpy rules (#15218)	2023-03-27 20:37:53 -07:00
_ds_modifier.py	support latest deepspeed version for optim (#15682)	2023-04-25 20:12:23 -07:00
_megatron_modifier.py
_modifier.py
_modifier_registry.py
_multi_tensor_apply.py
config.py	Bump ruff in CI (#15533)	2023-04-17 10:11:44 -07:00
fp16_optimizer.py	Adopt linrtunner as the linting tool - take 2 (#15085)	2023-03-24 15:29:03 -07:00
fused_adam.py	Adding this set_to_none flag to zero_grad to have signature parity with pytorch Adam (#16375)	2023-06-19 17:27:41 -07:00
lr_scheduler.py