Summary: Unit Test to validate the stateful and non-stateful loads. This test is a follow up to the fix in [#138575](https://github.com/pytorch/pytorch/pull/138575) which addresses an issue in stateful dict's in-place updates in distributed checkpoint loading. Also, added additional code comments regarding the stateful and non-stateful loads. Test Plan: ``` buck2 test //caffe2/test/distributed/checkpoint/e2e:test_e2e_save_and_load ``` https://www.internalfb.com/intern/testinfra/testrun/8162774562859797 Differential Revision: D65188659 Pull Request resolved: https://github.com/pytorch/pytorch/pull/139251 Approved by: https://github.com/LucasLLC, https://github.com/fegin