Fix various typos (#1926)

* Fix various typos

* Update changelog

---------

Co-authored-by: Antonin Raffin <antonin.raffin@ensta.org>
This commit is contained in:
Chris Schindlbeck 2024-05-15 15:19:39 +02:00 committed by GitHub
parent 766b9e9f7d
commit 4317c62598
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
23 changed files with 32 additions and 31 deletions

View file

@ -192,7 +192,7 @@ All the following examples can be executed online using Google Colab notebooks:
<b id="f1">1</b>: Implemented in [SB3 Contrib](https://github.com/Stable-Baselines-Team/stable-baselines3-contrib) GitHub repository. <b id="f1">1</b>: Implemented in [SB3 Contrib](https://github.com/Stable-Baselines-Team/stable-baselines3-contrib) GitHub repository.
Actions `gym.spaces`: Actions `gym.spaces`:
* `Box`: A N-dimensional box that containes every point in the action space. * `Box`: A N-dimensional box that contains every point in the action space.
* `Discrete`: A list of possible actions, where each timestep only one of the actions can be used. * `Discrete`: A list of possible actions, where each timestep only one of the actions can be used.
* `MultiDiscrete`: A list of possible actions, where each timestep only one action of each discrete set can be used. * `MultiDiscrete`: A list of possible actions, where each timestep only one action of each discrete set can be used.
* `MultiBinary`: A list of possible actions, where each timestep any of the actions can be used in any combination. * `MultiBinary`: A list of possible actions, where each timestep any of the actions can be used in any combination.

View file

@ -128,7 +128,7 @@ Multiprocessing: Unleashing the Power of Vectorized Environments
:param env_id: the environment ID :param env_id: the environment ID
:param num_env: the number of environments you wish to have in subprocesses :param num_env: the number of environments you wish to have in subprocesses
:param seed: the inital seed for RNG :param seed: the initial seed for RNG
:param rank: index of the subprocess :param rank: index of the subprocess
""" """
def _init(): def _init():

View file

@ -30,6 +30,7 @@ Deprecations:
Others: Others:
^^^^^^^ ^^^^^^^
- Fix various typos (@cschindlbeck)
Bug Fixes: Bug Fixes:
^^^^^^^^^^ ^^^^^^^^^^
@ -403,7 +404,7 @@ Breaking Changes:
^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
- Removed shared layers in ``mlp_extractor`` (@AlexPasqua) - Removed shared layers in ``mlp_extractor`` (@AlexPasqua)
- Refactored ``StackedObservations`` (it now handles dict obs, ``StackedDictObservations`` was removed) - Refactored ``StackedObservations`` (it now handles dict obs, ``StackedDictObservations`` was removed)
- You must now explicitely pass a ``features_extractor`` parameter when calling ``extract_features()`` - You must now explicitly pass a ``features_extractor`` parameter when calling ``extract_features()``
- Dropped offline sampling for ``HerReplayBuffer`` - Dropped offline sampling for ``HerReplayBuffer``
- As ``HerReplayBuffer`` was refactored to support multiprocessing, previous replay buffer are incompatible with this new version - As ``HerReplayBuffer`` was refactored to support multiprocessing, previous replay buffer are incompatible with this new version
- ``HerReplayBuffer`` doesn't require a ``max_episode_length`` anymore - ``HerReplayBuffer`` doesn't require a ``max_episode_length`` anymore
@ -535,7 +536,7 @@ Bug Fixes:
Deprecations: Deprecations:
^^^^^^^^^^^^^ ^^^^^^^^^^^^^
- You should now explicitely pass a ``features_extractor`` parameter when calling ``extract_features()`` - You should now explicitly pass a ``features_extractor`` parameter when calling ``extract_features()``
- Deprecated shared layers in ``MlpExtractor`` (@AlexPasqua) - Deprecated shared layers in ``MlpExtractor`` (@AlexPasqua)
Others: Others:
@ -746,7 +747,7 @@ Bug Fixes:
- Fixed a bug in ``HumanOutputFormat``. Distinct keys truncated to the same prefix would overwrite each others value, - Fixed a bug in ``HumanOutputFormat``. Distinct keys truncated to the same prefix would overwrite each others value,
resulting in only one being output. This now raises an error (this should only affect a small fraction of use cases resulting in only one being output. This now raises an error (this should only affect a small fraction of use cases
with very long keys.) with very long keys.)
- Routing all the ``nn.Module`` calls through implicit rather than explict forward as per pytorch guidelines (@manuel-delverme) - Routing all the ``nn.Module`` calls through implicit rather than explicit forward as per pytorch guidelines (@manuel-delverme)
- Fixed a bug in ``VecNormalize`` where error occurs when ``norm_obs`` is set to False for environment with dictionary observation (@buoyancy99) - Fixed a bug in ``VecNormalize`` where error occurs when ``norm_obs`` is set to False for environment with dictionary observation (@buoyancy99)
- Set default ``env`` argument to ``None`` in ``HerReplayBuffer.sample`` (@qgallouedec) - Set default ``env`` argument to ``None`` in ``HerReplayBuffer.sample`` (@qgallouedec)
- Fix ``batch_size`` typing in ``DQN`` (@qgallouedec) - Fix ``batch_size`` typing in ``DQN`` (@qgallouedec)
@ -1658,4 +1659,4 @@ And all the contributors:
@anand-bala @hughperkins @sidney-tio @AlexPasqua @dominicgkerr @Akhilez @Rocamonde @tobirohrer @ZikangXiong @ReHoss @anand-bala @hughperkins @sidney-tio @AlexPasqua @dominicgkerr @Akhilez @Rocamonde @tobirohrer @ZikangXiong @ReHoss
@DavyMorgan @luizapozzobon @Bonifatius94 @theSquaredError @harveybellini @DavyMorgan @FieteO @jonasreiher @npit @WeberSamuel @troiganto @DavyMorgan @luizapozzobon @Bonifatius94 @theSquaredError @harveybellini @DavyMorgan @FieteO @jonasreiher @npit @WeberSamuel @troiganto
@lutogniew @lbergmann1 @lukashass @BertrandDecoster @pseudo-rnd-thoughts @stefanbschneider @kyle-he @PatrickHelm @corentinlger @lutogniew @lbergmann1 @lukashass @BertrandDecoster @pseudo-rnd-thoughts @stefanbschneider @kyle-he @PatrickHelm @corentinlger
@marekm4 @stagoverflow @rushitnshah @markscsmith @NickLucche @marekm4 @stagoverflow @rushitnshah @markscsmith @NickLucche @cschindlbeck

View file

@ -48,7 +48,7 @@ def maybe_make_env(env: Union[GymEnv, str], verbose: int) -> GymEnv:
"""If env is a string, make the environment; otherwise, return env. """If env is a string, make the environment; otherwise, return env.
:param env: The environment to learn from. :param env: The environment to learn from.
:param verbose: Verbosity level: 0 for no output, 1 for indicating if envrironment is created :param verbose: Verbosity level: 0 for no output, 1 for indicating if environment is created
:return A Gym (vector) environment. :return A Gym (vector) environment.
""" """
if isinstance(env, str): if isinstance(env, str):

View file

@ -606,7 +606,7 @@ class StopTrainingOnMaxEpisodes(BaseCallback):
self.n_episodes = 0 self.n_episodes = 0
def _init_callback(self) -> None: def _init_callback(self) -> None:
# At start set total max according to number of envirnments # At start set total max according to number of environments
self._total_max_episodes = self.max_episodes * self.training_env.num_envs self._total_max_episodes = self.max_episodes * self.training_env.num_envs
def _on_step(self) -> bool: def _on_step(self) -> bool:

View file

@ -397,7 +397,7 @@ def _check_render(env: gym.Env, warn: bool = False) -> None: # pragma: no cover
"you may have trouble when calling `.render()`" "you may have trouble when calling `.render()`"
) )
# Only check currrent render mode # Only check current render mode
if env.render_mode: if env.render_mode:
env.render() env.render()
env.close() env.close()

View file

@ -189,7 +189,7 @@ class ResultsWriter:
filename = os.path.realpath(filename) filename = os.path.realpath(filename)
# Create (if any) missing filename directories # Create (if any) missing filename directories
os.makedirs(os.path.dirname(filename), exist_ok=True) os.makedirs(os.path.dirname(filename), exist_ok=True)
# Append mode when not overridding existing file # Append mode when not overriding existing file
mode = "w" if override_existing else "a" mode = "w" if override_existing else "a"
# Prevent newline issue on Windows, see GH issue #692 # Prevent newline issue on Windows, see GH issue #692
self.file_handler = open(filename, f"{mode}t", newline="\n") self.file_handler = open(filename, f"{mode}t", newline="\n")

View file

@ -922,7 +922,7 @@ class ContinuousCritic(BaseModel):
By default, it creates two critic networks used to reduce overestimation By default, it creates two critic networks used to reduce overestimation
thanks to clipped Q-learning (cf TD3 paper). thanks to clipped Q-learning (cf TD3 paper).
:param observation_space: Obervation space :param observation_space: Observation space
:param action_space: Action space :param action_space: Action space
:param net_arch: Network architecture :param net_arch: Network architecture
:param features_extractor: Network to extract features :param features_extractor: Network to extract features

View file

@ -46,7 +46,7 @@ def window_func(var_1: np.ndarray, var_2: np.ndarray, window: int, func: Callabl
def ts2xy(data_frame: pd.DataFrame, x_axis: str) -> Tuple[np.ndarray, np.ndarray]: def ts2xy(data_frame: pd.DataFrame, x_axis: str) -> Tuple[np.ndarray, np.ndarray]:
""" """
Decompose a data frame variable to x ans ys Decompose a data frame variable to x and ys
:param data_frame: the input data :param data_frame: the input data
:param x_axis: the axis for the x and y output :param x_axis: the axis for the x and y output

View file

@ -6,7 +6,7 @@ import numpy as np
class RunningMeanStd: class RunningMeanStd:
def __init__(self, epsilon: float = 1e-4, shape: Tuple[int, ...] = ()): def __init__(self, epsilon: float = 1e-4, shape: Tuple[int, ...] = ()):
""" """
Calulates the running mean and std of a data stream Calculates the running mean and std of a data stream
https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Parallel_algorithm
:param epsilon: helps with arithmetic issues :param epsilon: helps with arithmetic issues

View file

@ -189,7 +189,7 @@ class MlpExtractor(nn.Module):
# save dimensions of layers in policy and value nets # save dimensions of layers in policy and value nets
if isinstance(net_arch, dict): if isinstance(net_arch, dict):
# Note: if key is not specificed, assume linear network # Note: if key is not specified, assume linear network
pi_layers_dims = net_arch.get("pi", []) # Layer sizes of the policy network pi_layers_dims = net_arch.get("pi", []) # Layer sizes of the policy network
vf_layers_dims = net_arch.get("vf", []) # Layer sizes of the value network vf_layers_dims = net_arch.get("vf", []) # Layer sizes of the value network
else: else:

View file

@ -24,7 +24,7 @@ MaybeCallback = Union[None, Callable, List["BaseCallback"], "BaseCallback"]
PyTorchObs = Union[th.Tensor, TensorDict] PyTorchObs = Union[th.Tensor, TensorDict]
# A schedule takes the remaining progress as input # A schedule takes the remaining progress as input
# and ouputs a scalar (e.g. learning rate, clip range, ...) # and outputs a scalar (e.g. learning rate, clip range, ...)
Schedule = Callable[[float], float] Schedule = Callable[[float], float]

View file

@ -71,7 +71,7 @@ def _convert_space(space: Union["gym.Space", gymnasium.Space]) -> gymnasium.Spac
:return: Patched space (gymnasium Space) :return: Patched space (gymnasium Space)
""" """
# Gymnasium space, no convertion to be done # Gymnasium space, no conversion to be done
if isinstance(space, gymnasium.Space): if isinstance(space, gymnasium.Space):
return space return space

View file

@ -111,7 +111,7 @@ class VecNormalize(VecEnvWrapper):
raise ValueError( raise ValueError(
f"VecNormalize only supports `gym.spaces.Box` observation spaces but {obs_key} " f"VecNormalize only supports `gym.spaces.Box` observation spaces but {obs_key} "
f"is of type {self.observation_space.spaces[obs_key]}. " f"is of type {self.observation_space.spaces[obs_key]}. "
"You should probably explicitely pass the observation keys " "You should probably explicitly pass the observation keys "
" that should be normalized via the `norm_obs_keys` parameter." " that should be normalized via the `norm_obs_keys` parameter."
) )

View file

@ -255,7 +255,7 @@ class HerReplayBuffer(DictReplayBuffer):
Get the samples corresponding to the batch and environment indices. Get the samples corresponding to the batch and environment indices.
:param batch_indices: Indices of the transitions :param batch_indices: Indices of the transitions
:param env_indices: Indices of the envrionments :param env_indices: Indices of the environments
:param env: associated gym VecEnv to normalize the :param env: associated gym VecEnv to normalize the
observations/rewards when sampling, defaults to None observations/rewards when sampling, defaults to None
:return: Samples :return: Samples
@ -294,7 +294,7 @@ class HerReplayBuffer(DictReplayBuffer):
Get the samples, sample new desired goals and compute new rewards. Get the samples, sample new desired goals and compute new rewards.
:param batch_indices: Indices of the transitions :param batch_indices: Indices of the transitions
:param env_indices: Indices of the envrionments :param env_indices: Indices of the environments
:param env: associated gym VecEnv to normalize the :param env: associated gym VecEnv to normalize the
observations/rewards when sampling, defaults to None observations/rewards when sampling, defaults to None
:return: Samples, with new desired goals and new rewards :return: Samples, with new desired goals and new rewards
@ -357,7 +357,7 @@ class HerReplayBuffer(DictReplayBuffer):
Sample goals based on goal_selection_strategy. Sample goals based on goal_selection_strategy.
:param batch_indices: Indices of the transitions :param batch_indices: Indices of the transitions
:param env_indices: Indices of the envrionments :param env_indices: Indices of the environments
:return: Sampled goals :return: Sampled goals
""" """
batch_ep_start = self.ep_start[batch_indices, env_indices] batch_ep_start = self.ep_start[batch_indices, env_indices]

View file

@ -26,7 +26,7 @@ class Actor(BasePolicy):
""" """
Actor network (policy) for SAC. Actor network (policy) for SAC.
:param observation_space: Obervation space :param observation_space: Observation space
:param action_space: Action space :param action_space: Action space
:param net_arch: Network architecture :param net_arch: Network architecture
:param features_extractor: Network to extract features :param features_extractor: Network to extract features

View file

@ -21,7 +21,7 @@ class Actor(BasePolicy):
""" """
Actor network (policy) for TD3. Actor network (policy) for TD3.
:param observation_space: Obervation space :param observation_space: Observation space
:param action_space: Action space :param action_space: Action space
:param net_arch: Network architecture :param net_arch: Network architecture
:param features_extractor: Network to extract features :param features_extractor: Network to extract features

View file

@ -74,7 +74,7 @@ class DummyDictEnv(gym.Env):
@pytest.mark.parametrize("env_cls", [DummyEnv, DummyDictEnv]) @pytest.mark.parametrize("env_cls", [DummyEnv, DummyDictEnv])
def test_env(env_cls): def test_env(env_cls):
# Check the env used for testing # Check the env used for testing
# Do not warn for assymetric space # Do not warn for asymmetric space
check_env(env_cls(), warn=False, skip_render_check=True) check_env(env_cls(), warn=False, skip_render_check=True)
@ -86,7 +86,7 @@ def test_replay_buffer_normalization(replay_buffer_cls):
buffer = replay_buffer_cls(100, env.observation_space, env.action_space, device="cpu") buffer = replay_buffer_cls(100, env.observation_space, env.action_space, device="cpu")
# Interract and store transitions # Interact and store transitions
env.reset() env.reset()
obs = env.get_original_obs() obs = env.get_original_obs()
for _ in range(100): for _ in range(100):
@ -125,7 +125,7 @@ def test_device_buffer(replay_buffer_cls, device):
buffer = replay_buffer_cls(100, env.observation_space, env.action_space, device=device) buffer = replay_buffer_cls(100, env.observation_space, env.action_space, device=device)
# Interract and store transitions # Interact and store transitions
obs = env.reset() obs = env.reset()
for _ in range(100): for _ in range(100):
action = env.action_space.sample() action = env.action_space.sample()

View file

@ -161,7 +161,7 @@ def test_features_extractor_target_net(model_class, share_features_extractor):
if model_class == TD3: if model_class == TD3:
assert id(model.policy.actor_target.features_extractor) != id(model.policy.critic_target.features_extractor) assert id(model.policy.actor_target.features_extractor) != id(model.policy.critic_target.features_extractor)
# Critic and target should be equal at the begginning of training # Critic and target should be equal at the beginning of training
params_should_match(model.critic.parameters(), model.critic_target.parameters()) params_should_match(model.critic.parameters(), model.critic_target.parameters())
# TD3 has also a target actor net # TD3 has also a target actor net

View file

@ -326,7 +326,7 @@ def test_vec_normalize(model_class):
def test_dict_nested(): def test_dict_nested():
""" """
Make sure we throw an appropiate error with nested Dict observation spaces Make sure we throw an appropriate error with nested Dict observation spaces
""" """
# Test without manual wrapping to vec-env # Test without manual wrapping to vec-env
env = DummyDictEnv(nested_dict_obs=True) env = DummyDictEnv(nested_dict_obs=True)

View file

@ -384,7 +384,7 @@ def test_truncate_last_trajectory(n_envs, recwarn, n_steps, handle_timeout_termi
# for all episodes that are not finished before truncate_last_trajectory: timeouts should be 1 # for all episodes that are not finished before truncate_last_trajectory: timeouts should be 1
if handle_timeout_termination: if handle_timeout_termination:
assert (replay_buffer.timeouts[pos - 1, env_idx_not_finished] == 1).all() assert (replay_buffer.timeouts[pos - 1, env_idx_not_finished] == 1).all()
# episode length sould be != 0 -> episode can be sampled # episode length should be != 0 -> episode can be sampled
assert (replay_buffer.ep_length[pos - 1] != 0).all() assert (replay_buffer.ep_length[pos - 1] != 0).all()
# replay buffer should not have changed after truncate_last_trajectory (except dones[pos-1]) # replay buffer should not have changed after truncate_last_trajectory (except dones[pos-1])

View file

@ -479,7 +479,7 @@ def test_human_output_format_custom_test_io(base_class):
class DummySuccessEnv(gym.Env): class DummySuccessEnv(gym.Env):
""" """
Create a dummy success environment that returns wether True or False for info['is_success'] Create a dummy success environment that returns whether True or False for info['is_success']
at the end of an episode according to its dummy successes list at the end of an episode according to its dummy successes list
""" """
@ -536,7 +536,7 @@ def test_rollout_success_rate_on_policy_algorithm(tmp_path):
Test if the rollout/success_rate information is correctly logged with on policy algorithms Test if the rollout/success_rate information is correctly logged with on policy algorithms
To do so, create a dummy environment that takes as argument dummy successes (i.e when an episode) To do so, create a dummy environment that takes as argument dummy successes (i.e when an episode)
is going to be successfull or not. is going to be successful or not.
""" """
STATS_WINDOW_SIZE = 10 STATS_WINDOW_SIZE = 10

View file

@ -484,7 +484,7 @@ def test_non_dict_obs_keys():
with pytest.raises(ValueError, match=".*is applicable only.*"): with pytest.raises(ValueError, match=".*is applicable only.*"):
_make_warmstart(lambda: DummyRewardEnv(), norm_obs_keys=["key"]) _make_warmstart(lambda: DummyRewardEnv(), norm_obs_keys=["key"])
with pytest.raises(ValueError, match=".* explicitely pass the observation keys.*"): with pytest.raises(ValueError, match=".* explicitly pass the observation keys.*"):
_make_warmstart(lambda: DummyMixedDictEnv()) _make_warmstart(lambda: DummyMixedDictEnv())
# Ignore Discrete observation key # Ignore Discrete observation key