stable-baselines3/docs/index.rst

106 lines
2.4 KiB
ReStructuredText
Raw Normal View History

2020-05-07 08:10:51 +00:00
.. Stable Baselines3 documentation master file, created by
2019-09-26 09:46:40 +00:00
sphinx-quickstart on Thu Sep 26 11:06:54 2019.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
2020-05-08 11:09:38 +00:00
Welcome to Stable Baselines3 docs! - RL Baselines Made Easy
===========================================================
2019-09-26 09:46:40 +00:00
2020-05-07 08:10:51 +00:00
`Stable Baselines3 <https://github.com/DLR-RM/stable-baselines3>`_ is a set of improved implementations of reinforcement learning algorithms in PyTorch.
2020-05-08 11:09:38 +00:00
It is the next major version of `Stable Baselines <https://github.com/hill-a/stable-baselines>`_.
2020-05-07 08:10:51 +00:00
Github repository: https://github.com/DLR-RM/stable-baselines3
2019-09-26 09:46:40 +00:00
2020-05-05 15:41:57 +00:00
RL Baselines3 Zoo (collection of pre-trained agents): https://github.com/DLR-RM/rl-baselines3-zoo
2019-09-26 09:46:40 +00:00
2020-05-05 15:41:57 +00:00
RL Baselines3 Zoo also offers a simple interface to train, evaluate agents and do hyperparameter tuning.
2019-09-26 09:46:40 +00:00
2020-05-07 08:10:51 +00:00
Main Features
--------------
- Unified structure for all algorithms
- PEP8 compliant (unified code style)
- Documented functions and classes
- Tests, high code coverage and type hints
- Clean code
- Tensorboard support
2020-05-07 08:10:51 +00:00
2019-09-26 09:46:40 +00:00
.. toctree::
:maxdepth: 2
:caption: User Guide
2020-05-07 08:10:51 +00:00
guide/install
2019-09-26 09:46:40 +00:00
guide/quickstart
2020-05-07 08:10:51 +00:00
guide/rl_tips
guide/rl
2020-05-07 14:08:23 +00:00
guide/algos
guide/examples
2019-09-26 09:46:40 +00:00
guide/vec_envs
2020-05-07 08:10:51 +00:00
guide/custom_env
2020-05-07 14:08:23 +00:00
guide/custom_policy
2020-05-07 08:10:51 +00:00
guide/callbacks
guide/tensorboard
2020-05-08 09:58:43 +00:00
guide/rl_zoo
2020-05-07 08:10:51 +00:00
guide/migration
guide/checking_nan
2020-05-08 14:20:21 +00:00
guide/developer
2019-09-26 09:46:40 +00:00
.. toctree::
:maxdepth: 1
:caption: RL Algorithms
modules/base
2020-01-20 15:19:35 +00:00
modules/a2c
2019-09-26 09:46:40 +00:00
modules/ppo
modules/sac
modules/td3
Implement DQN (#28) * Created DQN template according to the paper. Next steps: - Create Policy - Complete Training - Debug * Changed Base Class * refactor save, to be consistence with overriding the excluded_save_params function. Do not try to exclude the parameters twice. * Added simple DQN policy * Finished learn and train function - missing correct loss computation * changed collect_rollouts to work with discrete space * moved discrete space collect_rollouts to dqn * basic dqn working * deleted SDE related code * added gradient clipping and moved greedy policy to policy * changed policy to implement target network and added soft update(in fact standart tau is 1 so hard update) * fixed policy setup * rebase target_update_intervall on _n_updates * adapted all tests all tests passing * Move to stable-baseline3 * Fixes for DQN * Fix tests + add CNNPolicy * Allow any optimizer for DQN * added some util functions to create a arbitrary linear schedule, fixed pickle problem with old exploration schedule * more documentation * changed buffer dtype * refactor and document * Added Sphinx Documentation Updated changelog.rst * removed custom collect_rollouts as it is no longer necessary * Implemented suggestions to clean code and documentation. * extracted some functions on tests to reduce duplicated code * added support for exploration_fraction * Fixed exploration_fraction * Added documentation * Fixed get_linear_fn -> proper progress scaling * Merged master * Added nature reference * Changed default parameters to https://www.nature.com/articles/nature14236/tables/1 * Fixed n_updates to be incremented correctly * Correct train_freq * Doc update * added special parameter for DQN in tests * different fix for test_discrete * Update docs/modules/dqn.rst Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org> * Update docs/modules/dqn.rst Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org> * Update docs/modules/dqn.rst Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org> * Added RMSProp in optimizer_kwargs, as described in nature paper * Exploration fraction is inverse of 50.000.000 (total frames) / 1.000.000 (frames with linear schedule) according to nature paper * Changelog update for buffer dtype * standard exlude parameters should be always excluded to assure proper saving only if intentionally included by ``include`` parameter * slightly more iterations on test_discrete to pass the test * added param use_rms_prop instead of mutable default argument * forgot alpha * using huber loss, adam and learning rate 1e-4 * account for train_freq in update_target_network * Added memory check for both buffers * Doc updated for buffer allocation * Added psutil Requirement * Adapted test_identity.py * Fixes with new SB3 version * Fix for tensorboard name * Convert assert to warning and fix tests * Refactor off-policy algorithms * Fixes * test: remove next_obs in replay buffer * Update changelog * Fix tests and use tmp_path where possible * Fix sampling bug in buffer * Do not store next obs on episode termination * Fix replay buffer sampling * Update comment * moved epsilon from policy to model * Update predict method * Update atari wrappers to match SB2 * Minor edit in the buffers * Update changelog * Merge branch 'master' into dqn * Update DQN to new structure * Fix tests and remove hardcoded path * Fix for DQN * Disable memory efficient replay buffer by default * Fix docstring * Add tests for memory efficient buffer * Update changelog * Split collect rollout * Move target update outside `train()` for DQN * Update changelog * Update linear schedule doc * Cleanup DQN code * Minor edit * Update version and docker images Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
2020-06-29 09:16:54 +00:00
modules/dqn
2019-09-26 09:46:40 +00:00
2020-05-07 08:10:51 +00:00
.. toctree::
:maxdepth: 1
:caption: Common
2020-05-08 11:09:38 +00:00
common/atari_wrappers
common/cmd_util
2020-05-07 08:10:51 +00:00
common/distributions
common/evaluation
common/env_checker
2020-05-08 11:09:38 +00:00
common/monitor
common/logger
common/noise
common/utils
2019-09-26 09:46:40 +00:00
.. toctree::
:maxdepth: 1
:caption: Misc
misc/changelog
2020-05-07 08:10:51 +00:00
misc/projects
2019-09-26 09:46:40 +00:00
2020-05-05 13:02:35 +00:00
Citing Stable Baselines3
2020-05-05 14:28:38 +00:00
------------------------
2019-09-26 09:46:40 +00:00
To cite this project in publications:
.. code-block:: bibtex
2020-05-05 14:28:38 +00:00
@misc{stable-baselines3,
2020-02-03 14:57:37 +00:00
author = {Raffin, Antonin and Hill, Ashley and Ernestus, Maximilian and Gleave, Adam and Kanervisto, Anssi and Dormann, Noah},
2020-05-05 13:02:35 +00:00
title = {Stable Baselines3},
2019-09-26 09:46:40 +00:00
year = {2019},
publisher = {GitHub},
journal = {GitHub repository},
2020-05-05 14:28:38 +00:00
howpublished = {\url{https://github.com/DLR-RM/stable-baselines3}},
2019-09-26 09:46:40 +00:00
}
Indices and tables
-------------------
* :ref:`genindex`
* :ref:`search`
* :ref:`modindex`