saymrwulf/stable-baselines3
1
0
Fork 0
mirror of https://github.com/saymrwulf/stable-baselines3.git synced 2026-09-04 20:23:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
319 commits 18 branches 32 tags 5.2 MiB
Commit graph

191 commits

Author SHA1 Message Date
Dormann, Noah
1f0dd60b97 Fix saving on GPU - Loading on CPU (#45) 
* removed policy from save, changed th.loads to map to device

* found hack: catch pickle exception and trying th.load with mapping instead, otherwise raise exception with more information -> loading cuda on cpu raises exception -> leads to th.load with map being called

* deleted todo

* updated changelog

* start of saving refactor

* first working c

* all tests pass, save refactored

* - backwards compatibilty not always
- make pytest all passing
- make typing all passing

* Fixes and simplify the save method

* Remove unused param

* Fix backward compat

* Fix docstring
 2020-01-31 13:06:55 +01:00
Antonin Raffin
6ae8421615 Update docstring 2020-01-28 10:28:44 +01:00
Antonin Raffin
5c94a225ef Minor edit to the doc 2020-01-28 10:24:02 +01:00
Antonin Raffin
a628354721 Refactor evaluation 2020-01-27 15:53:27 +01:00
Antonin Raffin
b66003cfb3 Add callback support 2020-01-27 14:32:31 +01:00
Antonin Raffin
fb57a6b80c Update docstring 2020-01-23 11:20:12 +01:00
Antonin Raffin
7265d9e352 Fix multiline f-string 2020-01-23 10:56:53 +01:00
Antonin Raffin
ff0eddfb17 Partially type base class 2020-01-22 17:51:27 +01:00
Antonin Raffin
44fce7c02a Fix typing errors and typos 2020-01-22 17:17:12 +01:00
Antonin Raffin
88f07bafb6 Convert format to f-strings 2020-01-22 16:39:25 +01:00
Antonin Raffin
37f9f13684 Revert all changes for python 2 
+ Add makefile and pytype
 2020-01-22 16:18:27 +01:00
Antonin Raffin
9e250b6818 Build doc 2020-01-20 16:19:35 +01:00
Antonin Raffin
0bed698ec5 Raise error for abstract methods 2020-01-20 12:57:40 +01:00
Antonin Raffin
e5c6601726 Update VecNormalize (pickling) and improve tests 2020-01-20 11:58:16 +01:00
Antonin Raffin
89db65b1fb Improve logger testing + add readers 2020-01-20 11:58:00 +01:00
Antonin Raffin
c542009641 Clean up code + bug fixes 2020-01-20 11:17:55 +01:00
Antonin Raffin
ea20721632 Add TODO 2020-01-15 15:58:45 +01:00
Antonin Raffin
d3a718b94e Add extra dependency 2020-01-08 11:26:57 +01:00
Antonin Raffin
299ca007b5 Add comment about warmup phase 2020-01-07 17:36:26 +01:00
Antonin Raffin
8831eff163 Unify evaluation 2020-01-07 14:00:03 +01:00
Antonin RAFFIN
aa7b91333e Add seeding for subproc vecenv 2019-12-30 12:01:37 +01:00
Antonin RAFFIN
4a79f7e5a7 Print std reward for evaluation 2019-12-24 13:12:04 +01:00
Antonin Raffin
7f34108ed6 Fix exp_ln computation 2019-12-20 18:02:01 +01:00
Antonin Raffin
57708a628c Add value function for SDE + TD3 2019-12-17 15:01:08 +01:00
Antonin Raffin
1d6f9bf100 Add sample freq for SDE 2019-12-17 11:47:21 +01:00
Antonin Raffin
233f346d53 Update todos 2019-12-06 17:46:56 +01:00
Antonin Raffin
0117cc37f4 Merge branch 'master' into feat/sde-features 2019-12-05 16:33:41 +01:00
Noah Dormann
aa67147796 clarified bytesIO use for load 2019-12-05 15:45:05 +01:00
Raffin, Antonin
bac9d4efed Update torchy_baselines/common/base_class.py 2019-12-05 14:53:14 +01:00
Raffin, Antonin
695cdc63a4 Update torchy_baselines/common/base_class.py 2019-12-05 14:52:59 +01:00
Raffin, Antonin
424a554567 Update docstring 2019-12-05 14:50:11 +01:00
Raffin, Antonin
464dd773e6 Update comment 2019-12-05 14:46:02 +01:00
Raffin, Antonin
03ecb17ef6 Update error message 2019-12-05 14:41:39 +01:00
Noah Dormann
88d4f44d55 added set_env test and set_env wrapping 2019-12-05 13:59:07 +01:00
Noah Dormann
cf1d7118a5 replaced file with file_path 2019-12-05 13:44:02 +01:00
Noah Dormann
8062ed6036 fixed load, to check if environment ist correctly 2019-12-05 13:36:19 +01:00
Noah Dormann
4b1bab7f85 implemented set_env method 2019-12-05 09:11:30 +01:00
Noah Dormann
8460bfe397 added some comments to _load_from_file 2019-12-05 08:56:04 +01:00
Noah Dormann
6560ae9952 using other_file instead of other_files 2019-12-05 08:52:26 +01:00
Noah Dormann
7c8d375bcb added get_parameter_list function 2019-12-05 08:50:11 +01:00
Noah Dormann
c3b0398d56 Changed load so it still works when env not saved 
improved save function
 2019-12-05 08:40:28 +01:00
Dormann, Noah
bea2799691 comment refactoring 
Co-Authored-By: Raffin, Antonin <Antonin.Raffin@dlr.de>
 2019-12-05 08:10:39 +01:00
Dormann, Noah
a756f40223 comment refactoring 
Co-Authored-By: Raffin, Antonin <Antonin.Raffin@dlr.de>
 2019-12-05 08:10:25 +01:00
Dormann, Noah
fdb544e775 comment refactoring 
Co-Authored-By: Raffin, Antonin <Antonin.Raffin@dlr.de>
 2019-12-05 08:10:14 +01:00
Dormann, Noah
aa66d2f82e comment refactoring 
Co-Authored-By: Raffin, Antonin <Antonin.Raffin@dlr.de>
 2019-12-05 08:09:51 +01:00
Dormann, Noah
85d3743224 added standart exclude parameters 
Co-Authored-By: Raffin, Antonin <Antonin.Raffin@dlr.de>
 2019-12-05 08:09:26 +01:00
Antonin Raffin
1ac1a7cad5 Reformat 2019-12-02 14:27:38 +01:00
Antonin Raffin
4e39a0627c Refactor: enable sde net arch for TD3 and SAC 2019-12-02 14:06:17 +01:00
Antonin Raffin
7a6a500398 Allow to use states directly as features for sde 2019-12-02 11:48:16 +01:00
Antonin RAFFIN
879191b26a Bug fix in SAC with constant ent coeff + try batch sde matrices 2019-12-01 13:11:13 +01:00
Noah Dormann
6928879f5a Refactored doc-strings 2019-11-28 16:30:13 +01:00
Noah Dormann
7ce610fade Deleted exact match parameter of load_parameters 2019-11-28 16:11:35 +01:00
Noah Dormann
ee6f938ddc Added option to explicitly specify excluded parameters 2019-11-28 15:42:53 +01:00
Noah Dormann
e95858784a Formatted all files 2019-11-28 15:38:04 +01:00
Noah Dormann
751ccf85e7 _setup_model() is now called when model is loaded 2019-11-28 15:33:52 +01:00
Noah Dormann
9ff59eaf3d Added attribute self.policy_class to prevent errors when using self.policy as class 2019-11-28 15:25:01 +01:00
Noah Dormann
e26564e0ec Added function for setting up any attributes that weren't saved and thus not loaded 2019-11-28 13:35:16 +01:00
Noah Dormann
c75582dfbe resolving conflicts 
# Conflicts:
#	torchy_baselines/a2c/a2c.py
#	torchy_baselines/ppo/ppo.py

Added optimizer params test
 2019-11-28 12:12:06 +01:00
Noah Dormann
812cab84ac Changed PPO deterministic 2019-11-28 11:20:40 +01:00
Noah Dormann
b75ffe166a Cleared base_class.load description 2019-11-28 10:36:21 +01:00
Antonin Raffin
fbe29a7298 Track down autograd error 
"Trying to backward through the graph a second time" -> added a comment
 2019-11-27 17:29:47 +01:00
Antonin Raffin
5483e02d1a Add SDE support for SAC 2019-11-26 15:26:12 +01:00
Antonin Raffin
5d6649d92b Enable separate feature extraction for SDE 2019-11-25 14:54:13 +01:00
Antonin Raffin
d0003ee4ec Enable kwargs for proba dist 2019-11-25 14:00:21 +01:00
Antonin Raffin
5bbb14188d Unify A2C and TD3 SDE implementation 2019-11-25 13:19:33 +01:00
Antonin Raffin
c47be0086e Add docstrings 2019-11-22 17:24:47 +01:00
Antonin Raffin
03d2ab10f8 Fix clipped action when adapting noise with TD3 2019-11-22 15:04:34 +01:00
Antonin Raffin
604a19fbc3 Cleanup + update doc 2019-11-22 13:33:12 +01:00
Antonin Raffin
b84e5e9e27 Move flexible mlp to common 2019-11-22 13:06:41 +01:00
Antonin Raffin
99ea0b3a54 Cleanup 2019-11-22 11:42:58 +01:00
Noah Dormann
4f8f936451 Don't save replay_buffer by default 2019-11-21 17:27:46 +01:00
Noah Dormann
2d72f6d1b5 Added SAC, TD3, A2C 
Missing CEMRL
 2019-11-21 16:46:53 +01:00
Noah Dormann
775a50cc5c saving all variables now added a2c support 2019-11-21 16:24:18 +01:00
Noah Dormann
526c37bf1f refactored the assets in test_save_load 
fixed base_class 'params.pth'
 2019-11-21 15:44:57 +01:00
Noah Dormann
26f31fd25b corrected comment sections 2019-11-21 14:55:56 +01:00
Noah Dormann
fb5f192fc4 Implemented Changes suggested from Antonin-Raffin 
Added Optimizer saving
 2019-11-21 14:39:44 +01:00
Noah Dormann
a7655ca6e1 Reformated every file with PEP 8 errors 2019-11-21 13:01:03 +01:00
Noah Dormann
4b6234a1c8 finished test_save_load.py test 2019-11-21 11:39:47 +01:00
Antonin Raffin
d8a7556d84 Merge branch 'feat/sde' into feat/offpolicy-sde 2019-11-18 15:14:05 +01:00
Antonin Raffin
ef59a7e431 Update version + add docstring 2019-11-18 15:11:19 +01:00
Antonin Raffin
b9c20d443d Update doc + add test for tanh bijector 2019-11-18 15:04:07 +01:00
Antonin Raffin
5d353d598c Start cleanup + update docstrings 2019-11-18 14:09:31 +01:00
Antonin Raffin
cdb62a93fe Bug fix for off-policy normalization 
Now working properly
 2019-11-15 11:00:31 +01:00
Antonin Raffin
5278a6f3f8 Testing off policy normalization 2019-11-14 14:35:00 +01:00
Antonin Raffin
a08382faab Add sde update for TD3 2019-11-12 18:37:13 +01:00
Noah Dormann
6cf80ccfe2 reordered imports 2019-11-12 17:12:10 +01:00
Noah Dormann
cc744a48b5 first save and load features 2019-11-12 17:03:57 +01:00
Antonin Raffin
715865a0fe Add noise clipping 2019-11-08 13:17:38 +01:00
Antonin Raffin
f4546837c3 Add std to logger 2019-11-07 17:41:28 +01:00
Antonin Raffin
db87e0d36a Quick and dirty SDE version for TD3 2019-11-07 17:31:52 +01:00
Antonin Raffin
c6f90b9c3c Improve VecNormalize syncing for evaluation 2019-11-07 11:17:26 +01:00
Antonin Raffin
72a6f18e43 Add sde test + fix random seed 2019-10-31 14:14:30 +01:00
Antonin Raffin
925afe784c SDE on latent_pi 2019-10-31 11:44:27 +01:00
Antonin Raffin
862ae666b5 Try squashing the sde 2019-10-30 15:30:09 +01:00
Antonin Raffin
0174ec269e Clean up 2019-10-29 18:43:16 +01:00
Antonin Raffin
9e8f6e0020 Add default filename for monitor 2019-10-29 18:42:34 +01:00
Antonin Raffin
42d50ed09b Add expln 2019-10-29 15:15:54 +01:00
Antonin Raffin
69a348276e Add classic advantage computation 2019-10-29 12:36:40 +01:00
Antonin Raffin
c15b4bda1e Add first draft of SDE 2019-10-28 18:24:13 +01:00
Antonin Raffin
df1e7aa000 Add docstring 2019-10-28 17:42:39 +01:00