[D] Stable baselines integration #384
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jannikgro
changed the title
* introduced self.network in actorcritic_agent * added network_architecture in QLearningAgent * changed actorcritic_agent to network_architecture * set back training_scenario * am_configuration initialize rl-agent via marketplace
NikkelM
requested changes
Apr 6, 2022
There was a problem hiding this comment.
You wrote that you want to do a refactoring to remove some code duplication. To keep in line with our motto of keeping PR's small, I would prefer if you did so in a separate PR, perhaps right after this one is merged.
In one of my comments I mentioned a bug regarding the _on_training_end, I am currently user-testing if this also occurs for our other trainings or is limited to the StableBaselines-Training.
Update: Yes, the bug I mentioned also occurred when I trained a QLearningAgent
Additional edit: It seems like this was due to an unset datapath, so we should perhaps include a check that the datapath is set before starting the training
…2021 into stable_baselines_integration
blackjack2693
approved these changes
Apr 7, 2022
Co-authored-by: Johann Schulze Tast <[email protected]>
…2021 into stable_baselines_integration
NikkelM
approved these changes
Apr 8, 2022
felix-20
added a commit
that referenced
this pull request
Apr 11, 2022
commit 091eee6 Author: Nikkel Mollenhauer <[email protected]> Date: Mon Apr 11 10:43:00 2022 +0200 Tests for `config_validation.py` (#404) * Don't get default data (shouldn't be necessary) * Unpack default data * Refactored utils functions to return dicts instead of strings * Adapted to new mock format * Adapted to new mock format * Some first tests * More tests * Fixed testcase-names * Moved file-endings to initial function call * Fixed tests * More asserts * More tests * More tests * More tests * `validate_sub_keys`-tests commit f8bc162 Author: jannikgro <[email protected]> Date: Fri Apr 8 13:20:42 2022 +0200 [D] Stable baselines integration (#384) * refactored reinforcement learning agent to accept marketplace * adapted test_exampleprinter.py to marketplace initialization * add market option to accept continuos actions * fixed action space check * initial stable baselines integration * Agent init by env (#390) * introduced self.network in actorcritic_agent * added network_architecture in QLearningAgent * changed actorcritic_agent to network_architecture * set back training_scenario * am_configuration initialize rl-agent via marketplace * added final analyse to stable baselines training * added more stable baselines algorithms * added ppo algorithm * introduced stable_baselines_folder * renamed training to callback * satisfied linter * fixed loading problem * try to make tqdm run in stable_baselines * make tqdm running * reduced pmonitoring episodes in sb training * save model only if significantly better * fixed too long test time bug * moved back to 250 episodes testing * set timeout to 15 minutes * added first batch of fixes to @NikkelM feedback * added type annotations and asserts in stable_baselines_model * added sbtraining to training_scenario * applied comments in am_configuration * solved .dat problem and fixed crashing asserts * reintroduced _end_of_training * removed deprecated if * Moved '.dat' to function call instead of appending within function * Fixed assert * fixed model file ending bug * Add short explanation docstring Co-authored-by: Johann Schulze Tast <[email protected]> * fixed wrong docstring * Fixed tests Co-authored-by: NikkelM <[email protected]> Co-authored-by: Johann Schulze Tast <[email protected]>
felix-20
added a commit
that referenced
this pull request
Apr 12, 2022
* ability to add `DockerInfo` for multiple container * support for starting multiple container on docker side * multiple experiments are supported on the webserver * fix #380 * name from `names` for container * merge development commit 091eee6 Author: Nikkel Mollenhauer <[email protected]> Date: Mon Apr 11 10:43:00 2022 +0200 Tests for `config_validation.py` (#404) * Don't get default data (shouldn't be necessary) * Unpack default data * Refactored utils functions to return dicts instead of strings * Adapted to new mock format * Adapted to new mock format * Some first tests * More tests * Fixed testcase-names * Moved file-endings to initial function call * Fixed tests * More asserts * More tests * More tests * More tests * `validate_sub_keys`-tests commit f8bc162 Author: jannikgro <[email protected]> Date: Fri Apr 8 13:20:42 2022 +0200 [D] Stable baselines integration (#384) * refactored reinforcement learning agent to accept marketplace * adapted test_exampleprinter.py to marketplace initialization * add market option to accept continuos actions * fixed action space check * initial stable baselines integration * Agent init by env (#390) * introduced self.network in actorcritic_agent * added network_architecture in QLearningAgent * changed actorcritic_agent to network_architecture * set back training_scenario * am_configuration initialize rl-agent via marketplace * added final analyse to stable baselines training * added more stable baselines algorithms * added ppo algorithm * introduced stable_baselines_folder * renamed training to callback * satisfied linter * fixed loading problem * try to make tqdm run in stable_baselines * make tqdm running * reduced pmonitoring episodes in sb training * save model only if significantly better * fixed too long test time bug * moved back to 250 episodes testing * set timeout to 15 minutes * added first batch of fixes to @NikkelM feedback * added type annotations and asserts in stable_baselines_model * added sbtraining to training_scenario * applied comments in am_configuration * solved .dat problem and fixed crashing asserts * reintroduced _end_of_training * removed deprecated if * Moved '.dat' to function call instead of appending within function * Fixed assert * fixed model file ending bug * Add short explanation docstring Co-authored-by: Johann Schulze Tast <[email protected]> * fixed wrong docstring * Fixed tests Co-authored-by: NikkelM <[email protected]> Co-authored-by: Johann Schulze Tast <[email protected]> * clean up docker port * fix docker test * docstrings * implement #356 * missed `handle_requests` on last commit * Merge branch 'development' into 379-multiple-experiments
felix-20
added a commit
that referenced
this pull request
Apr 20, 2022
commit 8652a67 Author: Judith <[email protected]> Date: Mon Apr 11 18:17:20 2022 +0200 missed `handle_requests` on last commit commit a354f89 Author: Judith <[email protected]> Date: Mon Apr 11 18:16:15 2022 +0200 implement #356 commit fd54365 Author: Judith <[email protected]> Date: Mon Apr 11 17:56:35 2022 +0200 docstrings commit 9f43c9d Author: Judith <[email protected]> Date: Mon Apr 11 15:07:47 2022 +0200 fix docker test commit b3e6e80 Merge: 6f64f9e 091eee6 Author: felix-20 <[email protected]> Date: Mon Apr 11 14:31:39 2022 +0200 Merge branch 'development' into 379-multiple-experiments commit 6f64f9e Author: Judith <[email protected]> Date: Mon Apr 11 14:20:47 2022 +0200 clean up docker port commit 8c35e8a Author: Judith <[email protected]> Date: Mon Apr 11 11:33:30 2022 +0200 merge development commit 091eee6 Author: Nikkel Mollenhauer <[email protected]> Date: Mon Apr 11 10:43:00 2022 +0200 Tests for `config_validation.py` (#404) * Don't get default data (shouldn't be necessary) * Unpack default data * Refactored utils functions to return dicts instead of strings * Adapted to new mock format * Adapted to new mock format * Some first tests * More tests * Fixed testcase-names * Moved file-endings to initial function call * Fixed tests * More asserts * More tests * More tests * More tests * `validate_sub_keys`-tests commit f8bc162 Author: jannikgro <[email protected]> Date: Fri Apr 8 13:20:42 2022 +0200 [D] Stable baselines integration (#384) * refactored reinforcement learning agent to accept marketplace * adapted test_exampleprinter.py to marketplace initialization * add market option to accept continuos actions * fixed action space check * initial stable baselines integration * Agent init by env (#390) * introduced self.network in actorcritic_agent * added network_architecture in QLearningAgent * changed actorcritic_agent to network_architecture * set back training_scenario * am_configuration initialize rl-agent via marketplace * added final analyse to stable baselines training * added more stable baselines algorithms * added ppo algorithm * introduced stable_baselines_folder * renamed training to callback * satisfied linter * fixed loading problem * try to make tqdm run in stable_baselines * make tqdm running * reduced pmonitoring episodes in sb training * save model only if significantly better * fixed too long test time bug * moved back to 250 episodes testing * set timeout to 15 minutes * added first batch of fixes to @NikkelM feedback * added type annotations and asserts in stable_baselines_model * added sbtraining to training_scenario * applied comments in am_configuration * solved .dat problem and fixed crashing asserts * reintroduced _end_of_training * removed deprecated if * Moved '.dat' to function call instead of appending within function * Fixed assert * fixed model file ending bug * Add short explanation docstring Co-authored-by: Johann Schulze Tast <[email protected]> * fixed wrong docstring * Fixed tests Co-authored-by: NikkelM <[email protected]> Co-authored-by: Johann Schulze Tast <[email protected]> commit 0183876 Author: Judith <[email protected]> Date: Mon Apr 11 11:31:15 2022 +0200 name from `names` for container commit ad70b7c Author: Judith <[email protected]> Date: Sun Apr 10 16:32:47 2022 +0200 fix #380 commit 8e1314c Author: Judith <[email protected]> Date: Sun Apr 10 16:05:28 2022 +0200 multiple experiments are supported on the webserver commit 53a4000 Author: Judith <[email protected]> Date: Fri Apr 8 14:36:44 2022 +0200 support for starting multiple container on docker side commit 3c94f43 Author: Judith <[email protected]> Date: Fri Apr 8 11:46:07 2022 +0200 ability to add `DockerInfo` for multiple container
felix-20
added a commit
that referenced
this pull request
Jul 20, 2022
* merge multiple experiments commit 8652a67 Author: Judith <[email protected]> Date: Mon Apr 11 18:17:20 2022 +0200 missed `handle_requests` on last commit commit a354f89 Author: Judith <[email protected]> Date: Mon Apr 11 18:16:15 2022 +0200 implement #356 commit fd54365 Author: Judith <[email protected]> Date: Mon Apr 11 17:56:35 2022 +0200 docstrings commit 9f43c9d Author: Judith <[email protected]> Date: Mon Apr 11 15:07:47 2022 +0200 fix docker test commit b3e6e80 Merge: 6f64f9e 091eee6 Author: felix-20 <[email protected]> Date: Mon Apr 11 14:31:39 2022 +0200 Merge branch 'development' into 379-multiple-experiments commit 6f64f9e Author: Judith <[email protected]> Date: Mon Apr 11 14:20:47 2022 +0200 clean up docker port commit 8c35e8a Author: Judith <[email protected]> Date: Mon Apr 11 11:33:30 2022 +0200 merge development commit 091eee6 Author: Nikkel Mollenhauer <[email protected]> Date: Mon Apr 11 10:43:00 2022 +0200 Tests for `config_validation.py` (#404) * Don't get default data (shouldn't be necessary) * Unpack default data * Refactored utils functions to return dicts instead of strings * Adapted to new mock format * Adapted to new mock format * Some first tests * More tests * Fixed testcase-names * Moved file-endings to initial function call * Fixed tests * More asserts * More tests * More tests * More tests * `validate_sub_keys`-tests commit f8bc162 Author: jannikgro <[email protected]> Date: Fri Apr 8 13:20:42 2022 +0200 [D] Stable baselines integration (#384) * refactored reinforcement learning agent to accept marketplace * adapted test_exampleprinter.py to marketplace initialization * add market option to accept continuos actions * fixed action space check * initial stable baselines integration * Agent init by env (#390) * introduced self.network in actorcritic_agent * added network_architecture in QLearningAgent * changed actorcritic_agent to network_architecture * set back training_scenario * am_configuration initialize rl-agent via marketplace * added final analyse to stable baselines training * added more stable baselines algorithms * added ppo algorithm * introduced stable_baselines_folder * renamed training to callback * satisfied linter * fixed loading problem * try to make tqdm run in stable_baselines * make tqdm running * reduced pmonitoring episodes in sb training * save model only if significantly better * fixed too long test time bug * moved back to 250 episodes testing * set timeout to 15 minutes * added first batch of fixes to @NikkelM feedback * added type annotations and asserts in stable_baselines_model * added sbtraining to training_scenario * applied comments in am_configuration * solved .dat problem and fixed crashing asserts * reintroduced _end_of_training * removed deprecated if * Moved '.dat' to function call instead of appending within function * Fixed assert * fixed model file ending bug * Add short explanation docstring Co-authored-by: Johann Schulze Tast <[email protected]> * fixed wrong docstring * Fixed tests Co-authored-by: NikkelM <[email protected]> Co-authored-by: Johann Schulze Tast <[email protected]> commit 0183876 Author: Judith <[email protected]> Date: Mon Apr 11 11:31:15 2022 +0200 name from `names` for container commit ad70b7c Author: Judith <[email protected]> Date: Sun Apr 10 16:32:47 2022 +0200 fix #380 commit 8e1314c Author: Judith <[email protected]> Date: Sun Apr 10 16:05:28 2022 +0200 multiple experiments are supported on the webserver commit 53a4000 Author: Judith <[email protected]> Date: Fri Apr 8 14:36:44 2022 +0200 support for starting multiple container on docker side commit 3c94f43 Author: Judith <[email protected]> Date: Fri Apr 8 11:46:07 2022 +0200 ability to add `DockerInfo` for multiple container * first attempt for websocket * websocket on docker site working * just send all of it if things have changed * started on webserver site * webserver sends push notification to user about stopped container * commit * websocket to ssl * fix * try * started on database manager * more db * fix? * debug statements * more types * does table exist fix * colorful terminal output * commit * debug * extra health checker * debug for force stop * some logging * fix tests * telegram notifications * better logging * better logging * fix test? * different logging level * small fixes * started on system monitoring * system monitoring * csv possibility for system data * two buttons in webserver * more fail prove * fix configuration form * silent_starter * logging statements for silent starter * debug * debug * more debugging * fix error * more debugging * more local monitoring * adjusted logging in silent starter * gpu * os.system * subprocess * Squashed commit of the following: commit af12e13 Merge: bc9d3a3 ab704ca Author: Nikkel Mollenhauer <[email protected]> Date: Tue Jun 7 18:20:56 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit bc9d3a3 Author: Nikkel Mollenhauer <[email protected]> Date: Tue Jun 7 18:20:50 2022 +0200 Removed dead methods commit ab704ca Author: felix-20 <[email protected]> Date: Tue Jun 7 16:55:34 2022 +0200 improved prefill (#496) * improved prefill with consideration of the current formdata * remove debug statements * remove print debug * Added lxml dependency * Removed debug comment Co-authored-by: Nikkel Mollenhauer <[email protected]> commit 8839f90 Author: Judith <[email protected]> Date: Tue Jun 7 12:01:43 2022 +0200 remove `config_is_final` commit 53d79f9 Author: NikkelM <[email protected]> Date: Tue Jun 7 10:41:51 2022 +0200 Review feedback by @SinNeax commit 99ab681 Author: NikkelM <[email protected]> Date: Sat Jun 4 15:02:34 2022 +0200 Fixed config validation commit 6a09267 Merge: 179e0da e623413 Author: Judith <[email protected]> Date: Fri Jun 3 14:50:15 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit 179e0da Author: Judith <[email protected]> Date: Fri Jun 3 14:49:49 2022 +0200 fix javascript error commit e623413 Author: NikkelM <[email protected]> Date: Fri Jun 3 14:38:20 2022 +0200 Added `config_type` field to default modelfiles commit bd15108 Merge: dac92d3 507cbc1 Author: NikkelM <[email protected]> Date: Fri Jun 3 09:45:11 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit dac92d3 Author: NikkelM <[email protected]> Date: Fri Jun 3 09:44:49 2022 +0200 Fixed tests commit 507cbc1 Author: Judith <[email protected]> Date: Fri Jun 3 07:52:30 2022 +0200 dynamic table for sim_market possible commit bce03bd Author: Judith <[email protected]> Date: Thu Jun 2 20:15:17 2022 +0200 fix webserver tests commit 57dbb33 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 14:29:40 2022 +0200 Webserver format commit b3780cd Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 14:15:01 2022 +0200 Fixed validation commit c9657a9 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 13:50:20 2022 +0200 Added back needed functionality of validating "complete" configs commit dca24f3 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 13:34:21 2022 +0200 Added some tests commit fff9b47 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 13:12:41 2022 +0200 Renamed `market` to `sim_market` commit caf5c01 Merge: b8c43b7 f50fe65 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 11:30:18 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit b8c43b7 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 11:29:34 2022 +0200 New config format commit f50fe65 Author: Judith <[email protected]> Date: Thu Jun 2 11:21:10 2022 +0200 implement new config validation for webserver commit 135e8f3 Author: Judith <[email protected]> Date: Wed Jun 1 20:52:12 2022 +0200 fix webserver tests commit d326245 Author: Judith <[email protected]> Date: Wed Jun 1 17:26:50 2022 +0200 fix prefill commit 70492c8 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 17:08:34 2022 +0200 Fixed remaining tests(?) commit 59136b1 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 16:35:00 2022 +0200 Fixed most tests commit c912309 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 16:11:20 2022 +0200 Removed references to `class`-field commit 2ccee78 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 15:12:56 2022 +0200 Removed `class` keywords from config files commit c51dab7 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 15:09:46 2022 +0200 renamed rl_config to q_learning_config commit f9aa012 Author: NikkelM <[email protected]> Date: Tue May 31 19:49:14 2022 +0200 Fixed Agent_monitoring with stable_baselines agents commit 16df8c4 Merge: 7610c6e 26cb54b Author: Judith <[email protected]> Date: Tue May 31 13:50:24 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit 7610c6e Author: Judith <[email protected]> Date: Tue May 31 13:50:14 2022 +0200 script for creating rl model commit 26cb54b Author: NikkelM <[email protected]> Date: Tue May 31 11:21:46 2022 +0200 Simplified and extended key validation commit 0fa7eb5 Author: Judith <[email protected]> Date: Mon May 30 14:17:22 2022 +0200 rl config works in view commit da05474 Author: NikkelM <[email protected]> Date: Fri May 27 13:37:39 2022 +0200 New feather website commit dd40104 Author: NikkelM <[email protected]> Date: Fri May 27 12:56:28 2022 +0200 Removed debug comment, fixed policyanalyzer commit 1d025c9 Author: NikkelM <[email protected]> Date: Fri May 27 12:39:17 2022 +0200 Reintroduced `main` for testing purposes commit c90a8d6 Author: NikkelM <[email protected]> Date: Fri May 27 12:26:10 2022 +0200 Fixed config validation for webserver commit 4107aaf Author: NikkelM <[email protected]> Date: Fri May 27 10:57:04 2022 +0200 Added some debugging to docker commit 2588f6d Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 22:22:39 2022 +0200 preconditions for actor critic parameters and stable baselines parameter have been created commit 4e174e6 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:42:23 2022 +0200 restored changes commit 14f181d Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:37:23 2022 +0200 restored last changes commit 3d3ac9d Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:34:11 2022 +0200 more information commit 31a212f Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:31:31 2022 +0200 more information commit 1b139e0 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:28:02 2022 +0200 test commit 1e80e8c Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:25:16 2022 +0200 next try commit 09a433e Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:22:45 2022 +0200 another printf try commit de9f5eb Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:19:07 2022 +0200 more printf debug commit 511d38a Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:16:11 2022 +0200 printf debug commit 1d1a3c4 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:12:06 2022 +0200 restored last changes commit 61c0d8e Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:09:22 2022 +0200 tried without reading check commit 068abaa Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:03:54 2022 +0200 tried with dirty fix commit f1cdf45 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:02:03 2022 +0200 debug print commit 75a674e Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:56:08 2022 +0200 restored docker_manager commit 39bc238 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:51:31 2022 +0200 tried with assert False commit 6986513 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:21:59 2022 +0200 small change in docker_manager commit 561fc00 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:13:18 2022 +0200 tried to fix config_validation commit 3fded29 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 17:25:10 2022 +0200 reintroduced test_hyperparameter_config_market commit b6a4945 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 15:52:19 2022 +0200 reintroduced test_hyperparameter_config_rl commit 7cef3e7 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 15:19:29 2022 +0200 added rules and verifications commit 473ddf7 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 12:43:12 2022 +0200 introduced JSONConfigurable class and demanded class entry in config commit 36024c8 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 11:50:48 2022 +0200 solved #484 without check * automatically write template files * try fix pre-commit * try fix pre-commit * try except * more debug * more debug * debugging :) * hotfix for qlearning file * adopting to the given agent works again * restore docker manager and app.py * try restore again * webserver tests are running again merge from dev was very very strange * new rl_config * precommit? * precommit! * assert print Co-Authored-By: Nikkel Mollenhauer <[email protected]> * remove print Co-Authored-By: Nikkel Mollenhauer <[email protected]> * new regex for matching ce agents Co-Authored-By: Nikkel Mollenhauer <[email protected]> * fixed fixed price agent Co-Authored-By: Nikkel Mollenhauer <[email protected]> * comment out assert Co-Authored-By: Nikkel Mollenhauer <[email protected]> Co-Authored-By: jannikgro <[email protected]> * delete some files * bcolors back * readme websocket * debug websocket * fixed websocket * small websocket fix * logging for websocket * more logging? * logging * logging relevant content * reset? * reset webserver * relevant gitignore stuff * some resets in docker manager * init files again? * separate_markets again * remove unnecessary code * fixed docker tests, introduce Mocked logger * logger into DockerManager * Fixed typo bug * log to file when executing `docker_manager` Co-authored-by: Nikkel Mollenhauer <[email protected]> Co-authored-by: jannikgro <[email protected]> Co-authored-by: Johann Schulze Tast <[email protected]>
felix-20
added a commit
that referenced
this pull request
Jul 30, 2022
* merge multiple experiments commit 8652a67 Author: Judith <[email protected]> Date: Mon Apr 11 18:17:20 2022 +0200 missed `handle_requests` on last commit commit a354f89 Author: Judith <[email protected]> Date: Mon Apr 11 18:16:15 2022 +0200 implement #356 commit fd54365 Author: Judith <[email protected]> Date: Mon Apr 11 17:56:35 2022 +0200 docstrings commit 9f43c9d Author: Judith <[email protected]> Date: Mon Apr 11 15:07:47 2022 +0200 fix docker test commit b3e6e80 Merge: 6f64f9e 091eee6 Author: felix-20 <[email protected]> Date: Mon Apr 11 14:31:39 2022 +0200 Merge branch 'development' into 379-multiple-experiments commit 6f64f9e Author: Judith <[email protected]> Date: Mon Apr 11 14:20:47 2022 +0200 clean up docker port commit 8c35e8a Author: Judith <[email protected]> Date: Mon Apr 11 11:33:30 2022 +0200 merge development commit 091eee6 Author: Nikkel Mollenhauer <[email protected]> Date: Mon Apr 11 10:43:00 2022 +0200 Tests for `config_validation.py` (#404) * Don't get default data (shouldn't be necessary) * Unpack default data * Refactored utils functions to return dicts instead of strings * Adapted to new mock format * Adapted to new mock format * Some first tests * More tests * Fixed testcase-names * Moved file-endings to initial function call * Fixed tests * More asserts * More tests * More tests * More tests * `validate_sub_keys`-tests commit f8bc162 Author: jannikgro <[email protected]> Date: Fri Apr 8 13:20:42 2022 +0200 [D] Stable baselines integration (#384) * refactored reinforcement learning agent to accept marketplace * adapted test_exampleprinter.py to marketplace initialization * add market option to accept continuos actions * fixed action space check * initial stable baselines integration * Agent init by env (#390) * introduced self.network in actorcritic_agent * added network_architecture in QLearningAgent * changed actorcritic_agent to network_architecture * set back training_scenario * am_configuration initialize rl-agent via marketplace * added final analyse to stable baselines training * added more stable baselines algorithms * added ppo algorithm * introduced stable_baselines_folder * renamed training to callback * satisfied linter * fixed loading problem * try to make tqdm run in stable_baselines * make tqdm running * reduced pmonitoring episodes in sb training * save model only if significantly better * fixed too long test time bug * moved back to 250 episodes testing * set timeout to 15 minutes * added first batch of fixes to @NikkelM feedback * added type annotations and asserts in stable_baselines_model * added sbtraining to training_scenario * applied comments in am_configuration * solved .dat problem and fixed crashing asserts * reintroduced _end_of_training * removed deprecated if * Moved '.dat' to function call instead of appending within function * Fixed assert * fixed model file ending bug * Add short explanation docstring Co-authored-by: Johann Schulze Tast <[email protected]> * fixed wrong docstring * Fixed tests Co-authored-by: NikkelM <[email protected]> Co-authored-by: Johann Schulze Tast <[email protected]> commit 0183876 Author: Judith <[email protected]> Date: Mon Apr 11 11:31:15 2022 +0200 name from `names` for container commit ad70b7c Author: Judith <[email protected]> Date: Sun Apr 10 16:32:47 2022 +0200 fix #380 commit 8e1314c Author: Judith <[email protected]> Date: Sun Apr 10 16:05:28 2022 +0200 multiple experiments are supported on the webserver commit 53a4000 Author: Judith <[email protected]> Date: Fri Apr 8 14:36:44 2022 +0200 support for starting multiple container on docker side commit 3c94f43 Author: Judith <[email protected]> Date: Fri Apr 8 11:46:07 2022 +0200 ability to add `DockerInfo` for multiple container * first attempt for websocket * websocket on docker site working * just send all of it if things have changed * started on webserver site * webserver sends push notification to user about stopped container * commit * websocket to ssl * fix * try * started on database manager * more db * fix? * debug statements * more types * does table exist fix * colorful terminal output * commit * debug * extra health checker * debug for force stop * some logging * fix tests * telegram notifications * better logging * better logging * fix test? * different logging level * small fixes * started on system monitoring * system monitoring * csv possibility for system data * two buttons in webserver * more fail prove * fix configuration form * silent_starter * logging statements for silent starter * debug * debug * more debugging * fix error * more debugging * more local monitoring * adjusted logging in silent starter * gpu * os.system * subprocess * Squashed commit of the following: commit af12e13 Merge: bc9d3a3 ab704ca Author: Nikkel Mollenhauer <[email protected]> Date: Tue Jun 7 18:20:56 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit bc9d3a3 Author: Nikkel Mollenhauer <[email protected]> Date: Tue Jun 7 18:20:50 2022 +0200 Removed dead methods commit ab704ca Author: felix-20 <[email protected]> Date: Tue Jun 7 16:55:34 2022 +0200 improved prefill (#496) * improved prefill with consideration of the current formdata * remove debug statements * remove print debug * Added lxml dependency * Removed debug comment Co-authored-by: Nikkel Mollenhauer <[email protected]> commit 8839f90 Author: Judith <[email protected]> Date: Tue Jun 7 12:01:43 2022 +0200 remove `config_is_final` commit 53d79f9 Author: NikkelM <[email protected]> Date: Tue Jun 7 10:41:51 2022 +0200 Review feedback by @SinNeax commit 99ab681 Author: NikkelM <[email protected]> Date: Sat Jun 4 15:02:34 2022 +0200 Fixed config validation commit 6a09267 Merge: 179e0da e623413 Author: Judith <[email protected]> Date: Fri Jun 3 14:50:15 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit 179e0da Author: Judith <[email protected]> Date: Fri Jun 3 14:49:49 2022 +0200 fix javascript error commit e623413 Author: NikkelM <[email protected]> Date: Fri Jun 3 14:38:20 2022 +0200 Added `config_type` field to default modelfiles commit bd15108 Merge: dac92d3 507cbc1 Author: NikkelM <[email protected]> Date: Fri Jun 3 09:45:11 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit dac92d3 Author: NikkelM <[email protected]> Date: Fri Jun 3 09:44:49 2022 +0200 Fixed tests commit 507cbc1 Author: Judith <[email protected]> Date: Fri Jun 3 07:52:30 2022 +0200 dynamic table for sim_market possible commit bce03bd Author: Judith <[email protected]> Date: Thu Jun 2 20:15:17 2022 +0200 fix webserver tests commit 57dbb33 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 14:29:40 2022 +0200 Webserver format commit b3780cd Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 14:15:01 2022 +0200 Fixed validation commit c9657a9 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 13:50:20 2022 +0200 Added back needed functionality of validating "complete" configs commit dca24f3 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 13:34:21 2022 +0200 Added some tests commit fff9b47 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 13:12:41 2022 +0200 Renamed `market` to `sim_market` commit caf5c01 Merge: b8c43b7 f50fe65 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 11:30:18 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit b8c43b7 Author: Nikkel Mollenhauer <[email protected]> Date: Thu Jun 2 11:29:34 2022 +0200 New config format commit f50fe65 Author: Judith <[email protected]> Date: Thu Jun 2 11:21:10 2022 +0200 implement new config validation for webserver commit 135e8f3 Author: Judith <[email protected]> Date: Wed Jun 1 20:52:12 2022 +0200 fix webserver tests commit d326245 Author: Judith <[email protected]> Date: Wed Jun 1 17:26:50 2022 +0200 fix prefill commit 70492c8 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 17:08:34 2022 +0200 Fixed remaining tests(?) commit 59136b1 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 16:35:00 2022 +0200 Fixed most tests commit c912309 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 16:11:20 2022 +0200 Removed references to `class`-field commit 2ccee78 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 15:12:56 2022 +0200 Removed `class` keywords from config files commit c51dab7 Author: Nikkel Mollenhauer <[email protected]> Date: Wed Jun 1 15:09:46 2022 +0200 renamed rl_config to q_learning_config commit f9aa012 Author: NikkelM <[email protected]> Date: Tue May 31 19:49:14 2022 +0200 Fixed Agent_monitoring with stable_baselines agents commit 16df8c4 Merge: 7610c6e 26cb54b Author: Judith <[email protected]> Date: Tue May 31 13:50:24 2022 +0200 Merge branch '484-configuration-remove-combined-config' of https://github.com/hpi-epic/BP2021 into 484-configuration-remove-combined-config commit 7610c6e Author: Judith <[email protected]> Date: Tue May 31 13:50:14 2022 +0200 script for creating rl model commit 26cb54b Author: NikkelM <[email protected]> Date: Tue May 31 11:21:46 2022 +0200 Simplified and extended key validation commit 0fa7eb5 Author: Judith <[email protected]> Date: Mon May 30 14:17:22 2022 +0200 rl config works in view commit da05474 Author: NikkelM <[email protected]> Date: Fri May 27 13:37:39 2022 +0200 New feather website commit dd40104 Author: NikkelM <[email protected]> Date: Fri May 27 12:56:28 2022 +0200 Removed debug comment, fixed policyanalyzer commit 1d025c9 Author: NikkelM <[email protected]> Date: Fri May 27 12:39:17 2022 +0200 Reintroduced `main` for testing purposes commit c90a8d6 Author: NikkelM <[email protected]> Date: Fri May 27 12:26:10 2022 +0200 Fixed config validation for webserver commit 4107aaf Author: NikkelM <[email protected]> Date: Fri May 27 10:57:04 2022 +0200 Added some debugging to docker commit 2588f6d Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 22:22:39 2022 +0200 preconditions for actor critic parameters and stable baselines parameter have been created commit 4e174e6 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:42:23 2022 +0200 restored changes commit 14f181d Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:37:23 2022 +0200 restored last changes commit 3d3ac9d Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:34:11 2022 +0200 more information commit 31a212f Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:31:31 2022 +0200 more information commit 1b139e0 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:28:02 2022 +0200 test commit 1e80e8c Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:25:16 2022 +0200 next try commit 09a433e Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:22:45 2022 +0200 another printf try commit de9f5eb Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:19:07 2022 +0200 more printf debug commit 511d38a Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:16:11 2022 +0200 printf debug commit 1d1a3c4 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:12:06 2022 +0200 restored last changes commit 61c0d8e Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:09:22 2022 +0200 tried without reading check commit 068abaa Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:03:54 2022 +0200 tried with dirty fix commit f1cdf45 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 19:02:03 2022 +0200 debug print commit 75a674e Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:56:08 2022 +0200 restored docker_manager commit 39bc238 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:51:31 2022 +0200 tried with assert False commit 6986513 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:21:59 2022 +0200 small change in docker_manager commit 561fc00 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 18:13:18 2022 +0200 tried to fix config_validation commit 3fded29 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 17:25:10 2022 +0200 reintroduced test_hyperparameter_config_market commit b6a4945 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 15:52:19 2022 +0200 reintroduced test_hyperparameter_config_rl commit 7cef3e7 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 15:19:29 2022 +0200 added rules and verifications commit 473ddf7 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 12:43:12 2022 +0200 introduced JSONConfigurable class and demanded class entry in config commit 36024c8 Author: Jan Niklas Groeneveld <[email protected]> Date: Thu May 26 11:50:48 2022 +0200 solved #484 without check * automatically write template files * try fix pre-commit * try fix pre-commit * try except * more debug * more debug * debugging :) * hotfix for qlearning file * adopting to the given agent works again * restore docker manager and app.py * try restore again * webserver tests are running again merge from dev was very very strange * new rl_config * precommit? * precommit! * assert print Co-Authored-By: Nikkel Mollenhauer <[email protected]> * remove print Co-Authored-By: Nikkel Mollenhauer <[email protected]> * new regex for matching ce agents Co-Authored-By: Nikkel Mollenhauer <[email protected]> * fixed fixed price agent Co-Authored-By: Nikkel Mollenhauer <[email protected]> * comment out assert Co-Authored-By: Nikkel Mollenhauer <[email protected]> Co-Authored-By: jannikgro <[email protected]> * delete some files * bcolors back * readme websocket * debug websocket * fixed websocket * small websocket fix * logging for websocket * more logging? * logging * try make container_heath_checker only run when there are actually container running * right keyword * cls * debug printing * try popen * debug print * make it work again? * should run monitoring for docker manager * make webserver work again * docstrings * fix tests * remove files add docstrings * remove more files Co-authored-by: Nikkel Mollenhauer <[email protected]> Co-authored-by: jannikgro <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to integrate stable baselines (closes #316). All needed functionality is delivered, but there is an ugly code duplication between stable_baselines_callback and training. It is an option (and my preferred one) to refactor this code in a way that the callback is called in our agents too. But this solution needs some bit efford and could be written as a new issue.