[RLlib] - Add "burn-in" period to the training of stateful `RLModules`.#49680

Open

simonsays1980 wants to merge 11 commits intoray-project:masterfrom simonsays1980:offpolicy-stateful-modules-add-burnin

+77-10

Commits on Jan 7, 2025

Overloaded 'InfiniteLookbackBuffer.extend' to work also with dictionaries which we need for states.
simonsays1980
committed
Added a burn-in period to the training of stateful modules. The 'AddStatesFromEpisodesToBatch' connector adjusts the sequence length to the burn-in period and the 'EpsiodeReplayBuffer' allows for a…
simonsays1980
committed
Merge branch 'master' into offpolicy-stateful-modules-add-burnin
simonsays1980
committed
Changed 'burnin' parameter to 'burn_in_len' and added a validation in moved i to 'DQNConfig' from 'AlgorithmConfig'. Added a validation for 'max_seq_len' to be larger than 'burn_in_len' + 1 to incl…
simonsays1980
committed
Merge branch 'master' into offpolicy-stateful-modules-add-burnin
simonsays1980
committed
Apply suggestions from code review
sven1977
authored
Renamed 'burn_in_len' in 'EpisodeReplayBuffer' to 'min_batch_length_T' and adjusted docstring. Added safeguard for algorithms like SAC that do not have a 'burn_in_len', yet.
simonsays1980
committed

Commits on Jan 8, 2025

Commits on Jan 10, 2025

Merge branch 'master' into offpolicy-stateful-modules-add-burnin
simonsays1980
committed