Exposing GenerationConfig in the GRPO Trainer

[Superskyyy]

Feature request

Often people need to customize the generation config, now it's embedded in the training loop. Should be easy to extract it out.

Motivation

Customization

Your contribution

I can help to contribute.

[qgallouedec]

More control on the generation does make sense. A reasonable way to allow for more control is probably to add more generation args in the GRPOConfig.
Are you willing to contribute?

[Superskyyy]

More control on the generation does make sense. A reasonable way to allow for more control is probably to add more generation args in the GRPOConfig. Are you willing to contribute?

Yes I will contribute this feature.

[Benjoyo]

More control on the generation does make sense. A reasonable way to allow for more control is probably to add more generation args in the GRPOConfig. Are you willing to contribute?

Yes I will contribute this feature.

Please add stop_strings or stopping criteria :)

Although I don’t see why not exposing the full generation config to avoid the next issue of this type in a few weeks.

[Superskyyy]

More control on the generation does make sense. A reasonable way to allow for more control is probably to add more generation args in the GRPOConfig. Are you willing to contribute?

Yes I will contribute this feature.

Please add stop_strings or stopping criteria :)

Although I don’t see why not exposing the full generation config to avoid the next issue of this type in a few weeks.

@qgallouedec wdyt? Either we should just directly expose the entire generation config because there are all kinds of tricks that people might want to tune there.