How to do filtering in the step 1 "Trajectory Collection and Filtering"

[DA21S321D]

The readme file instruct the LightGPT Training procedures by starting at Step 1: Imitation Fine-tuning, so how to perform the filtering in the picture ?

[DA21S321D]

I mean how to use that Critic to filter

[SQLai2099]

You can refer to our paper. We selectively retain only reasoning trajectories where the selected actions are associated with the highest estimated future reward.
Sorry for excluding the code of this process. We will add this script soon.

[Jarvis-K]

I use GPT-4-turbo to collect data and the imitation learning does show a good result without filtering.

[SQLai2099]

I use GPT-4-turbo to collect data and the imitation learning does show a good result without filtering.

Good to know. In my case, gpt-4-0613 still needs this filtering strategy.