Extremely slow improvement when pretraining on halfcheetah-expert-v2 dataset

[NathanWalt]

Thank you for sharing your excellent work. I'm currently trying to pretrain the diffusion policy model on halfcheetah-expert-v2 dataset, but the learned policy improve a lot slower than the one trained on halfcheetah-medium-v2 dataset. For example, after 100 epochs, the one trained on halfcheetah-expert-v2 achieves an average episode reward of around 600, while the one trained on halfcheetah-medium-v2 achieves around 4000.

To obtain the dataset, I ran python script/dataset/get_d4rl_dataset.py --env_name halfcheetah-expert-v2 --save_dir ./data/halfcheetah-expert-v2. To pretrain and evaluate the model, I simply change the env_name item in the corresponding cfg files from halfcheetah-medium-v2 to halfcheetah-expert-v2.

[allenzren]

Hi Weikang I haven't tried to use the expert dataset but this is somewhat unexpected. I can try getting my hands on it sometime this week.