[HANDS-ON BUG] The environment CartPole-v1 used in unit 4 has weird configurations

[k0T0z]

Describe the bug

Try to run the original notebook failed.

1. The env.reset() returns a tuple of two items for CartPole-v1 only. We expect only the state.
2. The env.step(action) returns a tuple of 5 items for CartPole-v1 only. We expect only 4 items in the tuple.
3. The softmax function has a dimension of 1 which is wrong, it should be 0.
4. The record_video() function fails for CartPole-v1 environment only for some reason.

Material

I didn't use Google Colab and here is my OS info:

• Windows 10

[k0T0z]

Hmmmmm, it seems that the code runs well on Colab but not locally!!

[simoninithomas]

Hey there 👋 sorry for the delay. But you're the first to encounter this issue. That might be also the version of Cartpole you use. For this we use Gym not Gymnasium "You may be wondering why we install gym and not gymnasium, a more recent version of gym? Because the gym-games we are using are not updated yet with gymnasium."

[k0T0z]

No problem at all, as I said the problem comes when running this code locally, dk why, maybe I made a mistake.

[simoninithomas]

I think it's because you might not use the same environment. Unit 4 uses gym == 0.21 not gymnasium 😄

[k0T0z]

Hmmm, you may be right yeah, I will check and confirm.

[simoninithomas]

Hey there 👋 Did you found if it was because of that?

[k0T0z]

Sorry for being late, yeah, it is definitely because of inconsistent versions so I am closing this.
Thank you.

[ra9hur]

Even I have been running this locally and using gymnasium. I made respective changes for env.reset() and env.step(action).

Training and evaluation went fine.

However, I am struck at record_video() while pushing to Hub. I get an error "The image must have at least two spatial dimensions".

I did change eval_env to include render_mode.
eval_env = gym.make(env_id, render_mode="rgb_array")

Any suggestions ?

unit4.zip

[ra9hur]

Restarted the system / notebook and this error does not show up. However, "push to hub" process takes up all the memory and finally crashes.