deep-rl-class icon indicating copy to clipboard operation
deep-rl-class copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

[HANDS-ON BUG] The environment ``CartPole-v1`` used in unit 4 has weird configurations

Open k0T0z opened this issue 1 year ago • 1 comments
trafficstars

Describe the bug

Try to run the original notebook failed.

  1. The env.reset() returns a tuple of two items for CartPole-v1 only. We expect only the state.
  2. The env.step(action) returns a tuple of 5 items for CartPole-v1 only. We expect only 4 items in the tuple.
  3. The softmax function has a dimension of 1 which is wrong, it should be 0.
  4. The record_video() function fails for CartPole-v1 environment only for some reason.

Material

I didn't use Google Colab and here is my OS info:

  • Windows 10

k0T0z avatar Jan 18 '24 16:01 k0T0z

Hmmmmm, it seems that the code runs well on Colab but not locally!!

k0T0z avatar Jan 21 '24 11:01 k0T0z

Hey there 👋 sorry for the delay. But you're the first to encounter this issue. That might be also the version of Cartpole you use. For this we use Gym not Gymnasium "You may be wondering why we install gym and not gymnasium, a more recent version of gym? Because the gym-games we are using are not updated yet with gymnasium."

simoninithomas avatar Mar 04 '24 16:03 simoninithomas

No problem at all, as I said the problem comes when running this code locally, dk why, maybe I made a mistake.

k0T0z avatar Mar 05 '24 13:03 k0T0z

I think it's because you might not use the same environment. Unit 4 uses gym == 0.21 not gymnasium 😄

simoninithomas avatar Mar 06 '24 15:03 simoninithomas

Hmmm, you may be right yeah, I will check and confirm.

k0T0z avatar Mar 06 '24 15:03 k0T0z

Hey there 👋 Did you found if it was because of that?

simoninithomas avatar Apr 18 '24 07:04 simoninithomas

Sorry for being late, yeah, it is definitely because of inconsistent versions so I am closing this. Thank you.

k0T0z avatar Apr 18 '24 23:04 k0T0z

Even I have been running this locally and using gymnasium. I made respective changes for env.reset() and env.step(action).

Training and evaluation went fine.

However, I am struck at record_video() while pushing to Hub. I get an error "The image must have at least two spatial dimensions".

I did change eval_env to include render_mode. eval_env = gym.make(env_id, render_mode="rgb_array")

Any suggestions ?

unit4.zip

ra9hur avatar Jun 05 '24 17:06 ra9hur

Restarted the system / notebook and this error does not show up. However, "push to hub" process takes up all the memory and finally crashes.

ra9hur avatar Jun 06 '24 07:06 ra9hur