How to collect information about (s,a,r,s_,done)

[wuyukun-tong]

Hi, How to collect information about (s,a,r,s_,done) in manual control mode in intersection）env. I want to test the imitation learning algorithm！ Thanks!

[eleurent]

Hi, Unfortunately the current software does not currently support this use case, so you will have to modify it a little.

Basically,

• Normally, for an RL setting, you (the agent) provide the action a to the environment, which returns (s', r, done, info) data
• In this 'manual control' mode, the provided action is ignored and replaced with user inputs
• but this user-input action is unfortunately not saved at the moment, so it cannot be accessed/returned by the environment

So I suggest that you edit the code such that, when the env.render() is called

https://github.com/eleurent/highway-env/blob/742ce2e69bf00169f2b250e0388c6b9926c026ec/highway_env/envs/common/abstract.py#L259

and env.viewer.handle_event() is also called (handling of user inputs) and forwards to the EventHandler,

https://github.com/eleurent/highway-env/blob/742ce2e69bf00169f2b250e0388c6b9926c026ec/highway_env/envs/common/graphics.py#L163

you internally save the action in the env, and return it as the step's action e.g. in the info field.