Other vehicle crash in intersection environment

[ahmedhatem3h]

Hi @eleurent , I wanted to train a dqn agent on the intersection environment. However, I found some issues and i don't know if this normal or not. When testing the environment I found that other vehicles can crash with each other at the intersection which blocks the way for the ego_vehicle. Also, when the ego_vehicle stops before the intersection, another vehicle can come from behind and hit the ego_vehicle, which is then considered as a collision and a negative reward is given although this was the optimal action (to stop and wait until the it can pass safely). Is there anything that I change do to fix this problem. Thank you in advance.

[eleurent]

Hi @ahmedhatem3h ,

When testing the environment I found that other vehicles can crash with each other at the intersection which blocks the way for the ego_vehicle

Yes, I'm sorry. It's quite hard to come up with good decentralized behaviors for other agents to avoid each other. Of course, this could be achieved by sophisticated centralized schedulers, or traffic lights, but I wanted to keep things simple and just added a rudimentary collision prediction in the behaviour of other vehicles. But of course, it sometime fails which results in collisions, blocking the way. I figured it was fine for my own purpose, since it did not happen too often and it's reasonable to expect the ego-vehicle to simply wait the end of episode in these situations. But I agree that it is not ideal, and I welcome any contribution on that matter.

Also, when the ego_vehicle stops before the intersection, another vehicle can come from behind and hit the ego_vehicle, which is then considered as a collision

This, however, should not happen (since the IDM model is supposed to handle collision avoidance with the leading vehicle on the lane), and I have not observed it. I'll check if I can reproduce it.

[ahmedhatem3h]

Hi @eleurent ,

This, however, should not happen (since the IDM model is supposed to handle collision avoidance with the leading vehicle on the lane), and I have not observed it. I'll check if I can reproduce it.

Well, in my case, it could be easily reproduced if you run the environment and choose to stop always (action 0). I also wanted to ask if there is a preferred simulation and policy frequencies for this environment in general and for the IDM controller (so there will be less collision). Also, were you able to make an agent that solve this environment? Thank you in advance.

[eleurent]

Well, in my case, it could be easily reproduced if you run the environment and choose to stop always (action 0).

Thanks , I'll try it and get back to you.

I also wanted to ask if there is a preferred simulation and policy frequencies for this environment in general and for the IDM controller (so there will be less collision).

I choose the simulation frequency around 15Hz, and the IDM controller (for other vehicles) is independent of the policy frequency (which is only for the ego vehicle).

Also, were you able to make an agent that solve this environment? Thank you in advance.

I had some relative success, documented here: https://eleurent.github.io/social-attention/ but I wouldn't consider it as 100% solved. It is implemented in https://github.com/eleurent/rl-agents/, the page mentioned above provides instructions to reproduce my results.

[ahmedhatem3h]

I had some relative success, documented here: https://eleurent.github.io/social-attention/ but I wouldn't consider it as 100% solved. It is implemented in https://github.com/eleurent/rl-agents/, the page mentioned above provides instructions to reproduce my results.

Okay thank you I will look into it.