question about switching kinematic model to dynamics model

[Exasor]

Hi, I used DDPG and kinematic model to train vehicle in highway env and got a nice performance, but when I used dynamics model and the same DDPG model, the agent can't learn to overtake well, and I also added the heading, lateral_speed, yaw_rate,d_lateral_speed and d_yaw_rate of ego car to obs, but it didn't perform better, so I think if there are some other changes I should make while using the dynamics model? Hoping for your suggestions.

[eleurent]

Hi, This is quite strange, as I expect the two models to be pretty close, especially is the tire friction coefficients are high enough. Have you tried training the dynamics model without the additional variables (lateral_speed, yaw_rate, etc.)? Maybe those are not so relevant to the decision, and are distracting the agents from other more useful inputs?

[Exasor]

Thanks a lot for your reply. I have tried the original KinematicObservation without the additional variables in both the original coefficient and the Carsim full size suv coefficient using dynamics model:

    MASS: float = 2532  # [kg]  
    LENGTH_A: float = 1.330 # [m]
    LENGTH_B: float = 1.616  # [m]
    INERTIA_Z: float = 3524.9  # [kg.m2]
    FRICTION_FRONT: float = 103467.96  # [N/rad]
    FRICTION_REAR: float = 103467.96   # [N/rad]

but the ego car usually sways, causing more collisions：

https://user-images.githubusercontent.com/56293255/113411502-82bc3400-93e8-11eb-92d1-4a09453ac036.mp4

in kinematic model, the ego car seldom perform like this and is more stable:

https://user-images.githubusercontent.com/56293255/113411910-93b97500-93e9-11eb-8af9-53dd9836ec0c.mp4

[Exasor]

Using the original coefficient, the ego car sways less, but it still can't overtake as well as the kinematic model performs:

https://user-images.githubusercontent.com/56293255/113412785-bba9d800-93eb-11eb-9349-2fde81408877.mp4

[Exasor]

I think the number of episodes may be not enough, and I will try more episodes.

[eleurent]

Yes these oscillations do not look right, the dynamics look quite unstable. Could this be due to numerical instability, caused by the integration time-step being too large?

On the other hand, it seems to me that the agent uses quite large steering angles, much larger than what I expect to be safe for highway driving, I can imagine that this would cause drifting. So it might help to decrease the maximum steering angle, in the action type configuration?

[Exasor]

I set 5Hz of simulation_frequency and pi/4 of maximum steering angle before, I will try a larger frequency and smaller angle. Thanks for your suggestions!

[eleurent]

Also since the car is drifting a lot I think that the yaw_rate variable is also relevant to the decision, to help stabilize the vehicle. Perhaps not other variables (d_yaw_rate, lateral_speed, d_lateral_speed), but at least this one.

[Exasor]

I will try it, thanks again.