How to use DDPG algorithm control the vehicle turn left in intersection env

[wuyukun-tong]

Mr. Leurent, when I used DQN algorithm to control vehicles in the environment of intersection, I take discrete actions and "lateral" are set as False. The vehicles can turn themselves at the intersection, realizing low-level controller control. However, when I use DDPG algorithm to control the vehicle, I use the continuous action and "lateral" are also set as False, the vehicle can only go straight without turning,when I set the longitude to true, the car was spinning in place and unable to stay in lane.Specially,the color of the controlled vehicle is different from that of the DQN algorithm , why does this situation occur? Can I solve this problem by setting lane departure as a reward function, and if so how can I add the reward function proposal to the "env.configuration" dictionary? I am looking forward to your reply. Thank you!

[eleurent]

Hi, Currently I have 3 action types:

• continuous actions, there the agent control the gas pedal (longitudinal) and / or the steering wheel (lateral). If one axis is disabled, the neutral command (0) is applied. Thus, if you disable lateral control, the vehicle only drives straight.
• discrete actions: it is the same definition as continuous actions, except that the action space is discretized.
• discrete meta actions: here, we use meta-action, which rely on low-level controllers (which produce the actual actions), namely a speed controller and a lateral position controller. The meta-actions allow to change the setpoints of these low-level controllers: change the desired speed, or change the lane being tracked. In that case, the space of possible setpoints is discrete (the lanes are discrete, and we choose a discrete set of allowed speeds)

From what I understand, what you want is a mixed (continuous and discrete) meta-action type, where you can set a continuous speed setpoint, and discrete lateral position (lane) set point. This is not currently implemented, and needs to be added.

Oh, and yes the vehicle color changes to indicate if we are doing low-level control (yellow) or high-level control (green). The tires also become visible.

[wuyukun-tong]

Thank you very much for your reply

At 2022-04-23 18:06:54, "Edouard Leurent" @.***> wrote:

Hi, Currently I have 3 action types:

continuous actions, there the agent control the gas pedal (longitudinal) and / or the steering wheel (lateral). If one axis is disabled, the neutral command (0) is applied. Thus, if you disable lateral control, the vehicle only drives straight. discrete actions: it is the same definition as continuous actions, except that the action space is discretized. discrete meta actions: here, we use meta-action, which rely on low-level controllers (which produce the actual actions), namely a speed controller and a lateral position controller. The meta-actions allow to change the setpoints of these low-level controllers: change the desired speed, or change the lane being tracked. In that case, the space of possible setpoints is discrete (the lanes are discrete, and we choose a discrete set of allowed speeds)

From what I understand, what you want is a mixed (continuous and discrete) meta-action type, where you can set a continuous speed setpoint, and discrete lateral position (lane) set point. This is not currently implemented, and needs to be added.

Oh, and yes the vehicle color changes to indicate if we are doing low-level control (yellow) or high-level control (green). The tires also become visible.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>