setting the velocity to 0 in each step

[goran-j]

What is the reason for setting the velocity to 0 after each step in the agent?

self.client.moveByVelocityAsync(vx=vx, vy=vy, vz=vz, duration=1, drivetrain=airsim.DrivetrainType.MaxDegreeOfFreedom, yaw_mode=airsim.YawMode(is_rate=False, yaw_or_rate=180 * (alpha + psi) / np.pi), vehicle_name=self.vehicle_name) time.sleep(0.07) self.client.moveByVelocityAsync(vx=0, vy=0, vz=0, duration=1, vehicle_name=self.vehicle_name)

[aqeelanwar]

There are two reasons to that

1. If we don’t set the velocity to zero, then with subsequent actions, the velocity vector keeps on accumulating and the speed of the drone keeps on increasing until it becomes so high that it crashes. So, we set the velocity, wait for the drone to travel for some time, and then set it back to zero.
2. In the RL algo (off-policy), we observe a state, move the drone, observe new state and reward, and based on this tuple we train the network. For a really slow CPU/GPU, the train step can take significant amount. During that time, if we don’t ensure that the drone is not moving, the drone would have traveled significant amount of distance, potentially losing information for intermediate states.

[goran-j]

Thank you for you answer, it makes sense (although maybe it would be more 'realistic' to pause the simulation and setting the max speed instead of stopping the drone)