There seems to be a bug in the step function of the robot_supervisor.py

[RLMilestone]

        if super(Supervisor, self).step(self.timestep) == -1:
            exit()

        self.apply_action(action)
        return (
            self.get_observations(),
            self.get_reward(action),
            self.is_done(),
            self.get_info(),
        )

In RL, it seems to be more natural to apply_action and then Supervisor.step(). Otherwise, you will not get correct response of your action (delay by one timestep!)

[tsampazk]

Hello @RLMilestone, thank you for opening this issue! This is a point of debate for the project since the very beginning. Indeed it seems more natural to step the controller after calling apply_action in the robot-supervisor scheme.

We will have to look into it in depth, because it would be nice for us to do the same in both the robot-supervisor scheme and the emitter-receiver scheme, but in the emitter-receiver scheme it might cause unforeseen issues due to the way the messages are transmitted.