Problems while training using SAC, DDPG and TD3

[zhengtiantian]

I have this problem when I use TD3, SAC or DDPG, how can I fix it? The scene is FlyThruGateAviary.

\myDrones\venv\lib\site-packages\stable_baselines3\common\policies.py:366: RuntimeWarning: invalid value encountered in divide return 2.0 * ((action - low) / (high - low)) - 1.0 \myDrones2\gym_pybullet_drones\control\DSLPIDControl.py:195: UserWarning: Gimbal lock detected. Setting third angle to zero since it is not possible to uniquely determine all angles. target_euler = (Rotation.from_matrix(target_rotation)).as_euler('XYZ', degrees=False) Traceback (most recent call last): File "\myDrones2\testlearn.py", line 53, in model.learn(total_timesteps=3000000) # Typically not enough File "\myDrones\venv\lib\site-packages\stable_baselines3\td3\td3.py", line 205, in learn return super(TD3, self).learn( File "\myDrones\venv\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 347, in learn rollout = self.collect_rollouts( File "\myDrones\venv\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 580, in collect_rollouts new_obs, rewards, dones, infos = env.step(actions) File "\myDrones\venv\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 162, in step return self.step_wait() File "\myDrones\venv\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 43, in step_wait obs, self.buf_rews[env_idx], self.buf_dones[env_idx], self.buf_infos[env_idx] = self.envs[env_idx].step( File "\myDrones\venv\lib\site-packages\stable_baselines3\common\monitor.py", line 90, in step observation, reward, done, info = self.env.step(action) File "\myDrones2\gym_pybullet_drones\envs\BaseAviary.py", line 336, in step clipped_action = np.reshape(self._preprocessAction(action), (self.NUM_DRONES, 4)) File "\myDrones2\gym_pybullet_drones\envs\single_agent_rl\BaseSingleAgentAviary.py", line 274, in _preprocessAction rpm, _, _ = self.ctrl.computeControl(control_timestep=self.AGGR_PHY_STEPS * self.TIMESTEP, File "\myDrones2\gym_pybullet_drones\control\DSLPIDControl.py", line 128, in computeControl rpm = self._dslPIDAttitudeControl(control_timestep, File "\myDrones2\gym_pybullet_drones\control\DSLPIDControl.py", line 234, in _dslPIDAttitudeControl target_rotation = (Rotation.from_quat([w, x, y, z])).as_matrix() File "_rotation.pyx", line 627, in scipy.spatial.transform._rotation.Rotation.from_quat File "_rotation.pyx", line 531, in scipy.spatial.transform._rotation.Rotation.init ValueError: Found zero norm quaternions in quat.

[JacopoPan]

@zhengtiantian

what you posted is an error in scipy.spatial.transform thrown by an invalid quaternion but I am not sure how you got there. You seem to be running testlearn.py which is something you adapted from the original code?

[zhengtiantian]

@zhengtiantian

what you posted is an error in scipy.spatial.transform thrown by an invalid quaternion but I am not sure how you got there. You seem to be running testlearn.py which is something you adapted from the original code?

Yes, I changed env to FlyThruGateAviary

[JacopoPan]

@zhengtiantian

I'd love to help and get to the bottom of this but this looks like a value error in scipy: to understand why it appears you need to at least share (1) some info about your system (os, python interpreter, related packages versions, etc.), (2) what you tired that worked (installation, basic scripts), (3) what you changed that started raising the error (so that I can replicate it). Ideally with terminal line instructions, actual code, and terminal output.