PX4-Autopilot
PX4-Autopilot copied to clipboard
PX4
Acceleration and Gyro sensors send STALE error messages
Describe the bug
When running gz simulation normally, two different errors appears shortly after starting:
ERROR [sensors] Accel #0 fail: STALE!
ERROR [sensors] Gyro #0 fail: STALE!
To my knowledge, this was first reported by @slgrobotics in https://github.com/PX4/PX4-Autopilot/pull/22550#issuecomment-1946286865.
To Reproduce
On current main branch (74303a79e195083144c9d63fd2346b5828015a31) run
make px4_sitl gz_x500
Expected behavior
No error should appear.
Screenshot / Media
No response
Flight Log
N/A
Software Version
Current main 74303a79e195083144c9d63fd2346b5828015a31
Flight controller
SITL
Vehicle type
None
How are the different components wired up (including port information)
No response
Additional context
No response
The error seems printed by this line of code related to the data validator: https://github.com/PX4/PX4-Autopilot/blob/74303a79e195083144c9d63fd2346b5828015a31/src/modules/sensors/voted_sensors_update.cpp#L312C14-L312C79
@dagar This issue comes from your PR: #22736
@frede791 is taking a look at why this happens.
There is also some other warning, telling us that there is no gyro selected, but that has been around for a while.
WARN [vehicle_angular_velocity] no gyro selected, using sensor_gyro:0 1310988
The error seems to be caused by removing the line param set-default SENS_IMU_MODE 0 from the rcS file. It seems that when running gz it no longer gets set to any value leading to the checkFailover function being called in the voted_sensors_update.cpp file which creates this error.
Hello, Is there any solution for these STALE messages. I am exactly in the same situation. Thanks
@goupil35000 a temporary solution until @dagar replies is to interactively rebase on your branch and remove the following commit:
https://github.com/PX4/PX4-Autopilot/pull/22736
@PerFrivik Many Thanks.
I will wait until @dagar's changes. I will retest after.
@dagar @Jaeyoung-Lim @PerFrivik @slgrobotics As discussed yesterday in the call, the issue can be fixed by adding some noise distributions to the imu sensor in the sdf file. I test ran this using the screenshot below and it worked fine. @Jaeyoung-Lim mentioned that we should use the values based on what was used in gazebo-classic. I will look into this.
@dagar Can the build-bot be requested or is the process of updating the submodule done automatically?
It is done in https://github.com/PX4/PX4-gazebo-models, but the submodule hash in PX4 main is old
@VBS2004 @slgrobotics @PerFrivik @goupil35000 The gz submodule has been updated by PR https://github.com/PX4/PX4-Autopilot/pull/22841. Please check that the models you need are working without the error message.
Confirming that the STALE messages are no longer there, when x500 or lawnmower are built from latest "main". Thanks!
Are these warnings/messages normal? Popped up consistently when SITL drone was sitting idle in background.
WARN [health_and_arming_checks] Preflight: GPS Vertical Pos Drift too high WARN [health_and_arming_checks] Preflight: GPS Vertical Pos Drift too high WARN [health_and_arming_checks] Preflight: GPS Vertical Pos Drift too high WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight: GPS Horizontal Pos Drift too high WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: height estimate not stable WARN [health_and_arming_checks] Preflight Fail: vertical velocity unstable WARN [health_and_arming_checks] Preflight Fail: vertical velocity unstable WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error WARN [health_and_arming_checks] Preflight Fail: vertical velocity unstable WARN [health_and_arming_checks] Preflight Fail: height estimate not stable WARN [health_and_arming_checks] Preflight Fail: vertical velocity unstable WARN [health_and_arming_checks] Preflight Fail: vertical velocity unstable WARN [health_and_arming_checks] Preflight Fail: Yaw estimate error
@vibgyor-s It seems the system has been idle for a long time (20+ minutes). I also receive these warnings, which could be related to the estimator. With minimal acceleration data for fusion, the estimator may struggle to handle drift effectively.
I get these as well. They are unrelated to the issue covered here I believe.
@vibgyor-s It seems the system has been idle for a long time (20+ minutes). I also receive these warnings, which could be related to the estimator. With minimal acceleration data for fusion, the estimator may struggle to handle drift effectively.
How to fix this.I am also receiving the same errors when running mavsdk example of gotolocation ,it just rolls and crashes.
@vibgyor-s It seems the system has been idle for a long time (20+ minutes). I also receive these warnings, which could be related to the estimator. With minimal acceleration data for fusion, the estimator may struggle to handle drift effectively.
How to fix this.I am also receiving the same errors when running mavsdk example of gotolocation ,it just rolls and crashes.
Do you also just crash, if you instantly take off and do the goto?
@VBS2004 Maybe this warrants making a separate issue?
Idk I am a complete beginner in this stuff Should I open a new issue?
It would be good to know why this is occurring as it is unexpected behaviour.
@frede791 :thinking: :sweat: This is on the newest main, and I ran git submodule update --recursive --init before running the sim.
I just did:
mkdir src0; cd src0
git clone https://github.com/PX4/PX4-Autopilot.git --single-branch --recursive
cd PX4-Autopilot/
make px4_sitl gz_r1_rover
There are no STALE messages, but the following still are present:
Warning [Utils.cc:130] [/sdf/world[@name="default"]
/spherical_coordinates/altitude:.../worlds/default.sdf:L153]: XML Element[altitude],
child of element[spherical_coordinates], not defined in SDF.
...
pxh>
pxh> WARN [health_and_arming_checks] Preflight: GPS Vertical Pos Drift too high
WARN [health_and_arming_checks] Preflight: GPS Vertical Pos Drift too high
WARN [health_and_arming_checks] Preflight: GPS Horizontal Pos Drift too high
The "altitude" thing has been fixed (by "elevation") - but the submodule hash hasn't been brought to latest.
The problems in gz_x500 is resolved ` trajectory_msg = TrajectorySetpoint() trajectory_msg.timestamp = int(Clock().now().nanoseconds / 1000) trajectory_msg.velocity[0] = float('nan') trajectory_msg.velocity[1] = float('nan') trajectory_msg.velocity[2] = float('nan') trajectory_msg.position[0] = 10 trajectory_msg.position[1] = 10 trajectory_msg.position[2] = -20 trajectory_msg.acceleration[0] = float('nan') trajectory_msg.acceleration[1] = float('nan') trajectory_msg.acceleration[2] = float('nan') trajectory_msg.yaw = float('nan') trajectory_msg.yawspeed = self.yaw
self.publisher_trajectory.publish(trajectory_msg)`
this code works without any issues except this time jump detected. Resetting time synchroniser.
But the example I took directly from mavsdk is not working it gives so many errors and drone just crashes as I have mentioned in this issue https://github.com/PX4/PX4-Autopilot/issues/22861
Here's the errors:
ERROR [health_and_arming_checks] angular velocity no longer valid (non-finite values) INFO [tone_alarm] battery warning (fast) WARN [health_and_arming_checks] Preflight Fail: ESC failure detected WARN [health_and_arming_checks] Preflight Fail: Flight termination active INFO [commander] Disarmed by lockdown WARN [health_and_arming_checks] Preflight Fail: height estimate error WARN [health_and_arming_checks] Preflight Fail: velocity estimate error WARN [health_and_arming_checks] Preflight Fail: position estimate error WARN [health_and_arming_checks] Preflight Fail: ESC failure detected WARN [health_and_arming_checks] Preflight Fail: Flight termination active
@PerFrivik Are you running the newest version of the rover?
The one on main, or do I need to checkout a different commit hash?
@PerFrivik - maybe, your "main" is not main enough. ;-)
Try:
mkdir src0; cd src0
git clone https://github.com/PX4/PX4-Autopilot.git --single-branch --recursive
cd PX4-Autopilot/
and then "meld <your PX4-Autopilot> <~/src0/PX4-Autopilot>"
I noticed many times before that git doesn't properly update submodules every time, so I use meld when in doubt.
I guess, my main was really not "main" enough hahah. I fixed it with a make distclean :+1:. Thanks for the help!