meshtastic
[Feature Request]: Use DOP from GPS devices to determine if we have a good enough position to save airtime
Platform
NRF52, ESP32, RP2040, Linux Native
Description
We are currently not using the DOP value to throw out bad position values causing smart position to use a lot of airtime while the unit is stationary.
Example image from @pdxlocations of a stationary vehicle generating 10K position packets with smart position on
@GPSFan , do you happen to have numbers for what makes a 'bad' DOP we could throw out?
DOP is a measure of how poor the geometry of the sats used in the fix is. Bad DOP means bad location accuracy, but you can have good DOP and a lousy fix. What constitutes bad DOP varies by whom you ask and in what context. For a surveyor, anything less than 1.5 is BAD. For a hiker 3-4 might be fine, as any position is better than no position. A good view of the sky should get you good DOP. In a canyon, the DOP might drop to 10+. Do you want to throw out the only position you have even if it is poor? Columbus would have killed for 100km accuracy.
In any case, Meshtastic only uses the GGA and RMC sentences so HDOP is all we get. Unfortunately the GSA sentence that contains the other DOPS can overload the serial input, without providing any thing more useful. In u-blox receivers its' firmware follows the NMEA spec and will only out GSA data for 12 sats. Most other receivers don't follow the spec and will output GSA data for as many sats as they are tracking. My Wireless Tracker was tracking 32 sats this morning and while it might be interesting for some users which sats were tracked, most don't care. 32 sats worth of GSA data is a lot, to deal with for only PDOP, HDOP and VDOP.
In the normal use case, Meshtastic does not allow the receiver to acquire more sats than necessary for the initial fix. This can lead to bad DOP even with a good sky view. Over time, the receiver may be able to acquire more sats. The GPS (u-blox) is then put to sleep for a while to save power. Receiver power can be switched off as well, in both cases the receiver will have to re-acquire a fix after it wake up. This is usually quite fast, but that leaves little time to acquire new sats.
Thank you so much for the most excellent explanation,@GPSFan!
I've been tracking some of the position data from my local mesh recently. Some examples below looking at 1 week of data.
A) +/- 100m accuracy - node is half way up a mountain, which blocks reception on some portion of the sky
B) +/- 1km accuracy - node is mid-way up a dense skyscraper forest
C) +/- 100km accuracy - receiver is at the bottom of a skyscraper canyon
Conveniently, I control the node in case C). Thinking to perhaps start there and maybe throw out the very worst of the positions, and hope it might help some of the other cases slightly. Plan is to get some HDOP values from the NMEA messages we have enabled right now. At least try throwing out HDOP >20.
How are you tracking that data, I'd like to do that too and don't have a clue how, is that in Meshtastic? or some external app?
Yeah, it's very useful!
Basically, I have a meshtastic device connected to a PC. I'm running some custom python code that connects to it using the serial connection and saves the various packets. The python code also runs a web service that will dump the data using geojson. Then, a little bit of javascript with leaflet to render the geojson to a map. Very much a work in progress and not ready for the light of day :)
Are you on the discord? I could walk you through something a bit less complicated?
I am cynfab on discord. That capability would be very useful in many situations, I've always wanted some sort of breadcrumb trail capability in Meshtastic. Maybe something that could be integrated into linux-native? No need for a web interface, just generate a .csv or .kml. I'm off grid tomorrow and most of the weekend, going to do some real world testing of my Meshtastic use-case, last time I did this sort of testing was a year ago, and Meshtastic has evolved a lot since then. we will see how it survives first (second) contact. Currently I use a separate GPS logger to record my path and then generate a .kml for google earth.
Yeah, it's very useful!
Basically, I have a meshtastic device connected to a PC. I'm running some custom python code that connects to it using the serial connection and saves the various packets. The python code also runs a web service that will dump the data using geojson. Then, a little bit of javascript with leaflet to render the geojson to a map. Very much a work in progress and not ready for the light of day :)
Hey, you may be interested in using Own tracks to make this work. I wrote up something similar at https://hackaday.com/2023/10/11/meshtastic-and-owntracks-to-kick-your-google-habit/
Owntracks sounds really neat, except for the connectivity issue, in my use case there is no connectivity to the rest of the world for MQTT. For use cases that have that connectivity.... muy bueno, but I need something that integrates with the Android/IOS app to record those positions. I guess I could have a local linux-native node along and do it on that device.
Real world test went well, track on Meshtastic map agreed in real time with the ground truth within the expected accuracy, 15 miles of off roading with 3 vehicles (each had a node), one breakdown which was reported via Meshtastic, I had to back up almost 1000 feet up hill to a turn around before I could head back to the vehicle with the problem a half mile away.
Oh wait, the "Export rangetest.csv in the Android app does almost what I want, just missing my own nodes position data.
Ran code capturing HDOP for several hours; interesting results.
Early conclusion is that we may want to consider the number of satellites as well as DOP. I had one HDOP=10.25 with 7 satellites and it was a decent position: +/- 30m.
However a HDOP of 7.18 with 4 satellites was out by 10km.
Continuing to try and find a reasonable upper bound for HDOP...
I've always wanted some sort of breadcrumb trail capability in Meshtastic.
See https://tracker.bircom.in it has "Show Tracklog" use mqtt.bircom.in user:uplink pass:uplink to show your mesh nodes.
Liam's map has "Position history" also.
This has been resolved through a number of other qualitative safeguard for position time sourcing
I suggest this is reopened and reexamined.
Replace the 100m smart positioning threshold with a fudge factor multiplied by the pdop|hdop figure, and/or some moving averages to remove two expected but opposite wild readings.
Also make the internal position figure unchanging unless the above circle of uncertainty is exceeded, to prevent constant updates within the margin of error.
@fifieldt what conversion from the protobuf integers did you use when when working with the dop values previously? Feels like we can throw out a lot of these bad positions, as Columbus was a long time ago.
@garthvh - only tried DOP from tinyGPS on a device - not from what comes over the mesh:
https://github.com/meshtastic/firmware/compare/master...fifieldt:meshtastic-firmware:gps-nolock-time
In my (admittedly limited) testing I was not able to find an upper limit for DOP that was both safe (did not get rid of positions there were close to where they should be) and useful (got rid of some bad positions).
I think we want to use PDOP as it has both horizontal and vertical errors, looks like anything over 6 in PDOP is junk
To get PDOP in our firmware we need to enable new NMEA sequences and I think do a little bit of TinyGPS bug fixing upstream. A nontrivial effort :)
I would be very careful making assumptions about particular values for DOP from tables found online :) I found many more useful positions at DOP 10 than I did bad ones.
Our filtering on the device GPS is really bad, we need to be filtering out way more positions, you can't generate a good gps track with the data we output and we send out lots of bad data. PDOP is exponential, so I don't think there is any value in keeping the higher numbers.
For smart position especially we need to be smart, we are using a bunch of airtime on bad positions. See https://github.com/meshtastic/firmware/issues/6785
Just to provide an update to anyone following this issue, on our position quality improvement plan:
- Write the code to stop the 'less good' position update immediately after lock (fixed in https://github.com/meshtastic/firmware/pull/8064 , showed some genuine improvement)
- Write the code to configure GPS properly, for movement thresholds based on use case (waiting for https://github.com/meshtastic/protobufs/pull/749 )
- Afterward, revisit HDOP testing to see if we can derive a useful threshold (results from last time are in the comments in this issue)
- If not, fork TinyGPS and implement GSA, so we can get PDOP
- Then, do PDOP testing to derive an appropriate threshold