minmea icon indicating copy to clipboard operation
minmea copied to clipboard

Lost of precision due to the use of float as type for degrees

Open jmfermun opened this issue 2 years ago • 4 comments

Function minmea_tocoord now converts positions from type minmea_float to type double.

Conversion from latitude and longitude to meters:

  • delta_latitude_in_meters = delta_latitude_in_degrees * 40008000 / 360
  • delta_longitude_in_meters = delta_longitude_in_degrees * 40075160 * cos(latitude_in_degrees) / 360
  • The below examples are centered in the conversions of longitude to meters in its worst case (latitude 0º).

The use of the type minmea_float for storing the positions seems to be convenient:

  • Mapping of digits in 32 bit integer range: 2**31 21474 83648 Latitude XDDMM.MMMMM Longitude DDDMM.MMMMM
  • It allows to store 5 decimals, so we will obtain an error of 0.00001 / 60 * 40075160 * cos(0) / 360 = 0.01855 m.
  • In the readme it is specified an error of 2 cm. It matches with the above value.

Use of float for storing the positions is not convenient:

  • The precision of float is between 6 and 7 digits.
  • The precission of double is between 15 and 16 digits.
  • The worst case is with a latitude 0.0 º (equator) and a longitude with 3 digits used by the degrees part.
  • If we use the type float, 4 digits are available for the decimal part of the longitude, so we will obtain an error of 0.0001 * 40075160 * cos(0) / 360 = 11.13 m.
  • If we use the type double, 13 digits are available for the decimal part of the longitude, so we will obtain an error of 0.0000000000001 * 40075160 * cos(0) / 360 = 0.00000001113 m.
  • We need at least 7 digits for the decimal part of the longitude to obtain an error of 0.0000001 * 40075160 * cos(0) / 360 = 0.01113 m.

References:

  • https://blog.demofox.org/2017/11/21/floating-point-precision/.
  • https://stackoverflow.com/questions/3024404/transform-longitude-latitude-into-meters

jmfermun avatar Jan 29 '23 19:01 jmfermun

I also would like this modification.

matthieu-c-tagheuer avatar Dec 08 '23 13:12 matthieu-c-tagheuer

+1, looks like a simple change. If there are MCUs out there that really can't or shouldn't be using doubles, then maybe provide a #ifdef MINMEA_USE_FLOAT for users that really need 32-bit floats?

KJ7LNW avatar Feb 20 '24 20:02 KJ7LNW

I also would like this modification. there is experience of experiencing problems with float-type GPS coordinate errors.

johlim avatar Feb 20 '24 22:02 johlim

The tests are not building. You should change the use of fabsf in tests.c to fabs to follow the type change

lucid-at-dream avatar Apr 16 '24 14:04 lucid-at-dream