dawarich icon indicating copy to clipboard operation
dawarich copied to clipboard

Speed values are incorrect

Open framesfree opened this issue 6 months ago • 20 comments

Since the introduction of the color-coded routes I noticed that the speed value that is shown on the map per point was strange. I didn't notice it right away since I though that maybe I was really slow in the traffic. However, today I was reviewing the data and discovered that I was going around 22 kph on a highway when I distinctly remember going around 80 there. I thought maybe it is Dawarich iOS client issue but I opened up my data from google takeout and here for instance it shows 35 km/h on a highway: Image I opened up the data (My data -> Points) and found those points that roughly correspond to the screenshot above:

Speed, km/h	Recorded At		Coordinates
33		15 Apr 2025, 09:53:09	49.5030488, 6.1521719	
34		15 Apr 2025, 09:53:42	49.508576, 6.1645845	
33		15 Apr 2025, 09:54:08	49.5105568, 6.17637	
22		15 Apr 2025, 09:54:43	49.5070466, 6.1903454	
27		15 Apr 2025, 09:55:11	49.5075086, 6.1995048

Simple calculation based on position shows that I was moving with the average speed of 109.8 kph and dawarich average is 30.5, which is 3.6x slower. Maybe there is a conversion issue in the units somewhere? The real question is how those numbers are calculated. I also checked the raw takeout file and it seems that the speed is not logged there. Maybe it's just my instance? Can anyone try to reproduce this?

My setup Dawarich is running on a VPS with docker compose (arm64/v8). "Default" dawarich_db "postgis/postgis:17-3.5-alpine" container is replaced (due to lack of support for arm64) with "postgres:17" + postgresql-17-postgis-3 installed.

Version Currently - 0.27.1. I am pretty sure the speed was showing correctly for this trip before, but I don't remember which version it was - for sure before route color coding but I also migrated to arm64/v8 since then.

To Reproduce Steps to reproduce the behavior:

  1. Go to 'Map'
  2. Load any route and hover over a datapoint
  3. Note the 'Current speed' value

framesfree avatar Jun 02 '25 22:06 framesfree

I'm also experiencing this issue. My settings are for imperial values and the values are wildly off. It appears that it is considering "stop" points that are several hours later than what actually took place. In the image I uploaded, I was driving on a 50 MPH roadway, and it shows 15mph. IT also shows a duration of six and a half hours, but I was in the car for a total 12 minutes only.

Image

TheBig-O avatar Jun 02 '25 23:06 TheBig-O

Same issue here. The real speed is around 110 km/h.

Image

On version 0.27.2, unit set to km/h, points recorded by Owntracks. This seems to affect all points in my database and not just recent ones.

Kiriyashiki avatar Jun 03 '25 06:06 Kiriyashiki

I've proposed a pull request to fix it: https://github.com/Freika/dawarich/pull/1322 Works on my instance, let's hope it doesn't break anything else

framesfree avatar Jun 04 '25 06:06 framesfree

Image

Same thing, way too low values.

baldemar-wuda avatar Jun 05 '25 19:06 baldemar-wuda

@framesfree The issue with this PR is that it will cause the same problem for those who don't experience wrong speed calculation. Have a look at https://github.com/Freika/dawarich/releases/tag/0.24.0

Freika avatar Jun 09 '25 14:06 Freika

Im on 0.28.1 and for me it shows the kms value in points and map If i multiply the values with 3.6 I get to the correct kmh value

Image

Image

Image

I have an even better example I dont think that my plane flew with 170-220 km/h

Im using Owntracks as tracker

Image

Image

narucx avatar Jun 17 '25 20:06 narucx

I have some extra information to add to the confusion. The following is a section of one of my tracks : When looking at the data in the (My Data > Points) page, the speed values are too low (I was cycling at the time), ~~but from what I've read, this is to be expected as I used OwnTracks which records in km/h, while Dawarich expects m/s.~~ Edit : Ignore that, it appears that Dawarich is correctly converting from km/s to m/s when receiving data from the Owntracks endpoint. The issue here is that the data on the Points page is not being converted back to km/s as the table header would suggest. Image

However on the Map screen, things get interesting. The speed values when hovering over a section of the route are roughly the same as shown in the points page. But when clicking on one of the points the speed is now showing the correct converted speed (m/s > km/h). https://github.com/user-attachments/assets/62a54ab8-2977-4b30-a638-f9d066728550

Apologies that my cursor was not captured in the video, I'm first hovering over each route section, followed by selecting each point. The values are :

Route:      9      8      7      4      10     10
Points: 34     30     26     15     37     37     29

Dawarich 0.28.1 OwnTracks 2.5.3 (Android) for location recording

Side note : The points on the map show the time with the timezone offset applied, but not on the points page which only shows the offset in the tooltip. This can be a little confusing to match up data between the two views.

23Suvu avatar Jul 02 '25 07:07 23Suvu

I have the same behavior. Speed is correct on the tooltips, but wrong in the route tooltips and the points page

Image

Image

Kiriyashiki avatar Jul 02 '25 07:07 Kiriyashiki

I think the speed unit shows on points page should be m/s, not km/h

leeaash avatar Jul 07 '25 03:07 leeaash

One thing i have noticed. i use MP/H and if the site ever changes to KM/S in the settings for the user, it breaks all the milage calculations on everything, including speed, miles tracked, etc. It converts them to KM but never back, and just seems permanent.

Huskiefluff avatar Jul 12 '25 12:07 Huskiefluff

I agree with @leeaash that m/s should be the base unit, since it makes conversion to all of the rest easier, and for some reason I'm able to use it across transport velocities, from walking to orbiting, whereas the usefulness of miles/hr and km/hr seem limited to driving.

MarkJaroski avatar Jul 12 '25 13:07 MarkJaroski

Okay, Owntracks endpoint you're all using (/api/v1/owntracks) to create your points from GPSLogger expects speed in km/h, because Owntracks sends it that way. My understanding is, the issue comes from GPSLogger sends speed in meters per second, is that right?

Freika avatar Jul 12 '25 22:07 Freika

Hello Freika, thanks for clarification but this seems surprising to me. If I understand correctly, GPS Logger can send speed data either as m/s (tag %SPD) or as km/h (%SPD_KPH)? This is also part of the Docs: https://dawarich.app/docs/tutorials/track-your-location/#gps-logger URL: http:///api/v1/owntracks/points?api_key=YOUR_API_KEY HTTP Body: { "_type" : "location", "t": "u", "acc": "%ACC", "alt": "%ALT", "batt": "%BATT", "bs": "%ISCHARGING", "lat": "%LAT", "lon": "%LON", "tst": "%TIMESTAMP", "vel": "%SPD" } and has also been discussed in https://github.com/Freika/dawarich/issues/700#issuecomment-2609095116.

Should I change in GPS Logger back to %SPD_KPH? Will that solve the issue?

Thanks.

zittel05 avatar Jul 13 '25 09:07 zittel05

@zittel05 if you could test if for me and see if the resulting values will be correct, I'll simply update docs.

Can you please check? Thanks

Freika avatar Jul 13 '25 09:07 Freika

Yes, I will start in 5 minutes for a short trip outside :-).

zittel05 avatar Jul 13 '25 09:07 zittel05

In my case, I'm using the Owntracks app/logger, and if I understand correctly, this logs in km/h. That is then converted to m/s when consumed by the Dawarich Owntracks endpoint. So at this point onward, Dawarich is storing m/s internally for all speed data.

I think the problem lies in the user facing display of this speed data, in that it's not always converted from m/s to km/h (or mi/h depending on the units set in the preferences).

In my previous comment above you can see that : Table on the Points page : Speed is displayed in m/s (while the table header shows km/h, missing conversion?) On the map, hovering a Route section : Speed is also displayed in m/s, same as the points page (I would expect km/h or mi/h as set in user preferences) On the map, selecting a single Point : Displayed speed is correctly converted to km/h

23Suvu avatar Jul 13 '25 10:07 23Suvu

Okay, I did the test, let me add my results to 23uvu's comments:

Logging with GPS Logger, I changed today the settings for user defined URL upload from "vel": "%SPD" to "vel: "%SPD_KPH".

  1. The speed in the Points page is displayed as km/h (as table header describes), the values are correct in km/h as recorded. Before changing to "%SPD_KPH" in GPS Logger (until yesterday), the points page showed speed values in m/s despite the table header.
  2. Hovering over the route in the map section today: speed is correctly shown as km/h, correct values as recorded. Yesterday's route with the old setting: km/h values are 3.6 times too low.
  3. Clicking on the route in the map section: speed is always "0 km/h", which might be okay, as there is no data on the route stored.
  4. Selecting a point in the map section: speed is displayed as "xy km/h" and the xy value is always 3.6 times too high (as converting from m/s to km/h, but the data point is already km/h).

My learning: GPS Logger can send speed data as defined in m/s ("vel": "%SPD") or km/h "vel: "%SPD_KPH" correctly. Dawarich is display sometimes the data values as transferred and stored (with "km/h" and sometimes multiplied by 3.6?

zittel05 avatar Jul 13 '25 11:07 zittel05

I am using OwnTracks for ongoing logging, but I believe that's an irrelevant detail. I am having the same issue as the other commenters. In order to clean up data from several other sources (imports from Traccar and Google Timeline), I have learned how to manually add and delete points using the Rails back end. As a result, I am able to confirm what's happening "internally" in terms of how Dawarich is storing and displaying data on my computer. Here is what I am seeing regarding the speed issue.

I am in the United States, and I know this makes the discussion of speed values more complicated, because have tried to set the display to mph wherever possible. There are places where speed values are still displayed in km/h or m/s, but I understand the unit conversion and this is something that could easily be fixed in a future Dawarich release. But in case this is contributing to the problem (I believe it is not), I will note that I set the following values in docker-compose.yml:

services: dawarich_app: environment: DISTANCE_UNIT: mi services: dawarich_sidekiq: environment: DISTANCE_UNIT: mi

Then inside Dawarich's web interface, I went to Settings->Map, and set Distance Unit: Miles. You will see that not all the conversions actually happened, though, which is a minor Dawarich bug, in my opinion, but not the primary bug here.

For comparison, I picked a selected point from yesterday (with points turned on in the map user interface), and got the following data.

Timestamp: 7/19/2025, 10:49:04 PM Latitude: 33.0096076 Longitude: -117.0962635 Altitude: 178m Speed: 99km/h Battery: 33% Id: 494595

(As you can see, it wasn't converted from 99 km/h to 61 mph, but I can do the conversion myself, and this does match the speed I was driving at that moment.) Hovering over the line next to the point in question brings up the following pop-up:

Start: 7/19/2025, 8:34:02 PM End: 7/19/2025, 10:53:34 PM Duration: 2h 20min Total Distance: 25.73 mi Current Speed: 17 mph

And of course this is wrong, because although here displayed in mph, the speed should really be 61 mph, and it's off by a factor of 3.6, which is the conversion factor from m/s to km/h as others have noted previously. As a separate matter, it would be very helpful if the pop-up when hovering over a path would show not just the "Start" and "End" times, but the timestamp of the closest point... the point which the current speed is derived from. That is a separate issue of course, but if that were added, it would largely alleviate the need to turn on points (risking accidentally dragging one of them), in order to find the timestamp of a specific line fragment.

The points table (My Data->Points) is also wrong, in some way. For the point in question it reports:

Speed, km/h: 27.5 Recorded at: 19 Jul 2025, 22:49:04 Coordinates: 33.0096076, -117.0962635

So as others have noted, the speed in the raw points display is displayed without conversion. In addition, it is claimed to be in km/h even though my units are set to mph, and the actual number is m/s.

Now for the important and new information to add to the discussion: the actual data stored in the database, accessed via the Rails console.

[4] pry(main)> Point.where(id:494595) Point Load (219.8ms) SELECT "points".* FROM "points" WHERE "points"."id" = $1 /* loading for pp */ LIMIT $2 [["id", 494595], ["LIMIT", 11]] => [#<Point:0x000076bebe83e390 id: 494595, battery_status: "charging", ping: nil, battery: 33, tracker_id: "62", topic: "owntracks/user/xqcq62", altitude: 178, longitude: nil, velocity: "27.5", trigger: "unknown", bssid: nil, ssid: nil, connection: "mobile", vertical_accuracy: 3, accuracy: 10, timestamp: 1752990544, latitude: nil, mode: nil, inrids: nil, in_regions: nil, raw_data: {"m" => 1, "bs" => 2, "_id" => "742e000e", "acc" => 10, "alt" => 178, "cog" => 317, "lat" => 33.0096076, "lon" => -117.0962635, "tid" => "62", "tst" => 1752990544, "vac" => 3, "vel" => 99, "batt" => 33, "conn" => "m", "_type" => "location", "point" => {"topic" => "owntracks/user/xqcq62", "created_at" => 1752990544}, "topic" => "owntracks/user/xqcq62", "action" => "create", "api_key" => "a2350cadf093674a96bcd96f5bad0621", "controller" => "api/v1/owntracks/points", "created_at" => 1752990544}, import_id: nil, city: nil, country: nil, created_at: "2025-07-19 23:11:06.271207000 -0700", updated_at: "2025-07-19 23:11:06.349423000 -0700", user_id: 1, geodata: {}, visit_id: nil, reverse_geocoded_at: nil, course: nil, course_accuracy: nil, external_track_id: nil, lonlat: [FILTERED], country_id: 155>]

As you can see, the speed reported via the raw_data record stored directly from OwnTracks is: "vel" => 99. In contrast, the authoritative speed stored in the database is: velocity: "27.5"

Since release 0.24.0 released 2025-02-10, as documented in the Dawarich changelog (https://dawarich.app/docs/changelog/), Dawarich is said to be converting kilometers per hour (as reported by OwnTracks) to meters per second for storage, and this documentation-by-changelog makes it clear that the data was imported and converted as designed, and that the velocity value stored in the database should be interpreted in meters per second, and in the case of my sample point, this meters per second value (27.5) does match the speed I was driving at, at that moment.

So that would make the problem in my case a display problem, not a data import problem... if we assume that the changelog documentation claiming the database is stored in meters per second is authoritative. When displaying a point (in a pop-up), the meters per second value is converted to kilometers per hour (should be miles per hour due to my settings), and that conversion is happening correctly, displaying 99 km/h, recovering the original value from OwnTracks. But when displaying the same data by hovering over the line (not clicking on a point), the speed value is not converted back from m/s to km/h by multiplying by 3.6 as it should be. In my case, with display in Miles, it is converting to mph by multiplying by the factor of 0.621371, as it should. The result is, instead of displaying the "mph" value as 27.5 or 99, or the correct 61 mph, it displays 17 mph (rounded from 16.944 mph). And of course in the table form (My data->Points), it's not being converted at all, and thus the claim of units being km/h makes the number off by a factor of 3.6.

This should be an easy fix one way or the other. Assuming the intent of the database storage is truly to use meters per second, I have now verified that the back-end data is correct on my system (consistent with those units), and that it is therefore just a display issue in several places. There may be data storage problems for some users, though. The changelog for 0.24.0 suggests that the intention was to store data in m/s internally due to most trackers reporting in m/s. But I haven't inspected the code to make sure that that's actually true.... that Dawarich is storing data from other trackers (besides OwnTracks) without scaling?

dvdbeattie avatar Jul 20 '25 18:07 dvdbeattie

I use the Dawarich iOS app. On the map it shows random speeds when I select a route. When I enable "points" on the map and then select a specific point it shows the correct speed in km/h. When I go to "points" the speed is NOT in km/h. At first I thought its random but I noticed that its actually not random and m/s instead of km/h so not what the column name actually says it is. As far as I know dawarich saves the speed always as m/s and then converts it to km/h. I'm not sure whats wrong here. The column name or if the conversion to km/h just doesn't happen.

What I still don't know is what the speed means when I hover over a route. Is it the average speed? I'm pretty sure it isn't tho because it would be way too low in my case.

binnichtaktiv avatar Oct 10 '25 10:10 binnichtaktiv

What I still don't know is what the speed means when I hover over a route. Is it the average speed? I'm pretty sure it isn't tho because it would be way too low in my case.

When you hover over a route, it's always a straight line between two points you're hovering on. The speed in the popup is basically the speed recorded between those two points.

Gotta remove it until it's fixed

Freika avatar Oct 11 '25 12:10 Freika