undefined method 'y' for nil after updating points

[raphpa]

I updated to 0.25.2 from 0.24.1 and ran into the undefined method 'y' for nil issue. I have tried updating the points according to the mentioned FAQ.

First I had to raise to container limits to 10G of RAM since the process was killed while loading in all the points. Now the update seems to run and finish just fine, but the error still comes up.

D, [2025-03-22T11:31:00.107382 #169] DEBUG -- :   User Load (3.6ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1  [["LIMIT", 1000]]
D, [2025-03-22T11:33:03.171518 #169] DEBUG -- :   Point Load (122789.4ms)  SELECT "points".* FROM "points" WHERE "points"."user_id" = $1  [["user_id", 1]]
D, [2025-03-22T11:35:56.213695 #169] DEBUG -- :   Point Update All (6494.6ms)  UPDATE "points" SET lonlat = ST_SetSRID(ST_MakePoint(longitude, latitude), 4326) WHERE "points"."user_id" = $1 AND "points"."lonlat" IS NULL  [["user_id", 1]]
=> nil
[2] pry(main)>

It might still run out of memory since I see it taking up pretty much all of the 10G. Is there any way to process only some points at a time in a loop?

[jjjonesjr33]

Having the same issue undefined method 'y' for nil

Extracted source (around line #11):

    @coordinates =
      @points.pluck(:lonlat, :battery, :altitude, :timestamp, :velocity, :id, :country)
             .map { |lonlat, *rest| [lonlat.y, lonlat.x, *rest.map(&:to_s)] }
    @distance = distance
    @start_at = Time.zone.at(start_at)
    @end_at = Time.zone.at(end_at)

[raphpa]

Got it actually resolved. There was one point not being updated for some reason.

pry(main)> user = User.find_by(email: '[email protected]')
pry(main)> points = user.tracked_points.where(lonlat: nil)
pry(main)> points.size

D, [2025-03-22T11:58:51.190191 #68] DEBUG -- :   Point Count (1.4ms)  SELECT COUNT(*) FROM "points" WHERE "points"."user_id" = $1 AND "points"."lonlat" IS NULL  [["user_id", 1]]
=> 1

The point was imported from the owntracks api endpoint, it has latitude: nil and longitude: nil in the data, but the raw data includes the correct lat and lon.

After dropping that point from the database it works again.

pry(main)> points.each(&:destroy)

However this seems to be a condition where automatic upgrading still fails.

Also the manual way does not really work well with a larger number of points (about 3M).

[ragnarkarlsson]

Edit: Ran the script on release notes and it seems it failed again, I don't remember now but think I restored DB which likely caused this. Using the PSQL in #948 and rerunning the script as per 0.25.0 has resolved my issue.

I on the other hand have 293966 points with lonlat: nil> imported from Google Maps Phone Timeline Export which contain the data in LatLng attribute of raw_data

This is on docker build 0.25.3 updated a few minutes ago.

[solderdot72]

Just upgraded from 0.24.1 to 0.25.3 and see that message. I got the impression that the migration provided with 0.25.2 and the rake fix from 0.25.3 should resolve all problems. Apparently my understanding is not correct.

I'm running Dawarich under Portainer. Where in Portainer can I open a shell to enter those commands? And in which container should I open the shell to enter the commands?

Right now I am restoring my 0.24.1 installation from a backup. So if an upgrade path that is suitable for the masses is supposed to be rolled out with one of the next releases, I'll gladly wait for that.

[thor-ondir]

Got it actually resolved. There was one point not being updated for some reason.

The point was imported from the owntracks api endpoint, it has latitude: nil and longitude: nil in the data, but the raw data includes the correct lat and lon.

After dropping that point from the database it works again.

I didn't want to throw away the data, so here's what I did: Get into the database and find out which points don't have latitude or longitude:

SELECT COUNT(1) FROM Points WHERE latitude IS NULL or longitude IS NULL;

For me that showed more than 2k Points, for some reason. The points did have all the data in raw_data:

SELECT id, raw_data->>'lon' as raw_longitude, raw_data->>'lat' AS raw_latitude FROM Points WHERE latitude IS NULL OR longitude IS NULL;

If that looks about right, you can backfill the Points from that raw data with

UPDATE Points SET latitude=subq.lat, longitude=subq.lon FROM (SELECT CAST(raw_data->>'lat' AS NUMERIC(10,6)) AS lat, CAST(raw_data->>'lon' AS NUMERIC(10,6)) AS lon, id FROM Points WHERE latitude IS NULL OR longitude IS NULL) AS subq WHERE Points.id = subq.id;

Then fix the lonlat issue with bin/rake points:migrate_to_lonlat, and you should be good to go.

[salzig]

I'm running Dawarich under Portainer. Where in Portainer can I open a shell to enter those commands? And in which container should I open the shell to enter the commands?

you need to start bundle exec rails console in any of sidekiq (dawarich_sidekiq) or app (dawarich_app) to get an interactive ruby/rails console.