dawarich Importing GPSLogger exports

OS & Hardware Docker-based on Arch on Proxmox

Version 0.23.5, 0.23.3

Describe the bug Import of *.gpx & *.geojson from GPSLogger exports is not possible.

To Reproduce Steps to reproduce the behavior:

Go to 'Imports'
Click on 'Durchsuchen'
Select any GPSLogger exported geojson or gpx file to open
Press 'Create Import' 6a. With *.geojson, observe import failed message (see logs below) 6b. With *.gpx, nothing happens. On returning to imports page, notification "unexpected token at '<?xml version="1.0" encoding="UT'" shows up on screen top right.

Expected behavior Expected imports to work.

Logs Import Failed Log Message:

[Import failed](http://REDACTED/notifications/4)
4 minutes ago
Import "REDACTED.geojson" failed: undefined method `flat_map' for nil, stacktrace: /var/app/app/services/google_maps/semantic_history_parser.rb:42:in `parse_json' /var/app/app/services/google_maps/semantic_history_parser.rb:14:in `call' /var/app/app/services/imports/create.rb:12:in `call' /var/app/app/models/import.rb:17:in `process!' /var/app/app/jobs/import_job.rb:10:in `perform' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:68:in `block in _perform_job' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:120:in `block in run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/translation.rb:9:in `block (2 levels) in ' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `instance_exec' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `block in run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/core_ext/time/zones.rb:65:in `use_zone' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/timezones.rb:9:in `block (2 levels) in ' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `instance_exec' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `block in run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:140:in `run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:67:in `_perform_job' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:32:in `_perform_job' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:51:in `perform_now' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:26:in `block in perform_now' /var/app/vendor/bundle/ruby/3.3.0/gems/activerecord-8.0.1/lib/active_record/railties/job_runtime.rb:13:in `block in instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:40:in `block in instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `block in instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:39:in `instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activerecord-8.0.1/lib/active_record/railties/job_runtime.rb:11:in `instrument' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:26:in `perform_now' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/logging.rb:32:in `block in perform_now' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/logging.rb:41:in `tag_logger' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/logging.rb:32:in `perform_now' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:29:in `block in execute' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:120:in `block in run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/railtie.rb:95:in `block (4 levels) in ' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:77:in `block in wrap' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/execution_wrapper.rb:87:in `wrap' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:74:in `wrap' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/railtie.rb:94:in `block (3 levels) in ' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `instance_exec' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `block in run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:140:in `run_callbacks' /var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:27:in `execute' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/active_job/queue_adapters/sidekiq_adapter.rb:10:in `perform' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:220:in `execute_job' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:185:in `block (4 levels) in process' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:180:in `traverse' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:183:in `block in traverse' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job/interrupt_handler.rb:9:in `call' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:182:in `traverse' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:183:in `block in traverse' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/metrics/tracking.rb:26:in `track' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/metrics/tracking.rb:134:in `call' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:182:in `traverse' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:173:in `invoke' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:184:in `block (3 levels) in process' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_retry.rb:118:in `local' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/rails.rb:16:in `block in call' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:77:in `block in wrap' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/execution_wrapper.rb:91:in `wrap' /var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:74:in `wrap' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/rails.rb:15:in `call' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:281:in `stats' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_logger.rb:15:in `call' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_retry.rb:85:in `global' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:132:in `block in dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_logger.rb:40:in `prepare' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:131:in `dispatch' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:183:in `block (2 levels) in process' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:182:in `handle_interrupt' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:182:in `block in process' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:181:in `handle_interrupt' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:181:in `process' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:86:in `process_one' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:76:in `run' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/component.rb:10:in `watchdog' /var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/component.rb:19:in `block in safe_thread'
Please, when reporting a bug to [Github Issues](https://github.com/Freika/dawarich/issues), don't forget to include logs from dawarich_app and dawarich_sidekiq docker containers. Thank you!

app:

D, [2025-01-25T10:02:27.728312 #126] DEBUG -- :   Import Create (4.5ms)  INSERT INTO "imports" ("name", "user_id", "source", "created_at", "updated_at", "raw_points", "doubles", "processed", "raw_data", "points_count") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) RETURNING "id"  [["name", "REDACTED.geojson"], ["user_id", 1], ["source", 0], ["created_at", "2025-01-25 10:02:27.633975"], ["updated_at", "2025-01-25 10:02:27.633975"], ["raw_points", 0], ["doubles", 0], ["processed", 0], ["raw_data", nil], ["points_count", 0]]

D, [2025-01-25T10:02:27.729504 #126] DEBUG -- :   ↳ app/controllers/imports_controller.rb:26:in `block in create'

D, [2025-01-25T10:02:27.797771 #126] DEBUG -- :   TRANSACTION (56.7ms)  COMMIT

D, [2025-01-25T10:02:27.798890 #126] DEBUG -- :   ↳ app/controllers/imports_controller.rb:26:in `block in create'

D, [2025-01-25T10:02:28.218711 #126] DEBUG -- :   TRANSACTION (6.2ms)  BEGIN

D, [2025-01-25T10:02:28.220048 #126] DEBUG -- :   ↳ app/controllers/imports_controller.rb:40:in `block in create'

D, [2025-01-25T10:02:28.283208 #126] DEBUG -- :   Import Update (54.5ms)  UPDATE "imports" SET "updated_at" = $1, "raw_data" = $2 WHERE "imports"."id" = $3  [["updated_at", "2025-01-25 10:02:28.112112"], ["raw_data", REDACTED], ["id", 35]]

D, [2025-01-25T10:02:28.306083 #126] DEBUG -- :   ↳ app/controllers/imports_controller.rb:40:in `block in create'

D, [2025-01-25T10:02:28.340199 #126] DEBUG -- :   TRANSACTION (15.1ms)  COMMIT

D, [2025-01-25T10:02:28.340958 #126] DEBUG -- :   ↳ app/controllers/imports_controller.rb:40:in `block in create'

I, [2025-01-25T10:02:28.391053 #126]  INFO -- : Enqueued ImportJob (Job ID: d6a80e9a-2a7c-42a3-934c-21fd904e9204) to Sidekiq(imports) with arguments: 1, 35

I, [2025-01-25T10:02:28.392017 #126]  INFO -- : ↳ app/controllers/imports_controller.rb:44:in `block in create'

I, [2025-01-25T10:02:28.404985 #126]  INFO -- : {"method":"POST","path":"/imports","format":"turbo_stream","controller":"ImportsController","action":"create","status":303,"allocations":85719,"duration":806.77,"view":0.0,"db":138.44,"location":"http://REDACTED/imports"}

D, [2025-01-25T10:02:28.492757 #126] DEBUG -- :   User Load (0.7ms)  SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2  [["id", 1], ["LIMIT", 1]]

D, [2025-01-25T10:02:28.493665 #126] DEBUG -- :   ↳ app/controllers/application_controller.rb:11:in `unread_notifications'

D, [2025-01-25T10:02:28.509975 #126] DEBUG -- :   Rendering layout layouts/application.html.erb

D, [2025-01-25T10:02:28.510176 #126] DEBUG -- :   Rendering imports/index.html.erb within layouts/application

D, [2025-01-25T10:02:28.593567 #126] DEBUG -- :   Import Exists? (61.4ms)  SELECT 1 AS one FROM "imports" WHERE "imports"."user_id" = $1 LIMIT $2 OFFSET $3  [["user_id", 1], ["LIMIT", 1], ["OFFSET", 0]]

D, [2025-01-25T10:02:28.594391 #126] DEBUG -- :   ↳ app/views/imports/index.html.erb:21

D, [2025-01-25T10:02:28.608778 #126] DEBUG -- :   Import Count (0.7ms)  SELECT COUNT(*) FROM "imports" WHERE "imports"."user_id" = $1  [["user_id", 1]]

D, [2025-01-25T10:02:28.609687 #126] DEBUG -- :   ↳ app/views/imports/index.html.erb:35

D, [2025-01-25T10:02:28.624345 #126] DEBUG -- :   Import Load (1.1ms)  SELECT "imports"."id", "imports"."name", "imports"."source", "imports"."created_at", "imports"."points_count" FROM "imports" WHERE "imports"."user_id" = $1 ORDER BY "imports"."created_at" DESC LIMIT $2 OFFSET $3  [["user_id", 1], ["LIMIT", 25], ["OFFSET", 0]]

D, [2025-01-25T10:02:28.626524 #126] DEBUG -- :   ↳ app/views/imports/index.html.erb:53

D, [2025-01-25T10:02:28.693377 #126] DEBUG -- :   Point Count (0.9ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 35]]

D, [2025-01-25T10:02:28.694208 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.731966 #126] DEBUG -- :   CACHE Point Count (0.1ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 35]]

D, [2025-01-25T10:02:28.732868 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.797100 #126] DEBUG -- :   Point Count (1.1ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1 AND "points"."reverse_geocoded_at" IS NOT NULL  [["import_id", 35]]

D, [2025-01-25T10:02:28.797972 #126] DEBUG -- :   ↳ app/models/import.rb:21:in `reverse_geocoded_points_count'

D, [2025-01-25T10:02:28.807453 #126] DEBUG -- :   Point Count (0.9ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 33]]

D, [2025-01-25T10:02:28.808507 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.821084 #126] DEBUG -- :   CACHE Point Count (0.0ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 33]]

D, [2025-01-25T10:02:28.821985 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.826896 #126] DEBUG -- :   Point Count (1.1ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1 AND "points"."reverse_geocoded_at" IS NOT NULL  [["import_id", 33]]

D, [2025-01-25T10:02:28.827764 #126] DEBUG -- :   ↳ app/models/import.rb:21:in `reverse_geocoded_points_count'

D, [2025-01-25T10:02:28.837515 #126] DEBUG -- :   Point Count (1.0ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 7]]

D, [2025-01-25T10:02:28.838590 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.896835 #126] DEBUG -- :   CACHE Point Count (0.0ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 7]]

D, [2025-01-25T10:02:28.897674 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.911267 #126] DEBUG -- :   Point Count (1.1ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1 AND "points"."reverse_geocoded_at" IS NOT NULL  [["import_id", 7]]

D, [2025-01-25T10:02:28.912272 #126] DEBUG -- :   ↳ app/models/import.rb:21:in `reverse_geocoded_points_count'

D, [2025-01-25T10:02:28.921801 #126] DEBUG -- :   Point Count (1.0ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 2]]

D, [2025-01-25T10:02:28.922682 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:28.928386 #126] DEBUG -- :   CACHE Point Count (0.1ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 2]]

D, [2025-01-25T10:02:28.929150 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:29.033321 #126] DEBUG -- :   Point Count (1.0ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1 AND "points"."reverse_geocoded_at" IS NOT NULL  [["import_id", 2]]

D, [2025-01-25T10:02:29.034218 #126] DEBUG -- :   ↳ app/models/import.rb:21:in `reverse_geocoded_points_count'

D, [2025-01-25T10:02:29.192929 #126] DEBUG -- :   Point Count (0.9ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 1]]

D, [2025-01-25T10:02:29.193855 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:29.195830 #126] DEBUG -- :   CACHE Point Count (0.0ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1  [["import_id", 1]]

D, [2025-01-25T10:02:29.196711 #126] DEBUG -- :   ↳ app/models/import.rb:7:in `points_count'

D, [2025-01-25T10:02:29.201524 #126] DEBUG -- :   Point Count (0.9ms)  SELECT COUNT(*) FROM "points" WHERE "points"."import_id" = $1 AND "points"."reverse_geocoded_at" IS NOT NULL  [["import_id", 1]]

D, [2025-01-25T10:02:29.202541 #126] DEBUG -- :   ↳ app/models/import.rb:21:in `reverse_geocoded_points_count'

D, [2025-01-25T10:02:29.291566 #126] DEBUG -- :   Notification Load (1.0ms)  SELECT "notifications".* FROM "notifications" WHERE "notifications"."user_id" = $1 AND "notifications"."read_at" IS NULL ORDER BY "notifications"."created_at" DESC  [["user_id", 1]]

D, [2025-01-25T10:02:29.292457 #126] DEBUG -- :   ↳ app/views/shared/_navbar.html.erb:72

I, [2025-01-25T10:02:29.309028 #126]  INFO -- : {"method":"GET","path":"/imports","format":"turbo_stream","controller":"ImportsController","action":"index","status":200,"allocations":26408,"duration":862.35,"view":720.52,"db":74.34}

sidekiq:


I, [2025-01-25T10:02:28.357721 #8]  INFO -- : start

I, [2025-01-25T10:02:28.383117 #8]  INFO -- : Performing ImportJob (Job ID: d6a80e9a-2a7c-42a3-934c-21fd904e9204) from Sidekiq(imports) enqueued at 2025-01-25T10:02:28.345709164Z with arguments: 1, 35

D, [2025-01-25T10:02:28.389750 #8] DEBUG -- :   User Load (1.5ms)  SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]

D, [2025-01-25T10:02:28.391007 #8] DEBUG -- :   ↳ app/jobs/import_job.rb:7:in `perform'

D, [2025-01-25T10:02:28.418522 #8] DEBUG -- :   Import Load (23.1ms)  SELECT "imports".* FROM "imports" WHERE "imports"."user_id" = $1 AND "imports"."id" = $2 LIMIT $3  [["user_id", 1], ["id", 35], ["LIMIT", 1]]

D, [2025-01-25T10:02:28.419658 #8] DEBUG -- :   ↳ app/jobs/import_job.rb:8:in `perform'

D, [2025-01-25T10:02:28.460866 #8] DEBUG -- :   TRANSACTION (1.4ms)  BEGIN

D, [2025-01-25T10:02:28.462548 #8] DEBUG -- :   ↳ app/services/notifications/create.rb:14:in `call'

D, [2025-01-25T10:02:28.466600 #8] DEBUG -- :   Notification Create (10.6ms)  INSERT INTO "notifications" ("title", "content", "user_id", "kind", "read_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id"  [["title", "Import failed"], ["content", "Import \"REDACTED.geojson\" failed: undefined method `flat_map' for nil, stacktrace: /var/app/app/services/google_maps/semantic_history_parser.rb:42:in `parse_json'\n/var/app/app/services/google_maps/semantic_history_parser.rb:14:in `call'\n/var/app/app/services/imports/create.rb:12:in `call'\n/var/app/app/models/import.rb:17:in `process!'\n/var/app/app/jobs/import_job.rb:10:in `perform'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:68:in `block in _perform_job'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:120:in `block in run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `instance_exec'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `block in run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `instance_exec'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `block in run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:140:in `run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:67:in `_perform_job'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:32:in `_perform_job'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:51:in `perform_now'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:26:in `block in perform_now'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activerecord-8.0.1/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:40:in `block in instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `block in instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/notifications.rb:210:in `instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:39:in `instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activerecord-8.0.1/lib/active_record/railties/job_runtime.rb:11:in `instrument'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/instrumentation.rb:26:in `perform_now'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/logging.rb:32:in `block in perform_now'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/logging.rb:41:in `tag_logger'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/logging.rb:32:in `perform_now'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:29:in `block in execute'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:120:in `block in run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/railtie.rb:95:in `block (4 levels) in <class:Railtie>'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:77:in `block in wrap'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/execution_wrapper.rb:87:in `wrap'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:74:in `wrap'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/railtie.rb:94:in `block (3 levels) in <class:Railtie>'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `instance_exec'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:129:in `block in run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/callbacks.rb:140:in `run_callbacks'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activejob-8.0.1/lib/active_job/execution.rb:27:in `execute'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/active_job/queue_adapters/sidekiq_adapter.rb:10:in `perform'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:220:in `execute_job'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:180:in `traverse'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job/interrupt_handler.rb:9:in `call'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:182:in `traverse'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/metrics/tracking.rb:26:in `track'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/metrics/tracking.rb:134:in `call'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:182:in `traverse'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/middleware/chain.rb:173:in `invoke'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_retry.rb:118:in `local'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/rails.rb:16:in `block in call'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:77:in `block in wrap'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/execution_wrapper.rb:91:in `wrap'\n/var/app/vendor/bundle/ruby/3.3.0/gems/activesupport-8.0.1/lib/active_support/reloader.rb:74:in `wrap'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/rails.rb:15:in `call'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:281:in `stats'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_logger.rb:15:in `call'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_retry.rb:85:in `global'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:132:in `block in dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/job_logger.rb:40:in `prepare'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:131:in `dispatch'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:182:in `handle_interrupt'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:182:in `block in process'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:181:in `handle_interrupt'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:181:in `process'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:86:in `process_one'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/processor.rb:76:in `run'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/component.rb:10:in `watchdog'\n/var/app/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.7/lib/sidekiq/component.rb:19:in `block in safe_thread'"], ["user_id", 1], ["kind", 2], ["read_at", nil], ["created_at", "2025-01-25 10:02:28.451567"], ["updated_at", "2025-01-25 10:02:28.451567"]]

D, [2025-01-25T10:02:28.468573 #8] DEBUG -- :   ↳ app/services/notifications/create.rb:14:in `call'

D, [2025-01-25T10:02:28.513967 #8] DEBUG -- :   TRANSACTION (5.3ms)  COMMIT

D, [2025-01-25T10:02:28.515005 #8] DEBUG -- :   ↳ app/services/notifications/create.rb:14:in `call'

I, [2025-01-25T10:02:28.522471 #8]  INFO -- : Performed ImportJob (Job ID: d6a80e9a-2a7c-42a3-934c-21fd904e9204) from Sidekiq(imports) in 141.18ms

I, [2025-01-25T10:02:28.524111 #8]  INFO -- : done

D, [2025-01-25T10:02:34.554809 #8] DEBUG -- : Flushed 2 metrics

Additional context I have about 10 years worth of GPSLogger exports that I would like to import.

Actual geo data redacted.

Jan 25 '25 10:01 PhilippMundhenk

@PhilippMundhenk feel free to send me a sample file so I could fix importing process, either on email in my profile or in discord dms (link in repo readme)

Jan 25 '25 11:01 Freika

I created abstracted files that I am comfortable with sharing and that fails the import.

test.geojson:

{"type": "FeatureCollection","features": [
{"type": "Feature","properties":{"time":"2025-01-24T06:19:29.726Z","provider":"network","time_long":1737699569726,"accuracy":16.258,"altitude":150.50001525878906},"geometry":{"type":"Point","coordinates":[7.1123456,32.1234567]}}
,{"type": "Feature","properties":{"time":"2025-01-24T06:25:34.379Z","provider":"network","time_long":1737699934379,"accuracy":37.324,"altitude":152.20001220703125},"geometry":{"type":"Point","coordinates":[7.1234567,32.2345678]}}
,{"type": "Feature","properties":{"time":"2025-01-24T06:27:36.544Z","provider":"network","time_long":1737700056544,"accuracy":30.607,"altitude":154.20001220703125},"geometry":{"type":"Point","coordinates":[7.1345678,32.3456789]}}
,{"type": "Feature","properties":{"time":"2025-01-24T06:28:38.141Z","provider":"network","time_long":1737700118141,"accuracy":29.432,"altitude":156.8000183105469},"geometry":{"type":"Point","coordinates":[7.145678,32.123456]}}
,{"type": "Feature","properties":{"time":"2025-01-24T06:29:50.184Z","provider":"gps","time_long":1737700190184,"accuracy":24.0,"altitude":158.3183745365673,"bearing":290.3627,"speed":13.519987},"geometry":{"type":"Point","coordinates":[7.112345678901234,32.12345678901234]}}
,{"type": "Feature","properties":{"time":"2025-01-24T06:29:51.184Z","provider":"gps","time_long":1737700191184,"accuracy":32.0,"altitude":160.6891025346219,"speed":0.0},"geometry":{"type":"Point","coordinates":[7.123456789012345,32.23456789012345]}}
,{"type": "Feature","properties":{"time":"2025-01-24T06:29:52.184Z","provider":"gps","time_long":1737700192184,"accuracy":24.0,"altitude":170.24735632104444,"speed":0.0},"geometry":{"type":"Point","coordinates":[7.134567890123456,32.12345678901234]}}
]}

test.gpx:

<?xml version="1.0" encoding="UTF-8" ?><gpx version="1.0" creator="GPSLogger 106 - http://gpslogger.mendhak.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.topografix.com/GPX/1/0" xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd"><time>2025-01-24T06:19:29.726Z</time><trk><trkseg><trkpt lat="32.1234567" lon="7.1123456"><ele>150.50001525878906</ele><time>2025-01-24T06:19:29.726Z</time><src>network</src></trkpt>
<trkpt lat="32.2345678" lon="7.1234567"><ele>152.20001220703125</ele><time>2025-01-24T06:25:34.379Z</time><src>network</src></trkpt>
<trkpt lat="32.3456789" lon="7.1345678"><ele>154.20001220703125</ele><time>2025-01-24T06:27:36.544Z</time><src>network</src></trkpt>
<trkpt lat="32.123456" lon="7.145678"><ele>156.8000183105469</ele><time>2025-01-24T06:28:38.141Z</time><src>network</src></trkpt>
<trkpt lat="32.12345678901234" lon="7.112345678901234"><ele>158.3183745365673</ele><time>2025-01-24T06:29:50.184Z</time><course>290.3627</course><speed>13.519987</speed><src>gps</src></trkpt>
<trkpt lat="32.23456789012345" lon="7.123456789012345"><ele>160.6891025346219</ele><time>2025-01-24T06:29:51.184Z</time><speed>0.0</speed><src>gps</src></trkpt>
<trkpt lat="32.12345678901234" lon="7.134567890123456"><ele>170.24735632104444</ele><time>2025-01-24T06:29:52.184Z</time><speed>0.0</speed><src>gps</src></trkpt>
</trkseg></trk></gpx>

Jan 25 '25 14:01 PhilippMundhenk

And here is a screenshot of the gpx failure. Note that this is not logged in the notifications, but only this notification shows up and vanishes after some time:

Jan 25 '25 14:01 PhilippMundhenk

@mbirth Please open a different issue

Feb 02 '25 18:02 Freika

@mbirth Please open a different issue

But it seems I'm hitting the exact same issue?

Handling issues with importing two different file formats is more convenient when I have separate issue with details for each

Feb 02 '25 18:02 Freika

I created abstracted files that I am comfortable with sharing and that fails the import.

test.geojson:

{"type": "FeatureCollection","features": [ {"type": "Feature","properties":{"time":"2025-01-24T06:19:29.726Z","provider":"network","time_long":1737699569726,"accuracy":16.258,"altitude":150.50001525878906},"geometry":{"type":"Point","coordinates":[7.1123456,32.1234567]}} ,{"type": "Feature","properties":{"time":"2025-01-24T06:25:34.379Z","provider":"network","time_long":1737699934379,"accuracy":37.324,"altitude":152.20001220703125},"geometry":{"type":"Point","coordinates":[7.1234567,32.2345678]}} ,{"type": "Feature","properties":{"time":"2025-01-24T06:27:36.544Z","provider":"network","time_long":1737700056544,"accuracy":30.607,"altitude":154.20001220703125},"geometry":{"type":"Point","coordinates":[7.1345678,32.3456789]}} ,{"type": "Feature","properties":{"time":"2025-01-24T06:28:38.141Z","provider":"network","time_long":1737700118141,"accuracy":29.432,"altitude":156.8000183105469},"geometry":{"type":"Point","coordinates":[7.145678,32.123456]}} ,{"type": "Feature","properties":{"time":"2025-01-24T06:29:50.184Z","provider":"gps","time_long":1737700190184,"accuracy":24.0,"altitude":158.3183745365673,"bearing":290.3627,"speed":13.519987},"geometry":{"type":"Point","coordinates":[7.112345678901234,32.12345678901234]}} ,{"type": "Feature","properties":{"time":"2025-01-24T06:29:51.184Z","provider":"gps","time_long":1737700191184,"accuracy":32.0,"altitude":160.6891025346219,"speed":0.0},"geometry":{"type":"Point","coordinates":[7.123456789012345,32.23456789012345]}} ,{"type": "Feature","properties":{"time":"2025-01-24T06:29:52.184Z","provider":"gps","time_long":1737700192184,"accuracy":24.0,"altitude":170.24735632104444,"speed":0.0},"geometry":{"type":"Point","coordinates":[7.134567890123456,32.12345678901234]}} ]}

test.gpx:
2025-01-24T06:19:29.726Z150.500015258789062025-01-24T06:19:29.726Znetwork
152.200012207031252025-01-24T06:25:34.379Znetwork 154.200012207031252025-01-24T06:27:36.544Znetwork 156.80001831054692025-01-24T06:28:38.141Znetwork 158.31837453656732025-01-24T06:29:50.184Z290.362713.519987gps 160.68910253462192025-01-24T06:29:51.184Z0.0gps 170.247356321044442025-01-24T06:29:52.184Z0.0gps

7 points out of 7 provided were created successfully. Let me know if it's still an issue

Jul 26 '25 11:07 Freika