dawarich Points created by the Home Assistant Integration don't get reverse geocoded unless triggered manually

OS & Hardware Proxmox VM

Version 0.26.3

Describe the bug Points that were created by the Home Assistant integration https://github.com/AlbinLind/dawarich-home-assistant do not get geocode information after they were created. For poits created by owntracks it does work, so it is not a problem of reverse geocodeing itself, but somehow on the trigger of it. If you "continue reverse geocoding" in the background jobs settings, all points including the one from the HA integration get reverse geocoded.

To Reproduce Steps to reproduce the behavior:

Install Home Assistant integration
Configure a location tracker for it and wait that it sends a datapoint to Dawarich (can be confirmed in the log of the dawarich_app container)
Look up the point in the DB and find that the reverse geocoding did not start.

Expected behavior All points, including the one from HA, get reverse geocoded after creation.

Screenshots

Logs no error can be found

Additional context I'm not sure if this is an issue of dawarich or the HA Integration, so I will report the issue there as well.

May 20 '25 13:05 noway42

For context, we are using POST on the /api/v1/points endpoint if that helps with debugging.

May 20 '25 14:05 AlbinLind

Please provide logs and your compose file

May 20 '25 17:05 Freika

Example from today:

Point created at 8:36 (or 6.36 w/o timezone):

Log dawarich_app:

D, [2025-05-21T06:36:04.293901 #126] DEBUG -- :   ↳ app/controllers/api_controller.rb:22:in 'ApiController#current_api_user'
D, [2025-05-21T06:36:04.321222 #126] DEBUG -- :   Point Upsert (3.6ms)  INSERT INTO "points" ("lonlat","battery_status","battery","timestamp","altitude","tracker_id","velocity","ssid","accuracy","vertical_accuracy","course_accuracy","course","raw_data","user_id","created_at","updated_at") VALUES ('**sensitive data removed**, NULL, 1747809364, 138, 'Dawarich', '0', 'unknown', 100, 100, 0.0, 0.0, '{"type":"Feature","geometry":{"type":"Point","coordinates":[**sensitive data removed**]},"properties":{"timestamp":"2025-05-21T08:36:04.256291+02:00","altitude":138.0,"speed":0,"horizontal_accuracy":100,"vertical_accuracy":100,"significant_change":"unknown","device_id":"Dawarich","wifi":"unknown","battery_state":"unknown","battery_level":0,"course":0,"course_accuracy":0}}', 1, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) ON CONFLICT ("lonlat","timestamp","user_id") DO UPDATE SET updated_at=(CASE WHEN ("points"."battery_status" IS NOT DISTINCT FROM excluded."battery_status" AND "points"."battery" IS NOT DISTINCT FROM excluded."battery" AND "points"."altitude" IS NOT DISTINCT FROM excluded."altitude" AND "points"."tracker_id" IS NOT DISTINCT FROM excluded."tracker_id" AND "points"."velocity" IS NOT DISTINCT FROM excluded."velocity" AND "points"."ssid" IS NOT DISTINCT FROM excluded."ssid" AND "points"."accuracy" IS NOT DISTINCT FROM excluded."accuracy" AND "points"."vertical_accuracy" IS NOT DISTINCT FROM excluded."vertical_accuracy" AND "points"."course_accuracy" IS NOT DISTINCT FROM excluded."course_accuracy" AND "points"."course" IS NOT DISTINCT FROM excluded."course" AND "points"."raw_data" IS NOT DISTINCT FROM excluded."raw_data") THEN "points".updated_at ELSE CURRENT_TIMESTAMP END),"battery_status"=excluded."battery_status","battery"=excluded."battery","altitude"=excluded."altitude","tracker_id"=excluded."tracker_id","velocity"=excluded."velocity","ssid"=excluded."ssid","accuracy"=excluded."accuracy","vertical_accuracy"=excluded."vertical_accuracy","course_accuracy"=excluded."course_accuracy","course"=excluded."course","raw_data"=excluded."raw_data" RETURNING id, timestamp, ST_X(lonlat::geometry) AS longitude, ST_Y(lonlat::geometry) AS latitude
D, [2025-05-21T06:36:04.321629 #126] DEBUG -- :   ↳ app/services/points/create.rb:21:in 'block in Points::Create#call'
I, [2025-05-21T06:36:04.322915 #126]  INFO -- : {"method":"POST","path":"/api/v1/points","format":"json","controller":"Api::V1::PointsController","action":"create","status":200,"allocations":4117,"duration":31.39,"view":0.16,"db":13.92,"unpermitted_params":["point"]}
D, [2025-05-21T06:36:04.766999 #126] DEBUG -- :   User Load (0.3ms)  SELECT "users".* FROM "users" WHERE "users"."api_key" IS NULL LIMIT $1  [["LIMIT", 1]]
D, [2025-05-21T06:36:04.767535 #126] DEBUG -- :   ↳ app/controllers/api_controller.rb:22:in 'ApiController#current_api_user'
I, [2025-05-21T06:36:04.767836 #126]  INFO -- : {"method":"GET","path":"/api/v1/health","format":"*/*","controller":"Api::V1::HealthController","action":"index","status":200,"allocations":903,"duration":1.94,"view":0.08,"db":0.26}

Log dawarich_sidekiq

I, [2025-05-21T06:19:58.626408 #8]  INFO -- : start
I, [2025-05-21T06:19:58.634666 #8]  INFO -- : Performing Cache::PreheatingJob (Job ID: d54492cc-a261-47ba-8b4f-1337a971331f) from Sidekiq(default) enqueued at 2025-05-21T06:19:58.305203574Z
D, [2025-05-21T06:19:58.637518 #8] DEBUG -- :   User Load (0.4ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1  [["LIMIT", 1000]]
D, [2025-05-21T06:19:58.638066 #8] DEBUG -- :   ↳ app/jobs/cache/preheating_job.rb:7:in 'Cache::PreheatingJob#perform'
D, [2025-05-21T06:19:58.714705 #8] DEBUG -- :    (73.3ms)          SELECT DISTINCT
          EXTRACT(YEAR FROM TO_TIMESTAMP(timestamp)) AS year,
          TO_CHAR(TO_TIMESTAMP(timestamp), 'Mon') AS month
        FROM points
        WHERE user_id = 1
        ORDER BY year DESC, month ASC
D, [2025-05-21T06:19:58.715741 #8] DEBUG -- :   ↳ app/models/user.rb:92:in 'block in User#years_tracked'
I, [2025-05-21T06:19:58.720497 #8]  INFO -- : Performed Cache::PreheatingJob (Job ID: d54492cc-a261-47ba-8b4f-1337a971331f) from Sidekiq(default) in 85.88ms
I, [2025-05-21T06:19:58.721374 #8]  INFO -- : done
D, [2025-05-21T06:20:00.778727 #8] DEBUG -- : Flushed 4 metrics
I, [2025-05-21T06:36:20.780127 #8]  INFO -- : start
I, [2025-05-21T06:36:20.797723 #8]  INFO -- : Performing Owntracks::PointCreatingJob (Job ID: 83f35c3f-53f4-4188-a32c-631ee0ec388f) from Sidekiq(default) enqueued at 2025-05-21T06:36:20.774681136Z with arguments: {"_type" => "location", "_id" => "592c7c61", "acc" => 4, "alt" => 134, "batt" => 96, "bs" => 1, "cog" => 34, "conn" => "m", "created_at" => 1747809381, "lat" => **sensitive data removed**, "lon" => **sensitive data removed**, "m" => 1, "t" => "p", "tid" => "nx", "topic" => "owntracks/user/**sensitive data removed**", "tst" => 1747806765, "vac" => 1, "vel" => 55, "api_key" => "**sensitive data removed**", "controller" => "api/v1/owntracks/points", "action" => "create", "point" => {"topic" => "owntracks/user/**sensitive data removed**", "created_at" => 1747809381}}, 1
D, [2025-05-21T06:36:20.855975 #8] DEBUG -- :   Point Exists? (2.5ms)  SELECT 1 AS one FROM "points" WHERE "points"."lonlat" = $1 AND "points"."timestamp" = $2 AND "points"."user_id" = $3 LIMIT $4  [["lonlat", "[FILTERED]"], ["timestamp", 1747806765], ["user_id", 1], ["LIMIT", 1]]
D, [2025-05-21T06:36:20.856462 #8] DEBUG -- :   ↳ app/models/concerns/point_validation.rb:11:in 'PointValidation#point_exists?'
I, [2025-05-21T06:36:20.856768 #8]  INFO -- : Performed Owntracks::PointCreatingJob (Job ID: 83f35c3f-53f4-4188-a32c-631ee0ec388f) from Sidekiq(default) in 59.21ms
I, [2025-05-21T06:36:20.857558 #8]  INFO -- : done
D, [2025-05-21T06:36:21.043912 #8] DEBUG -- : Flushed 2 metrics

compose file:

networks:
  dawarich:
services:
  dawarich_redis:
    image: redis:7.0-alpine
    container_name: dawarich_redis
    command: redis-server
    networks:
      - dawarich
    volumes:
      - /share/data/dawarich/dawarich_shared:/data
    restart: always
    healthcheck:
      test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]
      interval: 10s
      retries: 5
      start_period: 30s
      timeout: 10s

  dawarich_db:
    image: postgis/postgis:17-3.5-alpine
    shm_size: 1G
    container_name: dawarich_db
    volumes:
    - /share/data/dawarich/dawarich_db_data:/var/lib/postgresql/data
    - /share/data/dawarich/dawarich_shared:/var/shared
    networks:
      - dawarich
    environment:
      POSTGRES_USER: **sensitive data removed**
      POSTGRES_PASSWORD: **sensitive data removed**
    restart: always
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres -d dawarich_development" ]
      interval: 10s
      retries: 5
      start_period: 30s
      timeout: 10s

  dawarich_app:
    image: freikin/dawarich:latest
    container_name: dawarich_app
    volumes:
      - /share/data/dawarich/dawarich_public:/var/app/public
      - /share/data/dawarich/dawarich_watched:/var/app/tmp/imports/watched
      - /share/data/dawarich/dawarich_storage:/var/app/storage
    networks:
      - dawarich
    ports:
      - 3033:3000
    stdin_open: true
    tty: true
    entrypoint: web-entrypoint.sh
    command: ['bin/rails', 'server', '-p', '3000', '-b', '::']
    restart: on-failure
    environment:
      RAILS_ENV: development
      REDIS_URL: redis://dawarich_redis:6379/0
      DATABASE_HOST: dawarich_db
      DATABASE_USERNAME: **sensitive data removed**
      DATABASE_PASSWORD: **sensitive data removed**
      DATABASE_NAME: dawarich_development
      MIN_MINUTES_SPENT_IN_CITY: 60
      APPLICATION_HOST: localhost
      APPLICATION_HOSTS: **sensitive data removed**
      TIME_ZONE: Europe/Berlin
      APPLICATION_PROTOCOL: http
      NOMINATIM_API_HOST: nominatim.openstreetmap.org
      NOMINATIM_API_USE_HTTPS: true
      DISTANCE_UNIT: km
      PROMETHEUS_EXPORTER_ENABLED: false
      PROMETHEUS_EXPORTER_HOST: 0.0.0.0
      PROMETHEUS_EXPORTER_PORT: 9394
      ENABLE_TELEMETRY: false # More on telemetry: https://dawarich.app/docs/tutorials/telemetry
      SELF_HOSTED: "true"
      SIDEKIQ_USERNAME: **sensitive data removed**
      SIDEKIQ_PASSWORD: **sensitive data removed**
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "5"
    healthcheck:
      test: [ "CMD-SHELL", "wget -qO - http://127.0.0.1:3000/api/v1/health | grep -q '\"status\"\\s*:\\s*\"ok\"'" ]
      interval: 10s
      retries: 30
      start_period: 30s
      timeout: 10s
    depends_on:
      dawarich_db:
        condition: service_healthy
        restart: true
      dawarich_redis:
        condition: service_healthy
        restart: true
    deploy:
      resources:
        limits:
          cpus: '0.50'    # Limit CPU usage to 50% of one core
          memory: '4G'    # Limit memory usage to 4GB
  dawarich_sidekiq:
    image: freikin/dawarich:latest
    container_name: dawarich_sidekiq
    volumes:
      - /share/data/dawarich/dawarich_public:/var/app/public
      - /share/data/dawarich/dawarich_watched:/var/app/tmp/imports/watched
      - /share/data/dawarich/dawarich_storage:/var/app/storage
    networks:
      - dawarich
    stdin_open: true
    tty: true
    entrypoint: sidekiq-entrypoint.sh
    command: ['sidekiq']
    restart: on-failure
    environment:
      RAILS_ENV: development
      REDIS_URL: redis://dawarich_redis:6379/0
      DATABASE_HOST: dawarich_db
      DATABASE_USERNAME: **sensitive data removed**
      DATABASE_PASSWORD: **sensitive data removed**
      DATABASE_NAME: dawarich_development
      APPLICATION_HOSTS: localhost
      BACKGROUND_PROCESSING_CONCURRENCY: 1
      APPLICATION_PROTOCOL: http
      NOMINATIM_API_HOST: nominatim.openstreetmap.org
      NOMINATIM_API_USE_HTTPS: true
      DISTANCE_UNIT: km
      PROMETHEUS_EXPORTER_ENABLED: false
      PROMETHEUS_EXPORTER_HOST: dawarich_app
      PROMETHEUS_EXPORTER_PORT: 9394
      ENABLE_TELEMETRY: false # More on telemetry: https://dawarich.app/docs/tutorials/telemetry
      SELF_HOSTED: "true"
      SIDEKIQ_USERNAME: **sensitive data removed**
      SIDEKIQ_PASSWORD: **sensitive data removed**
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "5"
    healthcheck:
      test: [ "CMD-SHELL", "ps axu|grep -i [s]idekiq" ]
      interval: 10s
      retries: 30
      start_period: 30s
      timeout: 10s
    depends_on:
      dawarich_db:
        condition: service_healthy
        restart: true
      dawarich_redis:
        condition: service_healthy
        restart: true
      dawarich_app:
        condition: service_healthy
        restart: true
    deploy:
      resources:
        limits:
          cpus: '0.50'    # Limit CPU usage to 50% of one core
          memory: '4G'    # Limit memory usage to 4GB

The data which was processed by sidekiq at this time seems to be another point from my android owntracks app since the location and other data is slightly different.

This is how the DB looks like, you can see on first sight which points come from which source:

May 21 '25 07:05 noway42

Does anyone knows how to trigger the "Continue Reverse Geocoding" Job via console? Then I could set up a cron job to trigger it every hour as a workaround.

May 23 '25 12:05 noway42

Does anyone knows how to trigger the "Continue Reverse Geocoding" Job via console? Then I could set up a cron job to trigger it every hour as a workaround.

EnqueueBackgroundJob.perform_later("continue_reverse_geocoding", User.find_by(email: "[email protected]").id)

May 24 '25 19:05 Freika

thanks for this. this explained why I had to manually reverse geocode my points since I am using HA. Adding an hourly cron to do it is a great workaround.

May 30 '25 12:05 sbcrumb

I did some testing of this issue by calling different Dawarich API's directly (with HTTPie). The home assistant HACS integrations uses the Points API, while the manual home assistant integration instructions use the overland API.

When I call them manually with the same data (slightly different timestamps to prevent duplicates) I can see the point created through the overland API does get reverse geocoded, while the one created by the Points API does not. This is the same behaviour as with the HACS home assistant integration. You can see which APIs I used by looking at the device_id in the table below. So it looks like the Points API somehow forgets to call the Reverse Geocoding job (if that's something thats not triggered from the database but called upon creating a point by the dawarich code)

I'll change my home assistant implementation to use the REST command, that way I'll have more control and can use the overland API so I don't have to manually start the Continue Reverse Geocoding job (I have no idea how to do cron jobs, but I do know how to manually use REST apis in home assistant)

id	battery_status	ping	battery	tracker_id	topic	altitude	longitude	velocity	trigger	bssid	ssid	connection	vertical_accuracy	accuracy	timestamp	latitude	mode	inrids	in_regions	raw_data	import_id	city	country	created_at	updated_at	user_id	geodata	visit_id	reverse_geocoded_at	course	course_accuracy	external_track_id	lonlat	country_id
290070	NULL	NULL	NULL	pointsapi	NULL	0	NULL	92088	NULL	NULL	NULL	NULL	-1	5	1751576581	NULL	NULL	{}	{}	{"type": "Feature", "geometry": {"type": "Point", "coordinates": [-122.40530871, 37.74430413]}, "properties": {"speed": 92.088, "course": 27.07, "altitude": 0, "device_id": "pointsapi", "timestamp": "2025-07-03T21:03:01Z", "speed_accuracy": 0, "course_accuracy": 0, "vertical_accuracy": -1, "horizontal_accuracy": 5}}	NULL	NULL	NULL	3-7-2025 11:08	3-7-2025 11:08	1	{}	NULL	NULL	27.07000	0.00000	NULL	0101000020E6100000F98EF193F0995EC05A50945B45DF4240	NULL
290071	NULL	NULL	NULL	overlandapi	NULL	0	NULL	92088	NULL	NULL	NULL	NULL	-1	5	1751576641	NULL	NULL	{}	{}	{"type": "Feature", "geometry": {"type": "Point", "coordinates": [-122.40530871, 37.74430413]}, "properties": {"speed": 92.088, "course": 27.07, "altitude": 0, "device_id": "overlandapi", "timestamp": "2025-07-03T21:04:01Z", "speed_accuracy": 0, "course_accuracy": 0, "vertical_accuracy": -1, "horizontal_accuracy": 5}}	NULL	San Francisco	NULL	3-7-2025 11:10	3-7-2025 11:10	1	{"bbox": [-122.4052834, 37.7442168, -122.4051834, 37.7443168], "type": "Feature", "geometry": {"type": "Point", "coordinates": [-122.4052334, 37.7442668]}, "properties": {"lat": 37.7442668, "lon": -122.4052334, "city": "San Francisco", "rank": {"importance": 0.00000999999999995449, "popularity": 5.654072689836704}, "state": "California", "street": "James Lick Freeway", "suburb": "Bayview District", "country": "United States", "distance": 7.815283320012045, "place_id": "51cd0f1258ef995ec059d1bf6e2244df4240f00102f9010125600700000000c00203", "postcode": "94124", "timezone": {"name": "America/Los_Angeles", "offset_DST": "-07:00", "offset_STD": "-08:00", "abbreviation_DST": "PDT", "abbreviation_STD": "PST", "offset_DST_seconds": -25200, "offset_STD_seconds": -28800}, "formatted": "2996 James Lick Freeway, San Francisco, CA 94124, United States of America", "iso3166_2": "US-CA", "plus_code": "849VPHVV+PW", "datasource": {"url": "https://www.openstreetmap.org/copyright", "license": "Open Database License", "sourcename": "openstreetmap", "attribution": "© OpenStreetMap contributors"}, "state_code": "CA", "housenumber": "2996", "result_type": "building", "country_code": "us", "address_line1": "2996 James Lick Freeway", "address_line2": "San Francisco, CA 94124, United States of America", "plus_code_short": "PHVV+PW San Francisco, California, United States"}}	NULL	2025-07-03 11:10:17.297467	NULL	NULL	NULL	0101000020E6100000F98EF193F0995EC05A50945B45DF4240	NULL

Jul 03 '25 11:07 Dixet

I actually have the same issue with the Dawarich App (set to significant location in case that matters). Didn't have reverse geocoding enabled from first launch, but did so later, which was a couple of weeks ago. This is on docker/arm64 image and its running for days without the points without data in the stats changing. Only a manual trigger helps.

Jul 06 '25 15:07 raisinbear

Make sure your STORE_GEODATA env var is set to true

Aug 27 '25 09:08 Freika

Make sure your STORE_GEODATA env var is set to true

For me it is for both _sidekiq and _app container. Maybe (not entirely sure) it hadn’t been on first launch, but was introduced by me later when reading up on reverse geocoding. So potentially initial database creation and upload of the first couple of coordinates was done without these environment variables in case that matters.

Aug 27 '25 10:08 raisinbear