Lots of failed reverse geocoding and barely any suggested visits

[runzl0r]

OS & Hardware Ubuntu and Docker and iOS App

Version 0.31.0

Describe the bug I have a lot of failed jobs in reverse geocoding and get only some suggested visits

Expected behavior

Screenshots

How can I get rid of almost a third of failed jobs in Sidekiq? Or is there a job I can run to retry it? I already started reverse geocoding again but it didnt fix it. In addition I get rarely proposals in suggested visits. I had two in August even though I was 4 days on a roadtrip and had a lot of stops. Used the iOS app all the time.

[runzl0r]

Now that I scroll through the suggested visits it appears that they only come from Immich and not at all from my visits using the dawarich iOS app.

[runzl0r]

detection in the app definitely works. Do I have to disable suggestions if I use the app?

[runzl0r]

I watched it now a little longer. Reverse geocoding seems to work. But I get no suggested visits after using the app. What can I provide to solve this issue?

[Clear2437]

Have you been able to get your old points geocoded? For new points added its working for me, but I cannot find a way to get my old 65k points to get reverse geocoded. Ive also run the the reverse geocoding job in the admin panel to redo all points but it just starts and finishes a second later without changing anything. No errors in any logs though. It seems like its just not re-checking them.

[runzl0r]

@Freika Any idea why the suggestion box is kind of random? @Clear2437 You can remove the jobs in the console. I can't find the command right now, will link here once I find it.

This request resulted from the expectation that I find the iOS app notification: "Been at parents street 25 for 27 min" also somewhere in my local instance of dawarich. Expected place would be the suggestion area or confirmed visits. But that's not the case, so I opened this issue to see if my expectation is off or if I'm too stupid or misconfigured something..

[Freika]

detection in the app definitely works. Do I have to disable suggestions if I use the app?

these visits are not yet being sent to the server, it will be supported soon though

Have you been able to get your old points geocoded? For new points added its working for me, but I cannot find a way to get my old 65k points to get reverse geocoded. Ive also run the the reverse geocoding job in the admin panel to redo all points but it just starts and finishes a second later without changing anything. No errors in any logs though. It seems like its just not re-checking them.

Settings -> Background Jobs -> Start reverse geocoding

@runzl0r I think restarting reverse geocoding should help you as well

[runzl0r]

I did a restart and it didn't fix it but maybe my expectation is also off. All the suggestions from the app should also be visible in dawarich server. Not that they are sent but if the app is able to detect that I have been for 10 min at a supermarket, shouldn't the server at home also be able?

[Freika]

Not that they are sent but if the app is able to detect that I have been for 10 min at a supermarket, shouldn't the server at home also be able?

The iOS app relies on visits the iOS itself provides, while in Dawarich, we have our own implementation of visit detection. Once we implement sending visits from Dawarich to iOS to the Dawarich server, they will compliment each other

[flyingspud]

Came across this thread while dealing with my own high failure rates (in the millions lol)

My docker compose was referencing to my own hosted Photon server via HTTPS and reverse proxy, so I tried changing it to direct IP mapping without HTTPS, now it is starting to get processed with almost zero failures. Hope this nugget of info prove useful for your environment.

[runzl0r]

Sadly it is still not working and to be honest I don't know how to proceed. If I import something from Immich, I get the suggestions. If I import the points from the iOS app, there are no suggestions. Even if I restart reverse geocoding. I even travelled to different countries, stayed on big airports that were detected in the app, but in dawarich server there is no detection.

[Freika]

This many failures shows you have configuration issue, it's not normal. Why don't you share your compose file and logs?

[runzl0r]

this is my docker-compose:

networks:
  dawarich:

services:
  dawarich_redis:
    image: redis:7.4-alpine
    container_name: dawarich_redis
    command: redis-server
    networks:
      - dawarich
    volumes:
      - dawarich_shared:/data
    restart: always
    healthcheck:
      test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]
      interval: 10s
      retries: 5
      start_period: 30s
      timeout: 10s

  dawarich_db:
    image: postgis/postgis:17-3.5-alpine
    shm_size: 1G
    container_name: dawarich_db
    volumes:
      - dawarich_db_data:/var/lib/postgresql/data
      - dawarich_shared:/var/shared
      # - ./postgresql.conf:/etc/postgresql/postgresql.conf # Optional, uncomment if you want to use a custom config
    networks:
      - dawarich
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-postgres}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}
      POSTGRES_DB: ${POSTGRES_DB:-dawarich_development}
    restart: always
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-postgres} -d ${POSTGRES_DB:-dawarich_development}" ]
      interval: 10s
      retries: 5
      start_period: 30s
      timeout: 10s
    # command: postgres -c config_file=/etc/postgresql/postgresql.conf # Use custom config, uncomment if you want to use a custom config

  dawarich_app:
    image: freikin/dawarich:latest
    container_name: dawarich_app
    volumes:
      - dawarich_public:/var/app/public
      - dawarich_watched:/var/app/tmp/imports/watched
      - dawarich_storage:/var/app/storage
      - dawarich_db_data:/dawarich_db_data
    networks:
      - dawarich
    ports:
      - "${DAWARICH_APP_PORT:-3000}:3000"
      # - "${PROMETHEUS_PORT:-9394}:9394" # Prometheus exporter, uncomment if needed
    stdin_open: true
    tty: true
    entrypoint: web-entrypoint.sh
    command: ['bin/rails', 'server', '-p', '3000', '-b', '::']
    restart: on-failure
    environment:
      RAILS_ENV: ${RAILS_ENV:-development}
      REDIS_URL: ${REDIS_URL:-redis://dawarich_redis:6379}
      DATABASE_HOST: ${DATABASE_HOST:-dawarich_db}
      DATABASE_PORT: ${DATABASE_PORT:-5432}
      DATABASE_USERNAME: ${DATABASE_USERNAME:-postgres}
      DATABASE_PASSWORD: ${DATABASE_PASSWORD:-password}
      DATABASE_NAME: ${DATABASE_NAME:-dawarich_development}
      MIN_MINUTES_SPENT_IN_CITY: ${MIN_MINUTES_SPENT_IN_CITY:-60}
      APPLICATION_HOSTS: ${APPLICATION_HOSTS:-localhost,::1,127.0.0.1}
      TIME_ZONE: ${TIME_ZONE:-Europe/London}
      APPLICATION_PROTOCOL: ${APPLICATION_PROTOCOL:-http}
      PROMETHEUS_EXPORTER_ENABLED: ${PROMETHEUS_EXPORTER_ENABLED:-"false"}
      PROMETHEUS_EXPORTER_HOST: ${PROMETHEUS_EXPORTER_HOST:-0.0.0.0}
      PROMETHEUS_EXPORTER_PORT: ${PROMETHEUS_EXPORTER_PORT:-9394}
      SECRET_KEY_BASE: ${SECRET_KEY_BASE:-"CHANGE_ME"}
      RAILS_LOG_TO_STDOUT: ${RAILS_LOG_TO_STDOUT:-"true"}
      SELF_HOSTED: ${SELF_HOSTED:-"true"}
      STORE_GEODATA: ${STORE_GEODATA:-"true"}
      PHOTON_API_HOST: photon.vanoosterhout.cloud
      PHOTON_API_USE_HTTPS: true 
    logging:
      driver: "json-file"
      options:
        max-size: ${LOG_MAX_SIZE:-100m}
        max-file: ${LOG_MAX_FILE:-5}
    healthcheck:
      test: [ "CMD-SHELL", "wget -qO - http://127.0.0.1:3000/api/v1/health | grep -q '\"status\"\\s*:\\s*\"ok\"'" ]
      interval: 10s
      retries: 30
      start_period: 30s
      timeout: 10s
    depends_on:
      dawarich_db:
        condition: service_healthy
        restart: true
      dawarich_redis:
        condition: service_healthy
        restart: true
    deploy:
      resources:
        limits:
          cpus: ${APP_CPU_LIMIT:-0.50}
          memory: ${APP_MEMORY_LIMIT:-4G}

  dawarich_sidekiq:
    image: freikin/dawarich:latest
    container_name: dawarich_sidekiq
    volumes:
      - dawarich_public:/var/app/public
      - dawarich_watched:/var/app/tmp/imports/watched
      - dawarich_storage:/var/app/storage
    networks:
      - dawarich
    stdin_open: true
    tty: true
    entrypoint: sidekiq-entrypoint.sh
    command: ['sidekiq']
    restart: on-failure
    environment:
      RAILS_ENV: ${RAILS_ENV:-development}
      REDIS_URL: ${REDIS_URL:-redis://dawarich_redis:6379}
      DATABASE_HOST: ${DATABASE_HOST:-dawarich_db}
      DATABASE_PORT: ${DATABASE_PORT:-5432}
      DATABASE_USERNAME: ${DATABASE_USERNAME:-postgres}
      DATABASE_PASSWORD: ${DATABASE_PASSWORD:-password}
      DATABASE_NAME: ${DATABASE_NAME:-dawarich_development}
      APPLICATION_HOSTS: ${APPLICATION_HOSTS:-localhost,::1,127.0.0.1}
      BACKGROUND_PROCESSING_CONCURRENCY: ${BACKGROUND_PROCESSING_CONCURRENCY:-10}
      APPLICATION_PROTOCOL: ${APPLICATION_PROTOCOL:-http}
      PROMETHEUS_EXPORTER_ENABLED: ${PROMETHEUS_EXPORTER_ENABLED:-"false"}
      PROMETHEUS_EXPORTER_HOST: ${PROMETHEUS_EXPORTER_HOST_SIDEKIQ:-dawarich_app}
      PROMETHEUS_EXPORTER_PORT: ${PROMETHEUS_EXPORTER_PORT:-9394}
      SECRET_KEY_BASE: ${SECRET_KEY_BASE:-"CHANGE_ME"}
      RAILS_LOG_TO_STDOUT: ${RAILS_LOG_TO_STDOUT:-"true"}
      SELF_HOSTED: ${SELF_HOSTED:-"true"}
      STORE_GEODATA: ${STORE_GEODATA:-"true"}
      PHOTON_API_HOST: photon.vanoosterhout.cloud
      PHOTON_API_USE_HTTPS: true
    logging:
      driver: "json-file"
      options:
        max-size: ${LOG_MAX_SIZE:-100m}
        max-file: ${LOG_MAX_FILE:-5}
    healthcheck:
      test: [ "CMD-SHELL", "pgrep -f sidekiq" ]
      interval: 10s
      retries: 30
      start_period: 30s
      timeout: 10s
    depends_on:
      dawarich_db:
        condition: service_healthy
        restart: true
      dawarich_redis:
        condition: service_healthy
        restart: true
      dawarich_app:
        condition: service_healthy
        restart: true

volumes:
  dawarich_db_data:
  dawarich_shared:
  dawarich_public:
  dawarich_watched:
  dawarich_storage:

[flyingspud]

@runzl0r if everything is hosted on-prem, switch out the photon hostname to IP:port and remove "PHOTON_API_USE_HTTPS" PHOTON_API_HOST: IPaddress:port #PHOTON_API_USE_HTTPS: true give that a try, that resolve the issue for me.

[runzl0r]

Thanks, but I dont use a local photon instance. Geocoding service also seems to work, all the immich photos have an address but there is no visit detection from my points.