Docker Compose fails to update included files

[johntdyer]

Trying 8.0.1 w/ Docker Compose...

When I click Run in the UI:

I get the following error in the logs

02:58:09.848  WARN whats-up-docker/container: Error when running trigger (type=dockercompose, name=local) (Cannot read properties of undefined (reading 'getImageFullName'))

Docker Configs:

Wud Docker Compose file

services:
  whatsupdocker:
    container_name: "wud"
    restart: unless-stopped
    image: "getwud/wud:8.0.1"
    healthcheck:
      test: wget --no-verbose --tries=1 --no-check-certificate --spider http://127.0.0.1:3000
      interval: 10s
      timeout: 10s
      retries: 3
      start_period: 10s
    ports:
      - "3333:3000"
    environment:
      WUD_LOG_LEVEL: debug

      WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_FILE:  '/wud/docker-compose.yaml'
      # WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_PRUNE: true
      # WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_DRYRUN: true
      WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_AUTO: false
      # WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_THRESHOLD: minor
      # WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_ONCE: false
      # WUD_TRIGGER_DOCKERCOMPOSE_LOCAL_MODE: batch

      WUD_WATCHER_LOCAL_CRON: 0 1 * * *
      WUD_WATCHER_LOCAL_WATCHALL: true
      WUD_WATCHER_LOCAL_WATCHBYDEFAULT: true

      WUD_TRIGGER_DISCORD_LOCAL_BOTUSERNAME: WUD
      WUD_TRIGGER_DISCORD_LOCAL_URL: https://discord.com/api/webhooks/xxxxx/xxxxx-xxxxx-xxxxx

      WUD_REGISTRY_HUB_TOKEN: xxxxxx
      WUD_REGISTRY_HUB_LOGIN: xxxxx


      WUD_TRIGGER_MQTT_MOSQUITTO_HASS_ENABLED: true
      WUD_TRIGGER_MQTT_MOSQUITTO_PASSWORD: wud
      WUD_TRIGGER_MQTT_MOSQUITTO_USER: wud
      WUD_TRIGGER_MQTT_MOSQUITTO_URL: mqtt://192.168.120.254:1883
      WUD_TRIGGER_MQTT_MOSQUITTO_HASS_PREFIX: homeassistant

      WUD_WATCHER_UNRAID_WATCHALL: false
      WUD_WATCHER_UNRAID_CRON: 0 1 * * *
      WUD_WATCHER_UNRAID_HOST: 192.168.100.13
      WUD_WATCHER_UNRAID_WATCHBYDEFAULT: true

    labels:
      prometheus.io/scrape: false
      traefik.enable: true
      traefik.http.routers.wud.rule: "Host(`whatsup.xxx.xxx`) || Host(`wud.xxxx.xxxx`) || Host(`whatsupdocker.xxxx.xxxx`) || Host(`wud.xxx.xxx.xxxxx`) "


      # traefik.http.services.wud.loadbalancer.server.port: '3000'

      com.centurylinklabs.watchtower.enable: false

      wud.display.icon: "sh:whats-up-docker"
      wud.display.name: "whats up docker"
      wud.watch: true
      wud.tag.include: ^\d+\.\d+\.\d+$$
      wud.link.template: https://github.com/getwud/wud/releases/tag/$${major}.$${minor}.$${patch}


    networks:
      - proxy
      - backend

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/whats-up-docker:/store
      - /common/shared-config/docker-compose.yaml:/wud/docker-compose.yaml
      - /var/run/docker.sock:/var/run/docker.sock

the container I am trying to upgrade has the following docker-compose

Grafana ( Target ) Docker Compose file

services:
  grafana:
    image: grafana/grafana-oss:11.3.2
    hostname: grafana
    container_name: grafana
    restart: unless-stopped
    security_opt:
      - no-new-privileges=true
    user: "1000"
    environment:

      GF_INSTALL_PLUGINS: >-
        grafana-clock-panel,
        grafana-piechart-panel,
        simpod-json-datasource,
        volkovlabs-grapi-datasource,
        yesoreyeram-infinity-datasource,
        volkovlabs-variable-panel,
        yesoreyeram-boomtable-panel,
        volkovlabs-grapi-datasource,
        marcusolsson-static-datasource,
        marcusolsson-calendar-panel,
        marcusolsson-dynamictext-panel,
        volkovlabs-echarts-panel,
        grafana-bigquery-datasource,
        volkovlabs-table-panel,
        https://storage.googleapis.com/integration-artifacts/grafana-lokiexplore-app/grafana-lokiexplore-app-latest.zip;grafana-lokiexplore-app
      GF_DATABASE_TYPE: postgres
      GF_DATABASE_HOST: 192.168.100.13:5432
      GF_DATABASE_NAME: xxxxx
      GF_DATABASE_USER: xxxxx
      GF_DATABASE_PASSWORD: xxxxx
      #GF_DATABASE_SSL_MODE: require
      GF_INSTALL_IMAGE_RENDERER_PLUGIN: true
      GF_SERVER_ROOT_URL: https://grafana.xxxxxx.xxxxxxxxxx
      GF_RENDERING_SERVER_URL: http://grafana-image-renderer:8081/render



      GF_UNIFIED_ALERTING_CAPTURE_TIMEOUT: 10s
      GF_SMTP_ENABLED: true
      GF_SMTP_HOST: 192.168.100.3:587
      GF_SMTP_SKIP_VERIFY: false
      GF_SMTP_FROM_ADDRESS: [email protected]
      GF_AUTH_ANONYMOUS_ORG_ROLE: Admin
      GF_AUTH_ANONYMOUS_ORG_NAME: Main Org.
      GF_AUTH_ANONYMOUS_ENABLED: true
      GF_AUTH_DISABLE_LOGIN_FORM: false

      GF_LIVE_ALLOWED_ORIGINS: https://grafana.xxxxx.xxxxxxxxx
      GF_DEFAULT_INSTANCE_NAME: xxxxx xxxxx
      GF_PLUGIN_GRAFANA_IMAGE_RENDERER_RENDERING_IGNORE_HTTPS_ERRORS: true
      GF_FEATURE_TOGGLES_ENABLE: newNavigation
      GF_PATHS_PROVISIONING: /var/lib/grafana/conf/provisioning
      GF_DATAPROXY_MAX_IDLE_CONNECTIONS: 300
      GF_REMOTE_CACHE_TYPE: redis
      GF_REMOTE_CACHE_CONNSTR: addr=redis:6379,pool_size=100,db=0,ssl=false
      # GF_ENTERPRISE_LICENSE_VALIDATION_TYPE: aws
      GF_LOG_LEVEL: info
      GF_LOG_FILE_FORMAT: json
      GF_LOG_CONSOLE_FORMAT: json


    env_file:
      - .env
    networks:
      - proxy
      - database
      - redis
      - monitoring
    volumes:
      - grafana:/var/lib/grafana
    labels:
      prometheus.io/scrape: false
      traefik.enable: true
      traefik.http.routers.grafana.rule: "Host(`grafana.xxxx.xxxxx`)"


      wud.watch: "true"
      wud.display.icon: "si:grafana"
      wud.display.name: "Grafana"
      wud.tag.include: ^\d+\.\d+\.\d+$$
      wud.link.template: https://github.com/grafana/grafana/releases/tag/v$${major}.$${minor}.$${patch}

      kuma.__web: '{ "host": "grafana", "port" : 3000, "parent_name":"monitoring", "max_retries":3}'

      homepage.widget.type: grafana
      homepage.widget.url: https://grafana.xxxxx.xxxxxx
      homepage.widget.fields: dashboards
        datasources
        totalalerts
        alertstriggered

    healthcheck:
      interval: 10s
      test: /usr/bin/curl  --silent --fail -k localhost:3000 || exit 1
      timeout: 30s
      retries: 2
      start_period: 15s

    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p    /var/lib/grafana/conf/provisioning/access-control
        mkdir -p    /var/lib/grafana/conf/provisioning/alerting
        mkdir -p    /var/lib/grafana/conf/provisioning/datasources
        mkdir -p    /var/lib/grafana/conf/provisioning/dashboards
        mkdir -p    /var/lib/grafana/conf/provisioning/plugins
        cat <<EOF > /var/lib/grafana/conf/provisioning/datasources/ds.yaml
        apiVersion: 1
        # List of data sources to delete from the database.
        deleteDatasources:
          - name: Mimir
          - name: Loki
          - name: Loki Server
          - name: mimir
          - name: Prometheus
          - name: Mimir DS
        datasources:
        - name: Loki DS
          type: loki
          access: proxy
          orgId: 1
          url: http://loki:3100
          basicAuth: false
          isDefault: false
          version: 1
          editable: false

        - name: Mimir DS
          type: prometheus
          access: proxy
          orgId: 1
          url: http://mimir:9009/prometheus
          basicAuth: false
          isDefault: true
          version: 1
          editable: false
          jsonData:
            httpMethod: POST
            manageAlerts: true
            prometheusType: Prometheus
            prometheusVersion: 2.44.0
            cacheLevel: 'High'


        EOF
        /run.sh

[johntdyer]

Just found the trigger tester in the UI and tried to run it and when I do I get the following in the UI:

 Trigger executed with error (Error when running trigger dockercompose.local (Cannot read properties of undefined (reading 'dockerApi'))}) 

[johntdyer]

I think I figured out part of it... my docker-compose file is broken into more manageable units since I currently have 94 containers. I then include the path in my docker compose file which I think breaks your ability to find and update said file...

x-common-service-config: &common-service-config
  environment:
    TZ: America/New_York

  restart: unless-stopped
  logging:
    driver: "json-file"
    options:
      max-file: "10"
      max-size: "200k"
include:
  - path: '/common/shared-config/docker/lidarr.yaml'
  - path: '/common/shared-config/docker/ombi.yaml'
  - path: '/common/shared-config/docker/minio.yaml'
  - path: '/common/shared-config/docker/metermon.yaml'
  - path: '/common/shared-config/docker/dns_exporter.yaml'
  ...

I figured out a way to prove this by moving one of my units into the main file and running the trigger to update it. But I'd still like to keep my current pattern to better manage things. Do you think my approach could be supported too?

[coredump]

What I did, since I also have a bunch of included files, is to create a compose.override.yaml with only service names and image. That seems to be working fine with WUD now

[johntdyer]

@coredump , curious what that looks like in practice... coudl you share a sample to help illustrate ?

[coredump]

@coredump , curious what that looks like in practice... coudl you share a sample to help illustrate ?

I have a structure like this

.
├── compose.override.yaml
├── compose.yaml
├── frigate
│  └── compose.yaml
├── gatus
│  └── compose.yaml
├── gitea
│  └── compose.yaml
├── glances
│  └── compose.yaml
├── networks.compose.yaml

My main compose.yaml on the root directory has includes for all the other compose files from the specific directories, plus the volumes and network ones , for example

include:
  - path: volumes.compose.yaml
  - path: networks.compose.yaml
    # Services
  - path: frigate/compose.yaml
  - path: gitea/compose.yaml

And on the compose.override.yaml I have something like:

services:
  frigate:
    image: ghcr.io/blakeblackshear/frigate:stable
  gatus:
    image: twinproduction/gatus:v5.16.0
  gitea:
    image: twinproduction/gatus:v5.16.0

The documentation about the override file is from here

The override file is the one that is configured on WUD . It seems to be working, aka the updates that are found get updated and removed/added again with the new images. Ideally I would prefer WUD to have an option to only edit the file and not restart stuff but apparently the _AUTO doesn't work with dockercompose trigger.

[johntdyer]

So even after this change I am still getting

Cannot read properties of undefined (reading 'dockerApi')

[johntdyer]

here are the logs, at trace level, but honestly not seeing anything different then when running debug

(node:7) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
22:02:50.144  INFO whats-up-docker/trigger.dockercompose.debian: Registering for auto execution
22:02:50.160  INFO whats-up-docker/trigger.mqtt.mosquitto: Registering for auto execution
22:02:50.160  INFO whats-up-docker/registry: Register all components of kind registry for provider ecr
22:02:50.259  INFO whats-up-docker/registry.ecr.public: Register with configuration {}
22:02:50.259  INFO whats-up-docker/registry: Register all components of kind registry for provider gcr
22:02:50.261  INFO whats-up-docker/registry.gcr.public: Register with configuration {}
22:02:50.261  INFO whats-up-docker/registry: Register all components of kind registry for provider ghcr
22:02:50.262  INFO whats-up-docker/registry.ghcr.public: Register with configuration {}
22:02:50.263  INFO whats-up-docker/registry: Register all components of kind registry for provider hub
22:02:50.266  INFO whats-up-docker/registry.hub.public: Register with configuration {"token":"d**********************************Y","login":"johntdyer"}
22:02:50.266  INFO whats-up-docker/registry: Register all components of kind registry for provider quay
22:02:50.268  INFO whats-up-docker/registry.quay.public: Register with configuration {}
22:02:50.268  INFO whats-up-docker/registry: Register all components of kind registry for provider lscr
22:02:50.269  INFO whats-up-docker/registry.lscr.private: Register with configuration {"token":"g**************************************L","username":"johntdyer"}
22:02:50.430  INFO whats-up-docker/watcher.docker.unraid: Register with configuration {"cron":"0 1 * * *","host":"192.168.100.13","watchall":false,"watchbydefault":true,"socket":"/var/run/docker.sock","port":2375,"watchevents":true,"watchatstart":true}
22:02:50.430  INFO whats-up-docker/watcher.docker.unraid: Cron scheduled (0 1 * * *)
22:02:50.456  INFO whats-up-docker/watcher.docker.debian: Register with configuration {"cron":"0 1 * * *","watchall":true,"watchbydefault":true,"socket":"/var/run/docker.sock","port":2375,"watchevents":true,"watchatstart":true}
22:02:50.456  INFO whats-up-docker/watcher.docker.debian: Cron scheduled (0 1 * * *)
22:02:50.479  INFO whats-up-docker/registry: No authentication configured => Allow anonymous access
22:02:50.482  INFO whats-up-docker/authentication.anonymous.anonymous: Register with configuration {}
22:02:50.483 DEBUG whats-up-docker/api: API/UI enabled => Start Http listener on port 3000
22:02:50.490  WARN whats-up-docker: Anonymous authentication is enabled; please make sure that the app is not exposed to unsecure networks
22:02:50.496  INFO whats-up-docker/api: Server listening on port 3000 (HTTP)
22:02:51.455  INFO whats-up-docker/watcher.docker.unraid: Listening to docker events
22:02:51.479  INFO whats-up-docker/watcher.docker.debian: Listening to docker events
22:03:00.449  WARN whats-up-docker/container: Error when running trigger (type=dockercompose, name=debian) (Cannot read properties of undefined (reading 'dockerApi'))

config for WUD is as follows:

  whatsupdocker:
    env_file:
      - .env
    container_name: "wud"
    restart: unless-stopped
    image: "getwud/wud:8.0.1"
    ports:
      - "3333:3000"
    environment:
      WUD_LOG_LEVEL: trace

      WUD_TRIGGER_DOCKERCOMPOSE_DEBIAN_FILE:  '/wud/docker-compose.yaml'

      WUD_WATCHER_DEBIAN_CRON: 0 1 * * *
      WUD_WATCHER_DEBIAN_WATCHALL: true
      WUD_WATCHER_DEBIAN_WATCHBYDEFAULT: true

    labels:
      wud.watch: true
      wud.tag.include: ^\d+\.\d+\.\d+$$
      wud.link.template: https://github.com/getwud/wud/releases/tag/$${major}.$${minor}.$${patch}
    networks:
      - frontend
      - backend

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/whats-up-docker:/store
      - /common/shared-config/docker/docker-compose.yaml:/wud/docker-compose.yaml
      - /var/run/docker.sock:/var/run/docker.sock

Here is an example of my ./docker/grafana/compose.yaml file

services:

  grafana:
    hostname: grafana
    image: grafana/grafana-oss
    container_name: grafana
    restart: unless-stopped
    security_opt:
      - no-new-privileges=true
    user: "1000"
    environment:
      GF_INSTALL_PLUGINS: >-
        volkovlabs-table-panel
      GF_DATABASE_TYPE: postgres
      GF_DATABASE_HOST: 192.168.100.13:5432
......
.....

Here is my compose.override.yaml files relevent section defining the actual version

services:

  grafana:
    image: grafana/grafana-oss:11.5.2

and here is my compose.yaml that references the ./grafana/compose.yaml file

include:

  - path: './grafana/compose.yaml'

Any idea what I am doing wrong here?

[johntdyer]

@coredump / @fmartinou - Any thoughts here... really banging my head up against the wall on this one and would appreciate any guidance you guys might be able to provide

[coredump]

Not really. I thought mine was working but the Docker Compose trigger doesn't alter my file or do anything at all really, so I kinda stopped caring about it :(

[johntdyer]

@coredump - Well at least I know I am not alone or crazy that mine wasnt working :) I'll guess I'll just leave this issue open for @fmartinou or others to maybe chime in on at some point . Thanks and have a good weekend

[avargaskun]

I'm running into another instance of this issue. I split my compose project into 'modules' and include each module's partial compose file conditionally at the time docker compose up is called. I create multiple dockercompose triggers, one for each partial compose file that defines a service. The problem is, some of the partial compose files specify overrides for services they depend on, so there are sometimes 'incomplete' service definitions in a given compose yml file.

Example:

base/docker-compose.yml (fragment)

services:
  wud:
    container_name: wud
    image: ghcr.io/getwud/wud:8.0.1
    # ...
    environment:
      - TZ=${TZ}
      - WUD_WATCHER_LOCAL_CRON=0 8 * * *
      - WUD_WATCHER_LOCAL_WATCHBYDEFAULT=false
      - WUD_TRIGGER_DOCKERCOMPOSE_BASE_AUTO=false
      - WUD_TRIGGER_DOCKERCOMPOSE_BASE_FILE=/wud/compose/base/docker-compose.yml
      - WUD_TRIGGER_DOCKERCOMPOSE_BASE_PRUNE=true
    labels:
      # ...
      wud.watch: true
      wud.trigger.include: dockercompose.base
      wud.display.icon: sh-wud
      wud.tag.include: ^\d+\.\d+\.\d+$$
      wud.link.template: https://getwud.github.io/wud/#/changelog/?id=_$${major}$${minor}$${patch}

portainer/docker-compose.yml (fragment)

services:
  wud:
    environment:
      - WUD_TRIGGER_DOCKERCOMPOSE_PORTAINER_AUTO=false
      - WUD_TRIGGER_DOCKERCOMPOSE_PORTAINER_FILE=/wud/compose/portainer/docker-compose.yml
      - WUD_TRIGGER_DOCKERCOMPOSE_PORTAINER_PRUNE=true
  portainer:
    container_name: portainer
    image: portainer/portainer-ce:2.27.4
    # ...
    labels:
      # ...
      wud.watch: true
      wud.trigger.include: dockercompose.portainer
      wud.display.icon: sh-portainer
      wud.tag.include: ^\d+\.\d+\.\d+$$
      wud.link.template: https://github.com/portainer/portainer/releases/tag/$${major}.$${minor}.$${patch}

The problem is that the code for the Dockercompose trigger iterates the services in my partial file for portainer and finds the 'incomplete' definition for wud. Specifically, it tries to access properties of services.wud.image and fails because image is not defined for wud in this particular file.

[avargaskun]

Update: I put together a PR with a fix. I tested this against the scenario described above by manually applying the fix to the Dockercompose.js file after deploying the official WUD container image.

@fmartinou FYI