whats-up-docker icon indicating copy to clipboard operation
whats-up-docker copied to clipboard

Published 20 hours ago • verified publisher icon fmartinou

WUD_WATCHER_LOCAL_CRON Config Help/Question

Open samcro1967 opened this issue 1 year ago • 7 comments

I believe I have WUD configured correctly yo run each day at 1:00 AM, but it is running multiple times a day. It has been up 2 days and has run 122 times instead of 2. Hoping someone can help point me to what I am missing in the config.

Docker compose for WUD

  whatsupdocker:
    hostname: whatsupdocker
    image: fmartinou/whats-up-docker:latest
    container_name: whatsupdocker
    restart: always
    environment:
      - TZ=${TZ}
      # - WUD_LOG_LEVEL=debug
      - WUD_SERVER_PORT=8841
      - WUD_WATCHER_LOCAL_CRON=0 1 * * *
      - WUD_WATCHER_LOCAL_WATCHBYDEFAULT=true
      # - WUD.WATCH.DIGEST=false
      - WUD_REGISTRY_CUSTOM_URL=$LOCAL_REGISTRY
      - WUD_REGISTRY_GHCR_USERNAME=$EMAIL
      - WUD_REGISTRY_GHCR_TOKEN=$GHCR_TOKEN_WHATSUPDOCKER
      - WUD_REGISTRY_GCR_CLIENTEMAIL=$EMAIL
      - WUD_REGISTRY_GCR_PRIVATEKEY=$GCR_PRIVATEKEY
      - WUD_REGISTRY_HUB_LOGIN=$USER
      - WUD_REGISTRY_HUB_PASSWORD=$HUB_TOKEN
      - WUD_REGISTRY_LSCR_USERNAME=$EMAIL
      - WUD_REGISTRY_LSCR_TOKEN=$GHCR_TOKEN_WHATSUPDOCKER
      - WUD_TRIGGER_DOCKERCOMPOSE_WUD_FILE=/wud/docker-compose.yml
      - WUD_TRIGGER_DOCKERCOMPOSE_WUD_BACKUP=true
      - WUD_TRIGGER_DOCKERCOMPOSE_WUD_PRUNE=true
      - WUD_TRIGGER_DOCKERCOMPOSE_WUD_ONCE=false
      - WUD_TRIGGER_DOCKERCOMPOSE_WUD_MODE=batch
      - WUD_TRIGGER_APPRISE_LOCAL_URL=http://192.168.1.104:8084
      - WUD_TRIGGER_APPRISE_LOCAL_URLS=gotify://192.168.1.104:8132/<redacted>,https://discord.com/api/webhooks/<redacted>/<redacted>,pover://<redacted>@<redacted>
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ${docker}/whatsupdocker:/store
      - /etc/localtime:/etc/localtime:ro
      - ${docker}/docker-compose.yml:/wud/docker-compose.yml
    ports:
      - 8841:8841
    labels:
      - wud.watch=false
      - docker.build=docker
    networks:
      - wg-pia
    depends_on:
      - registry
    healthcheck:
      test: wget --no-verbose --tries=1 --no-check-certificate --spider http://192.168.1.104:8841
      interval: 60s
      timeout: 10s
      retries: 3
      start_period: 10s

docker ps -a --filter "name=whatsupdocker"

CONTAINER ID   IMAGE                              COMMAND                  CREATED      STATUS                PORTS                                                 NAMES
071230badba3   fmartinou/whats-up-docker:latest   "/usr/bin/entrypoint…"   2 days ago   Up 2 days (healthy)   3000/tcp, 0.0.0.0:8841->8841/tcp, :::8841->8841/tcp   whatsupdocker

docker logs whatsupdocker 2>&1 | grep 'Cron finished'

08:00:41.200  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
08:01:31.593  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
09:00:26.253  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 2 available updates)
09:00:39.445  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 2 available updates)
09:00:49.831  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
09:00:49.853  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
09:02:49.066  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
09:03:10.348  INFO whats-up-docker/watcher.docker.local: Cron finished (115 containers watched, 0 errors, 0 available updates)
09:05:44.818  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
09:19:03.828  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
09:19:46.562  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
12:00:24.095  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
12:03:35.331  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
13:00:30.806  INFO whats-up-docker/watcher.docker.local: Cron finished (115 containers watched, 0 errors, 0 available updates)
16:00:10.989  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:01:13.822  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:00:13.238  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:01:24.615  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 6 available updates)
20:01:52.017  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 5 available updates)
20:02:22.182  INFO whats-up-docker/watcher.docker.local: Cron finished (114 containers watched, 0 errors, 3 available updates)
20:02:25.350  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:02:27.575  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:02:38.161  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:33:59.134  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
00:00:17.804  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 1 available updates)
00:00:50.093  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
00:07:27.147  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
00:59:06.061  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 1 available updates)
00:59:10.899  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 1 available updates)
00:59:14.602  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:00:02.927  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:29:10.172  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:29:11.479  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:29:11.482  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:30:37.700  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:30:59.116  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 3 available updates)
01:31:18.421  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 1 available updates)
01:32:29.140  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:33:03.603  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 1 available updates)
01:33:36.026  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 2 available updates)
01:33:41.560  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:34:03.649  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:35:20.366  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:35:21.445  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:35:27.943  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:35:49.262  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:35:56.414  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:36:20.282  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:36:45.810  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:38:55.152  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:38:58.916  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
01:39:33.009  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
01:40:09.888  INFO whats-up-docker/watcher.docker.local: Cron finished (115 containers watched, 0 errors, 0 available updates)
01:40:45.351  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:41:32.162  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
02:00:20.085  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 1 available updates)
02:00:52.013  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
04:00:19.760  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
04:01:27.392  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
08:00:30.614  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 3 available updates)
08:00:54.674  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 2 available updates)
08:00:57.748  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
08:00:59.628  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
08:02:22.507  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
10:00:19.501  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
10:00:27.383  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
10:00:41.088  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
10:02:57.204  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
10:03:23.946  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 2 available updates)
10:03:51.874  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 1 available updates)
10:03:54.755  INFO whats-up-docker/watcher.docker.local: Cron finished (115 containers watched, 0 errors, 0 available updates)
10:05:47.239  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
12:00:23.971  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 2 available updates)
12:00:43.301  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
12:00:43.367  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
12:02:37.666  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:00:22.573  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 5 available updates)
16:00:43.260  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:00:43.330  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:00:43.565  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:00:52.438  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:01:27.130  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
16:02:02.956  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
20:00:23.923  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 3 available updates)
20:00:43.340  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 2 available updates)
20:00:48.928  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:01:40.301  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
20:03:33.624  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
00:00:21.807  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 1 available updates)
00:00:53.215  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
00:01:37.069  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
00:59:01.813  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
00:59:58.349  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:00:11.053  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:00:11.061  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:29:05.877  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
01:30:36.122  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 1 available updates)
01:31:05.095  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:32:04.964  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:32:28.707  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 1 available updates)
01:32:54.760  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 1 available updates)
01:33:08.904  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:33:50.572  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:34:31.423  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 1 available updates)
01:34:50.721  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:35:06.887  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:35:53.686  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:37:47.572  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:37:49.150  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:38:08.525  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 1 available updates)
01:38:21.747  INFO whats-up-docker/watcher.docker.local: Cron finished (118 containers watched, 0 errors, 0 available updates)
01:38:58.470  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 0 available updates)
01:40:06.188  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
01:40:44.044  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
02:00:18.166  INFO whats-up-docker/watcher.docker.local: Cron finished (117 containers watched, 0 errors, 2 available updates)
02:00:54.004  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
02:00:54.163  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
04:00:14.362  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
04:01:33.345  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 1 available updates)
04:01:50.856  INFO whats-up-docker/watcher.docker.local: Cron finished (116 containers watched, 0 errors, 0 available updates)
08:00:19.220  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)
08:03:54.231  INFO whats-up-docker/watcher.docker.local: Cron finished (0 containers watched, 0 errors, 0 available updates)

samcro1967 avatar Sep 17 '24 14:09 samcro1967

Hi,

At startup, the cron value is printed; is the parsed value correct?

May you try to enclose the env var with quotes?

- "WUD_WATCHER_LOCAL_CRON=0 1 * * *"

fmartinou avatar Sep 18 '24 13:09 fmartinou

I see the following int he logs: 08:32:24.194 INFO whats-up-docker/watcher.docker.local: Cron started (0 1 * * *)

samcro1967 avatar Sep 18 '24 13:09 samcro1967

You are providing a timezone via environment variable and by mounting your servers timezone into the container. Any chance those two are different? Maybe that could cause weird behavior? I'd remove one of those regardless.

proud-nerd avatar Sep 25 '24 14:09 proud-nerd

They are not different, but I have removed the volume mount for timezone and will see if it makes any difference.

samcro1967 avatar Sep 25 '24 15:09 samcro1967

Removing the volume mount made no difference.

WUD ran at midnight last night which is the same time a script kicks off that runs the plextraktsync container. The container ran for about 3 mins. It appears that every time a new container comes on line, goes offline, or is removed, WUD is performing a new scan and ignoring the cron schedule. The WUD log below shows it knows what time it is supposed to run, but seems to have ignored the schedule and ran anyway.

04:00:11.285  INFO whats-up-docker/watcher.docker.local: Cron started (0 1 * * *)
04:03:15.643  INFO whats-up-docker/watcher.docker.local: Cron started (0 1 * * *)

samcro1967 avatar Sep 27 '24 12:09 samcro1967

It appears that every time a new container comes on line, goes offline, or is removed, WUD is performing a new scan

That's likely because of the WUD_WATCHER_{watcher_name}_WATCHEVENTS property. If you want wud to scan only with the cron, you need to disable it. Can you try?

fmartinou avatar Sep 27 '24 13:09 fmartinou

That seems to have made an impact. When WUD_WATCHER_{watcher_name}_WATCHEVENTS is not set and defaults to true, WUD scans when a container is removed and readded. It does not when it is set to false.

A few suggestions (take them or leave them) I have would be as follows:

  1. Update the log entry from Cron started to something indicating that it is scanning due to a docker event and not due to the cron schedule. From the log message it appears the scan was initiated by the cron schedule instead of a docker event.

  2. Consider changing the default for WUD_WATCHER_{watcher_name}_WATCHEVENTS to false given the upcoming changes to Docker Hub rate limits. I run 100+ containers on the host and had 43,000+ line items show up in the docker pull request report in a 24 hour period. Not sure what Docker is considering a pull request as I did not have 43,000+ container updates in a 24 hour period. It was probably more like 10 container updates. Must count a docker pull to see if there is a new image and not just the actual pull of a new image as a pull request.

Another possibility could be to add a WATCHEVENTS to each registry type so it could be controlled by registry, but that seems a bit more complex to implement. In that scenario each event would have to be processed, determine which registry the event belongs to, see if the it is set to true or false, and then scan or not based on _WATCHEVENTS.

One last question, I noticed that the logs only contain the time and not the date. I am not seeing anything in the docs about configuring the content of a log message to be able to include the date. Am I missing something or is date just not added to the logs by design?

samcro1967 avatar Sep 27 '24 15:09 samcro1967