[Feature Request/Idea] Utilizing Redis TTL

[MineTech13]

I believe it would be advantageous for this system to utilize the Redis Time-To-Live (TTL) feature. By setting a TTL that is slightly longer than the polling interval, we could effectively achieve automatic removal of services that have failed or have been stopped incorrectly, ensuring that only active services remain in the Redis database.

[chetan]

Good suggestion, however it could mean that the service is briefly removed from traefik for a few seconds between the time that the TTL expires and the next poll interval, if traefik scans redis during that period.

If kop continues to run, the dead/inactive service should get removed anyway.

[MineTech13]

Sorry for the late reply it somehow went under my radar.

I'm very happy to see this addition as i have run a few times in the issue that the entire vm/lxc Container running KOP and the Service its exposing crashed or something happend where the TTL would have catched the weird edge case.

Thooo about the briefly removed portion i dont think soo if i understood the TTL correctly u can just rewrite the option and it will reset the TTL sooo as long as the next scan and update comes before the TTL runs out it should never disappear from the redis.

Cant wait to test this option gonna look into that after the weekend ^^

[MineTech13]

I tested v0.17 and it seems to not set the TTL in the redis DB

my docker-compose.yml

services:
  kop:
    image: ghcr.io/jittering/traefik-kop:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - REDIS_ADDR=[REDACTED]
      - REDIS_PASS=[REDACTED]
      - REDIS_TTL=120
      - DEBUG=true
      - BIND_IP=[REDACTED]
      - KOP_HOSTNAME=[REDACTED]

my redis db

[chetan]

Hmm ok, I'll take another look at it

[chetan]

Fixed the TTL issue, going to push it soon, but thinking about it some more, I don't think it's going to work the way you want. The polling mechanism does not actually push anything to redis if the docker status hasn't changed. i.e., each tick of the poller does not actually refresh the config in redis when there are no changes. So the TTL will just expire the records. kop doesn't actually do any reconciliation between redis & your docker config, but I do see what the issue is. I think in order to fix this scenario, we need to regularly update those records, even when there are no changes to keep the TTL moving forward so that they don't expire while everything is running smoothly.

That said, don't you really want to see the gateway timeout from traefik instead of the entire route disappearing? I'm not sure I see the point in removing it entirely when the service goes down like this.

[chetan]

Following up here, I think I could add an additional flag to force-push the current config even when no changes occurred on each tick of the poller loop.

[MineTech13]

Probably a good idea also gonna run another check on the ttl today sorry my past weeks where busy (nvmd. just seen u didnt got to it yet but gonna keep an eye out)

The point about removing entries entirely is that I'm running a deployment with around 12 traefik-kop instances, as I'm basically isolating each service on its own dedicated LXC container. These can be spun up or deleted fairly quickly, and I don't want to regularly perform database checks to remove services that are no longer present. I've had to do this several times because when I hit stop on an LXC container without logging in and stopping everything gracefully (while Docker handles system shutdown fine), I often end up with ghost entries due to Docker freezing its containers on shutdown and not properly stopping them.Another issue I encountered was when I made changes that didn't update correctly. I don't know the exact circumstances, but I had renamed some things and ended up with the old rules still present under the old names as ghost entries, plus the new rules on top of that. While the ghost entries didn't cause harm, I don't like that this can happen as it could cause weird behavior if it occurs at an unlucky moment. Troubleshooting that could be problematic if you don't know where to look.Having outdated entries disappear automatically is a smart way to ensure only active configuration remains in Redis. You're right that if you don't push every time, the TTL will run out. So the force push option would be a nice addition to make this a fully usable feature.