Scheduler not discovering docker-compose container based on labels

[ashleysommer]

Here's my minimal reproduction of the docker-compose file:

version: "3"
services:

  scheduler:
    image: mcuadros/ofelia
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    labels:
      ofelia.job-local.my-test-job.schedule: "@daily"
      ofelia.job-local.my-test-job.command: "echo 'still running'"
    command: ["daemon", "--docker"]

  test_schedule:
    image: alpine:3.9
    command: ["tail", "-f", "/dev/null"]
    depends_on:
      - scheduler
    labels:
      ofelia.enabled: "true"
      ofelia.job-exec.test-job1.schedule: "@every 5s"
      ofelia.job-exec.test-job1.command: "/bin/sh -c 'echo hello'"

Here is the output:

$> docker-compose  up                                                                                                                                               
Recreating docker_scheduler_1 ... done
Creating docker_test_schedule_1 ... done
Attaching to docker_scheduler_1, docker_test_schedule_1
scheduler_1      | scheduler.go:35 ▶ NOTICE New job registered "my-test-job" - "echo 'still running'" - "@daily"
scheduler_1      | scheduler.go:55 ▶ DEBUG Starting scheduler with 1 jobs

As you can see, the scheduler only registers the one job-local configuration, it does not discover the one from my second container.

I've based the config on the docker-compose file I saw in this issue: #76, on that page in the text I see it seems to work fine, the output in the text in that issue shows two jobs registered.

Using: Docker 18.09.7 Docker-compose 1.23.2 Ofelia: latest pull from dockerhub.

I'm hoping there's something simple and obvious I'm doing wrong. Is there any kind of additional debugging output I can use to check why its not working?

[taraspos]

Hey, problem is that ofelia can't discover labels in runtime, it reads the label on start. This means that the container must be running before ofelia.

So in your case, to fix your docker-compose, you need to change the dependencies.

So now, your test_schedule depends on scheduler but it needs to be another way around to work.

But yes, this is the missing feature that will need to be implemented.

[ashleysommer]

Hi @Trane9991

That swapping around the dependencies you suggested is actually the first thing I thought of and the first thing I tried when I was debugging this yesterday. It didn't make any difference and the same problem occurred.

However just now after seeing your response, I created a new docker-compose file with the dependencies swapped around, now it does work as expected. So that was the problem.

Thanks for your help.

[ashleysommer]

In regard to how to implement a feature to fix this, I think the only way to do it would be to somehow have a notification from the docker daemon to alert ofelia when a new container has started on the host, then ofelia can check if it has the right labels, and can register a new scheduled task.

That sounds like its easy on the surface, but likely hard to properly implement.

[taraspos]

@ashleysommer most likely ofelia will need to poll docker for new containers every few secs and update the jobs configuration. However not sure if this will be easy to fit in the current architecture.

[4n70w4]

Just need watch docker events https://docs.docker.com/engine/reference/commandline/events/ Example https://github.com/containous/traefik/blob/09c07f45ee999fad6dc97463b4b825fb08c7c34b/pkg/provider/docker/docker.go#L271

This is difficult for me, maybe you can figure it out? @Trane9991

[helmut72]

Hey, problem is that ofelia can't discover labels in runtime, it reads the label on start. This means that the container must be running before ofelia.

This becomes a showstopper for me, because I use some docker-compose files and even if I use one only, it's required to check if other containers are running or have a big list in 'depends on'.

[rdelcorro]

Fixed here: https://github.com/mcuadros/ofelia/pull/137