fix: enable logs from containers that are transient.

[adamamsmith]

Fix: loki.source.docker only collecting logs for containers in the running state

Slightly different implementation of https://github.com/grafana/alloy/pull/3449. All credit to @RasEza.

From his PR:

Due to the change in https://github.com/grafana/alloy/pull/742 loki.source.docker is no longer able to begin log collection for containers that only briefly have the "running" container state. This means that transient, short-lived or quickly restarting containers may never have their logs collected.

This allows for log collection of containers not in the running state if they have not been collected from already.

Which issue(s) this PR fixes

Fixes #3408

Notes to the Reviewer

PR Checklist

• [x] CHANGELOG.md updated
• [x] Tests updated

[CLAassistant]

All committers have signed the CLA.

[wildum]

Hey thanks for the contribution :) I tried it by starting a docker container and then quickly killing it but then it's not picked up by the discovery.docker so I don't get the logs. In which scenario is the container not running but still discoverable and its logs can be read?

[adamamsmith]

Hey thanks for the contribution :) I tried it by starting a docker container and then quickly killing it but then it's not picked up by the discovery.docker so I don't get the logs. In which scenario is the container not running but still discoverable and its logs can be read?

You also need to have the following config for your discovery.docker. Then containers that aren't running will be picked up by discovery.docker

          discovery.docker "linux" {
            host = "unix:///var/run/docker.sock"
            refresh_interval = "10s"

            filter {
              name    = "status"
              values  = ["created","restarting","running","removing","paused","exited","dead"]
            }
          }

[wildum]

You also need to have the following config for your discovery.docker. Then containers that aren't running will be picked up by discovery.docker

Thanks, it tested it, it works as expected :)