logspout
logspout copied to clipboard
gliderlabs
syslog adapter doesn't obey DNS TTLs
I've got an infrastructure wherein my syslog sink maintains a consistent hostname, but the underlying host/IP is subject to change at-will. Because the syslog adapter simply configures its connection (and performs DNS lookup) at "boot" time via transport.Dial, these underlying IP changes won't ever be picked up.
I could certainly restart my logspout container to pick up the new IP whenever I move my syslog sink, but that's a problem, too, since I then must restart all of the other containers on the host in question.
It'd be awesome if the syslog adapter observed DNS TTLs - it'd make life much easier for those of us using DNS-based service discovery.
We have this same issue and it's a real blocker on high availability for our infrastructure.
That'd do the job! Could go fancier and only reconnect if the resultant IP address from the dns lookup changes, but just doing a reconnect after hitting the TTL each time would solve our problem.
We're using round robin dns for our log servers and only returning healthy hosts. Currently logspout gets stuck trying to log to hosts that may not even exist anymore.
We have the same as if the server that the DNS query resolves to gets cached and if that server goes away logspout forever sends the data into the void since it never retries the DNS.
We have the same issue, and is there any update / resolution on this? Thanks.
I could certainly restart my logspout container to pick up the new IP whenever I move my syslog sink, but that's a problem, too, since I then must restart all of the other containers on the host in question.
Why would the other containers on the host need to be restarted?
I had not come across this idea previously. Does logspout not log from all containers running upon startup, and for any that startup afterward as well?
It was my understanding that logspout was "always" processing logs from all running containers.
The connection that logspout creates is a persistent one. It is not subject to reconnect based on DNS TTL. A DNS lookup should only be preformed at the time just before a connection would get established. This is pretty much how all clients work. If your endpoint becomes unavailable due to an IP change, which DNS picks up, then a new connection should be established eventually. There should be no logic in the application to take DNS TTL into account for reconnecting. This is un-heard-of behaviour, and would cause more problems then it would potentially fix.
Please accept the reconnect PR listed above #253 as a potential work-around to your issue. Then we can close this issue
