Run an "accessory" on every service server

[stepbeekio]

Problem

I'd love to be able to deploy an accessory to every server used by the service. This would be similar in spirit to a kubernetes daemonset. My motivation for this is primarily around monitoring applications, e.g. a datadog agent or log forwarding agent. I can workaround this right now by configuring these "node" level processes with terraform on server creation, but this strikes me as a blocker for many projects.

This would be a powerful tool to have in smoothing the transition from something like Heroku to IaaS imo. By combining this with traefik.args.accesslog: true we'd be able to enable a very simple mechanism for users to forward request logs to a log aggregator.

Super Hacky solution

Currently. each accessory is deployed to a single host. I could do a hacky change to enable this particular use-case by editing https://github.com/mrsked/mrsk/blob/main/lib/mrsk/cli/accessory.rb#L11 to accept an array of hosts rather than manually pasting the IP addresses in this array, but it feels pretty hacky.

accessories:
  datadog:
    image: gcr.io/datadoghq/agent:7
    host:
    - 1.1.1.1
    - 1.1.1.2
    ...

Slightly less hacky solution

Or I guess I could configure some conditional logic and add a new config variable like this:

accessories:
  datadog:
    image: gcr.io/datadoghq/agent:7
    daemon: true

Which triggered some conditional logic to grab the IPs from the top-level servers config in lib/mrsk/commands/accessory.rb?

I'd probably want this to blow up at initialization time if the host and daemon config variable were both set.

While writing this, I've started to wonder whether accessories should remain single instance containers like they are currently. Maybe what I'm describing is a completely new kind of entity? If someone from 37signals can weigh in how you're handling monitoring for services deployed with mrsk right now then that would be helpful.

[northeastprince]

This feels like something that should be in the application itself. For example, the newrelic-rpm gem for monitoring. That said, if you're deploying multiple things on the same server with MRSK, host level metrics could be very useful.

[stepbeekio]

@northeastprince Can you explain why? If I have a machine that's running multiple docker containers, I don't think it's unreasonable to want to be able to forward logs from each of them and monitor the resource utilization of the host itself. This is also in line with the 12-factor app advice around logging: https://12factor.net/logs so I'm unlikely to be the only developer who structures applications in this way.

[dhh]

Closed via https://github.com/mrsked/mrsk/pull/164