[ECS] [request]: Full Support for Running Fluent Bit as Daemon Service to collect all logs on an EC2 node

[PettitWesley]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

Support AWS for Fluent Bit running as a daemon service on ECS EC2, which means:

1. Easily support injecting task metadata into logs: https://github.com/aws/containers-roadmap/issues/1442
2. Support templating log group/log stream names so that they can be customized based on metadata (already implemented): https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch#log-stream-and-group-name-templating-using-record_accessor-syntax
3. Support on BOTH on Windows and Linux

Why support Fluent Bit as a Daemon in ECS EC2 when we already have FireLens?

Currently, ECS EC2 only supports a sidecar architecture for Fluent Bit with FireLens. This is in contrast to EKS EC2 where a daemon deployment of Fluent Bit is the standard. Some customers would prefer a daemon service for the following major reasons:

1. A single daemon for all tasks on an EC2 host should in general use less CPU and memory total than a sidecar for every task.
2. A daemon can easily use file system buffering since it can mount a host path to store the file buffer. Sidecars can not easily use file system buffers because when a task is restarted the buffer will be lost, and since many tasks with the same task definition may be scheduled on a single host, they can not all use the same host path to store their file buffers as their would be conflicts.
3. Daemons can be restarted and re-configured independently of the tasks they serve. Since the Daemon will primarily read log files (and Fluent Bit will store its file offset in a local DB), when the Daemon is restarted it can pick up where it left off. So the log agent can be restarted without log loss.

Which service(s) is this request for? ECS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? What outcome are you trying to achieve, ultimately, and why is it hard/impossible to do right now? What is the impact of not having this problem solved? The more details you can provide, the better we'll be able to understand and solve the problem.

[PettitWesley]

Once launched, this repo should be updated or archived: https://github.com/aws-samples/amazon-ecs-fluent-bit-daemon-service/issues/3

[romankydybets]

is there any updates on this?