[EKS]: amazon-cloudwatch-observability addon allow fluent-bit Only Install

[colinbjohnson]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

I would like the amazon-cloudwatch-observability to support a fluent-bit only install.

Notes:

• Currently AWS supports a "cloudwatch-agent" only install but a company I work with desires the opposite

Which service(s) is this request for?

EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

• Have the ease of use of the amazon-cloudwatch-observability addon
• Avoid installing unused/unwanted resources within an EKS cluster (the extra containers increase cost slightly and add additional operational complexity via extra pods)

Are you currently working around this issue?

We are using the addon as is.

Additional context

• having two addons might be worth considering for a number of reasons - namely that cloudwatch and fluentbit are, in fact, two different products
• from a customer's perspective this would maintain ease of use but allow greater control of what is installed in their cluster

[colinbjohnson]

To add some information regarding the impact of this to customers that do use the amazon-cloudwatch-observability addon but do not want the cloudwatch agent features:

• the increased pod count will be equal to the number of nodes running plus 1 additional pod (so a 4 node cluster would have 5 additional running pods)
• each server will have 1/4 less cores and 128 Mi less memory (from the cloudwatch-agent pod description) - so a 4 node cluster would have 1 less core and 512 Mi less memory
• additional costs will be incurred by shipping emf logs to CloudWatch logs (this looks to be $0.85/day or more depending on the number of pods running in a given cluster)

[sky333999]

Starting with v3.0.0-eksbuild.1 of the add-on, there is a new advanced config option, agents, which can be used to manage 0 or more CloudWatchAgent installations. The default is still the one CloudWatchAgent daemonset as before when agents is not specified.

So for your requirement, you should now be able to install the add-on with the following advanced config as an example:

{
    "agents": []
}

[colinbjohnson]

@sky333999 - thank you!!! Have deployed to a customer's staging cluster earlier today!