cloudwatch logging can not set retention

[matti]

Why do you want this feature?

First of all, I know this is not directly related to eksctl, but I still think that eksctl could make this smoother.

My nodes have:

iam:
  withAddonPolicies:
     cloudWatch: true

When I install https://github.com/aws/eks-charts/tree/master/stable/aws-for-fluent-bit chart there is a helm value to set cloudWatch.logRetentionDays: which is "never" by default. If I set this value in the chart, then the pods are crashing with:

time="2022-08-10T17:19:05Z" level=error msg="AccessDeniedException: User: arn:aws:sts::254324309357:assumed-role/eksctl-test-7-nodegroup-base-2-8-NodeInstanceRole-1TG07V2PQPOUS/i-00b9cd5b8155f8073 is not authorized to perform: logs:PutRetentionPolicy on resource: arn:aws:logs:eu-north-1:254324309357:log-group:/aws/eks/fluentbit-cloudwatch/logs:log-stream: because no identity-based policy allows the logs:PutRetentionPolicy action\n\tstatus code: 400, request id: df720515-e9ba-4ce1-adec-09f3fbd7145f"

as CloudWatchAgentServerPolicy doesn't have this, even if it can create the log group (!)

So, I tried to create a role and set it in chart, but this fails with is not authorized to perform: sts:AssumeRole on resource so now I don't know what to do next.

What feature/behavior/change do you want?

Something to make this smoother so that I could create short-lived ephemeral clusters that don't leave their logs after them with retentionpolicy Never.