Add internal Fargate container to ignore list

[ayozemr]

Describe what happened: When using Datadog Agent as sidecar container on Fargate (https://www.datadoghq.com/blog/monitor-aws-fargate/), a hidden internal container is discovered by Datadog called "internalecspause" with image "fg-proxy:tinyproxy"

Describe what you expected: This container should be filtered like its being done in Kubernetes (AFAIK), because its counting for billing and its a hidden container in AWS. Don't think nobody wants to metric it in DD.

Steps to reproduce the issue: Add Datadog Agent to the task definition of your service with ECS_FARGATE=true and other env vars. When task starts, you will see 3 containers in DD: Your service + DD Agent + AWS internal pause

Additional environment details (Operating System, Cloud provider, etc): AWS Fargate

Datadog agent task env vars: DD_AC_EXCLUDE | "image:." DD_AC_INCLUDE | "image:REDACTED.dkr.ecr.us-east-1.amazonaws.com/." DD_API_KEY | APIKEY DD_APM_ENABLED | true DD_APM_NON_LOCAL_TRAFFIC | true DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL | true DD_LOGS_ENABLED | true ECS_FARGATE | true

[danjamesmay]

You've posted your API key :1st_place_medal:

[ayozemr]

It was just a test one, but thanks anyway...

[jpbochi]

It would be even better if we could have a whitelist (or blacklist) of containers to look for (or ignore).

[ayozemr]

I found an ignore env var, but didnt work for me.

key: DD_AC_EXCLUDE value: image:fg-proxy image:datadog/agent

[irabinovitch]

@ayozemr Please note that containers in Fargate do not count towards billing at this time. We do plan to release billing for Fargate soon, but it will be task based. # of containers will not impact you.

[ayozemr]

Many thanks for the info! :-)

Do you know if they can be filtered anyway? Just to have dashboard clean

[irabinovitch]

@ayozemr The fargate integration doesn't yet support whitelisting / blacklisting containers via DD_AC_EXCLUDE. But we do intended to add that feature soon. We'll keep this issue open to track the feature request but we also have a similar internal ticket that I've added you to as a requester for follow up once its released.

[pygillier]

@irabinovitch any news on this ? Having a quite large ECS cluster leads to having a lot of theses "useless" containers showing in container view & cluttering it.

[ayozemr]

Yes, it is very messy. Especially in the honeycomb view

[vboulineau]

Hello, it has been a long (too long) wait, but we're implementing this feature, it'll be available in Agent 6/7.22

[jdmarquezf]

Hello, how are you? I have the same problem and I want to know if the charge for the containers is still being made, because I would like to stop monitoring the containers internal ecs pause

[swordfish444]

@vboulineau Has this feature been released? If yes, lets close this issue.

[raf-lv]

Any news on how to ignore certain containers from being monitored in Fargate?

[mhemani-reach]

You can add the following environment variables to your task definitions to exclude the pause container.

{
  "name": "DD_CONTAINER_EXCLUDE",
  "value": "name:aws-fargate-pause"
},
{
  "name": "DD_CONTAINER_EXCLUDE_METRICS",
  "value": "name:aws-fargate-pause"
}

[dd-octo-sts[bot]]

This issue has been automatically marked as stale because it has not had activity in the past 15 days.

It will be closed in 30 days if no further activity occurs. If this issue is still relevant, adding a comment will keep it open. Also, you can always reopen the issue if you missed the window.

Thank you for your contributions!

[gabedos]

Hello all, thanks for raising your concerns about filtering container monitoring on Fargate.

As mentioned by a comment above, you can use the DD_CONTAINER_EXCLUDE options to filter out certain containers from collection through attributes like image, name, and kube_namespace. You can reference the Container Discovery Management documentation for more details on configuring these options. These filtering configuration options are compatible with the ECS Fargate integration.

Additionally, we are also adding the aws-fargate-pause container to the default list of excluded pause containers from collection in this PR https://github.com/DataDog/datadog-agent/pull/43372 targeting Agent v7.74.0+.