[ECS] [bug]: Exec into all containers when pidMode=task

[robertjli]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

https://repost.aws/questions/QUbsgWD4yERDuphXDKOWbNFA/ecs-execute-command-not-working-for-pid-mode-task

At this time, the behavior of Amazon ECS is non-deterministic with respect to enableExecuteCommand when pidMode is set to task. The AWS SSM agent (which powers the feature) will be running in one of the containers only, but right now you cannot specify which container is the one in which it will run, nor can you specify that you want it to run in all of them.

It would be extremely useful to be able to exec into all containers for debugging.

Which service(s) is this request for? Fargate, ECS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

Currently, when deploying a task with sidecar containers, it is impossible to debug issues in a given container unless it's specified as the first one (and from the accepted answer, that is not even a guarantee). Specifically, I'm running an agent container that collects process-level metrics, and the agent has subcommands that I would like to run to assist in debugging.

Are you currently working around this issue?

No current workaround.

Additional context Anything else we should know?

Attachments If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[kellenanker7]

this is a pretty serious gap IMO. it's not obvious and is also not called out anywhere in the ECS docs; it took me several hours of debugging and searching to find the AWS repost thread quoted in this bug report.

[suemayaheldursi]

Hello - this is a huge problem and preventing us from debugging issues with our sidecar setup for logging. Are there any workarounds or solutions???

[rfreuler]

same here... this is a rather big problem running security containers in a true sidecar setup we loose the ability to troubleshoot the containers. I'd love to see a fix for this...

[and-stuber]

This is impairing my Monitoring with Elasticsearch in production environment. Would be nice see it moving on soon.

[JonKusz]

This issue forces us to choose between;

• Monitoring process-level metrics in our containers via a sidecar container, requiring pidMode to be set to task.
• Being able to exec into all containers to troubleshoot issues.

It would be optimal if we were able to achieve both.

[js-truework]

For our particular use case, it would also be acceptable if pidMode was a valid runtime option in boto3's ECS.Client.run_task or as an override. We don't ever really connect to existing running tasks (to avoid accidentally negatively impacting production web requests, etc), but rather spin up a task specifically for exec'ing into. So if our production containers could still have sidecar monitoring from datadog with pidMode set to task, but we have the option to spin up a new task based on a task definition but with pidMode being unset (or set to empty string or null or whatever is valid), that would be very helpful.

[sotzing]

Given the prevalence of sidecar container use - this seems like a major limitation/bug.

We're using the Datadog sidecar (which requires use of pidMode=task for process monitoring) but not being able to Exec into the container (which we do for debugging purposes) is a major issue for us...