Stacks Updater is triggered properly, but the container task does not complete

[JahedZ]

Describe the bug

In data.all, there is a scheduled Fargate task that is triggered each night, at 1:00 UTC. It automatically updates all stacks (environments, datasets) that have drifted from the CDK template.

These tasks are properly scheduled. However, the container does not complete the tasks due to an IAM permission error.

How to Reproduce

The logs are available in Cloudwatch under [demo/ecs/stacks-updater].

Expected behavior

The tasks should properly execute, and update old datasets/workspaces upon completion.

This allows data.all to maintain up-to-date resources. For example, if you update the definition of what an environment is, you will not have to manually update all existing tasks from the console, it will simply automatically happen, at most 24h after the CI/CD completion.

Your project

No response

Screenshots

OS

Any

Python version

3.8

AWS data.all version

v 1.0.0

Additional context

I will provide the code to

1/ Extend the IAM permissions for the container role, allowing successful task execution

2/ Extend the task execution to also update pipelines, notebook, and ML Studio profiles.

[github-actions[bot]]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed.