metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

Published 20 hours ago • verified publisher icon Netflix

Step functions task definition ContainerOverrides length over 8192

Open redabuspatrol opened this issue 3 years ago • 4 comments

Hi,

I am having the following error when launching a flow on AWS step functions.

 "Status": "FAILED",
  "StatusReason": "Container Overrides length must be at most 8192"

It seems my flow generates a task definition where the ContainerOverrides property is longer than 8192 chars. My flow runs fine on AWS Batch, though it seems SFN doesn't like it very much. Is there something I could try and do to get around this, or is this an issue with Metaflow itself?

Thanks a lot.

redabuspatrol avatar Mar 11 '22 15:03 redabuspatrol

Would it be possible for you to share the output of python flow.py step-functions create --only-json? If you want to share it privately, you can DM me at chat.metaflow.org. We have some checks in place to catch this edge case but it could very well be the case that we are violating some condition unintentionally.

savingoyal avatar Mar 11 '22 21:03 savingoyal

I just shared it with you over the outerbounds slack, please let me know if anything

redabuspatrol avatar Mar 14 '22 14:03 redabuspatrol

@redabuspatrol This PR will ensure that your workflow can execute again. We should be able to ship it out soon!

savingoyal avatar Mar 23 '22 22:03 savingoyal

I will spend some time in a bit further compressing the ContainerOverrides field.

savingoyal avatar Mar 23 '22 22:03 savingoyal