Allow `MAX_ATTEMPTS` configurable

Open dhpikolo opened this issue 10 months ago • 1 comments

In this PR, I allow configuration of MAX_ATTEMPTS envar

Add to the metaflow_config file
Mount the new envar to kubermetes, argo-workflows, airflow and aws-batch plugins.

New Behaviour

User can set METAFLOW_MAX_ATTEMPTS to configure MAX_ATTEMPTS internal envvar.

Resolves: https://github.com/Netflix/metaflow/issues/2276

Feb 19 '25 13:02 dhpikolo

I'll take a look a bit closer but one thing I can say already is that we probably want to min it with what is already there because otherwise if it is decreased, the client may misbehave. Related to that, I don't know if we can dis-associate it totally from the run in the client because again, otherwise, the client may misbehave.

Feb 19 '25 18:02 romain-intel