metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

Batch job fail when user name has spaces

Open Dudemullet opened this issue 5 years ago • 6 comments

Apple lets user names have spaces but when a user with spaces tries to run a batch job with Metaflow aws does not let job names have spaces. The following error is what a user would see.

An error occurred (ClientException) when calling the SubmitJob operation: Error executing request, Exception : Job name should match valid pattern, RequestId: 1db9be95-2aba-4ff0-b19d-a61435a95791

Dudemullet avatar Jan 14 '20 21:01 Dudemullet

Thanks for the bug report. I will take a look at the PR.

savingoyal avatar Jan 27 '20 18:01 savingoyal

Quick Fix: You can fix it by changing this function in the file at "/usr/local/lib/python3.7/site-packages/metaflow/plugins/aws/batch". As AWS Batch doesn't allow spaces or dots in their job names I replaced them with a space: user=user.replace(" ", "").replace(".", "")

def _job_name(self, user, flow_name, run_id, step_name, task_id, retry_count):
        return '{user}-{flow_name}-{run_id}-{step_name}-{task_id}-{retry_count}'.format(
            user=user.replace(" ", "").replace(".", ""),
            flow_name=flow_name,
            run_id=run_id,
            step_name=step_name,
            task_id=task_id,
            retry_count=retry_count,
        )

rmeinlsainsburys avatar Mar 24 '20 08:03 rmeinlsainsburys

@rmeinlsainsburys We actually have a PR #101 that will notify of the errors more precisely. Also, I believe the preferred method would be to set your USER environment variable.

  • You don't have to update your username at the system level (might be restricted)
  • don't have to update source code

Dudemullet avatar Mar 24 '20 15:03 Dudemullet

One more data point - https://twitter.com/johancarlin/status/1412700513842470913?s=20

savingoyal avatar Jul 07 '21 19:07 savingoyal

Curious, why is the above fix not yet implemented?

csetzkorn avatar Jul 26 '23 16:07 csetzkorn

Setting the METAFLOW_USER environment variable fixed it for me.

mmarqz avatar Oct 30 '25 05:10 mmarqz