Batch job fail when user name has spaces
Apple lets user names have spaces but when a user with spaces tries to run a batch job with Metaflow aws does not let job names have spaces. The following error is what a user would see.
An error occurred (ClientException) when calling the SubmitJob operation: Error executing request, Exception : Job name should match valid pattern, RequestId: 1db9be95-2aba-4ff0-b19d-a61435a95791
Thanks for the bug report. I will take a look at the PR.
Quick Fix:
You can fix it by changing this function in the file at "/usr/local/lib/python3.7/site-packages/metaflow/plugins/aws/batch".
As AWS Batch doesn't allow spaces or dots in their job names I replaced them with a space:
user=user.replace(" ", "").replace(".", "")
def _job_name(self, user, flow_name, run_id, step_name, task_id, retry_count):
return '{user}-{flow_name}-{run_id}-{step_name}-{task_id}-{retry_count}'.format(
user=user.replace(" ", "").replace(".", ""),
flow_name=flow_name,
run_id=run_id,
step_name=step_name,
task_id=task_id,
retry_count=retry_count,
)
@rmeinlsainsburys We actually have a PR #101 that will notify of the errors more precisely. Also, I believe the preferred method would be to set your USER environment variable.
- You don't have to update your username at the system level (might be restricted)
- don't have to update source code
One more data point - https://twitter.com/johancarlin/status/1412700513842470913?s=20
Curious, why is the above fix not yet implemented?
Setting the METAFLOW_USER environment variable fixed it for me.