metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

Get the value of reserved_params inside flow step

Open TonyIvchenko opened this issue 3 years ago • 4 comments

Is that possible to get the value or reserved_params inside FlowSpec step? As for instance, how I can get the value of max-workers param if it is set or default value if not?

TonyIvchenko avatar Apr 20 '22 22:04 TonyIvchenko

Not in a straightforward manner. There is an open ticket #571 to add them to the current object. One immediate workaround would be to launch your workflow as METAFLOW_MAX_WORKERS=100 python flow.py run and sniff the value of METAFLOW_MAX_WORKERS in os.environ but this approach wouldn't work when running on AWS Batch where the local environment variables are not passed all the way to AWS Batch.

savingoyal avatar Apr 20 '22 23:04 savingoyal

Thank you, unfortunately we need it in AWS Batch, do you have an ETA for #571

TonyIvchenko avatar Apr 20 '22 23:04 TonyIvchenko

I can get a PR out next week? Would that work?

savingoyal avatar Apr 20 '22 23:04 savingoyal

Absolutely, thank you!

TonyIvchenko avatar Apr 21 '22 16:04 TonyIvchenko