metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

Support parameter defaults that depend on other parameters

Open avicennax opened this issue 5 years ago • 2 comments

I'd like to be able to define parameters in my Flow as such:

experiment_name = Parameter(
    'run_name',
    help="Experiment version",
    default="v0"
)

data_path = Parameter(
    'data_path',
    default=f"s3://bucket/experiments/{experiment_name}/data"
)

scripts_path = Parameter(
    'scripts_path',
    default=f"s3://bucket/experiments/{experiment_name}/scripts"
)

Is there a way to easily do this or plans to support it? Naturally the example above fails since the string form parameters aren't generated until the flow is instantiated. However, the flow instance parameters are properties with no setters so they can't be formatted at flow run-time. I've also tried creating function wrappers but they fail due to type violations.

While I can create new formatted parameter string objects and bind them to the flow to use downstream - it feels like an anti-pattern.

avicennax avatar Feb 23 '20 23:02 avicennax

Thanks, interesting idea indeed but we will have to think through the evaluation order if parameters depend on other parameters.

savingoyal avatar Feb 28 '20 20:02 savingoyal

while not exactly what's asked in this issue, #1962 should help in many similar use cases.

savingoyal avatar Aug 16 '24 06:08 savingoyal