Add new token for timestamped study directories

[jwhite242]

This request is to make $(OUTPUT_PATH) behave more like the other labels and variables such that it is accessible without the timestamped sub directories appended to it when the spec is run. Making the timestamped sub directories accessible in another token in the workflow spec and meta files would be a useful side effect of this change.

[crkrenn]

I agree that $(OUTPUT_PATH) is unnecessarily confusing. I would propose that $(OUTPUT_PATH_ROOT) point to the root directory and $(OUTPUT_PATH) point to the run directory.

[crkrenn]

We had a brief conversation about this today.

Currently $(OUTPUT_PATH) in study.yaml/env/variables points to the directory in which timestamped directories are created, but $(OUTPUT_PATH) in cmd blocks points to the actual timestamped directory.

This causes some confusion and causes difficulty when programming for maestro and encore because encore adds another directory level.

A proposal would be to create a new variable $(STUDY_ROOT) that points to the directory in which timestamped directories are created.

A question is whether to give users a grace period for the transition. If yes, both $(OUTPUT_PATH) and $(STUDY_ROOT) could be used to set the directory in which timestamped directories are created, and $(STUDY_ROOT) would point to that directory.

If no grace period, setting $(OUTPUT_PATH) in study.yaml/env/variables would no longer be allowed.

If we choose a grace period, we should decide how long it should be. 3-6 months seems reasonable to me, and we could output warnings in the meantime.

-Chris

env:
    variables:
        OUTPUT_PATH: ./sample_output/hello_world

[jsemler]

It would be helpful if these options were also extended to maestro's CLI. Currently, maestro allows defining the output_path via a flag. The specified path is used as the study location and doesn't create a timestamped directory.

$ maestro run study.yaml --out path/to/study

Maestro will run the study in the location path/to/study.

It would be helpful to also add an option to specify a --study_root flag:

$ maestro run study.yaml --study_root path/to/studies

Maestro would then create the study with a timestamp at the path path/to/studies.

[FrankD412]

A proposal would be to create a new variable $(STUDY_ROOT) that points to the directory in which timestamped directories are created.

I envision this would be available alongside the $(OUTPUT_PATH) variable and another variable $(STUDY_ROOT) which returns to you the timestamped current study workspace.

A question is whether to give users a grace period for the transition. If yes, both $(OUTPUT_PATH) and $(STUDY_ROOT) could be used to set the directory in which timestamped directories are created, and $(STUDY_ROOT) would point to that directory.

If no grace period, setting $(OUTPUT_PATH) in study.yaml/env/variables would no longer be allowed.

If we choose a grace period, we should decide how long it should be. 3-6 months seems reasonable to me, and we could output warnings in the meantime.

I agree with the notion of a grace period/warning period, @crkrenn -- we do need to clarify if we're getting rid of OUTPUT_PATH or if it is a keyword alongside STUDY_ROOT. I believe the behavior we want is to be alongside. What do you think?

@jsemler @jwhite242

[jsemler]

I think having OUTPUT_PATH and STUDY_ROOT makes sense.