E3SM-Project
SCREAM provenance
Two sub-issues:
-
Inputs: We need to catalog any new input files (namelists, yaml etc.) that SCREAM uses to setup a run. This list needs to be propagated to CIME so that provenance capture logic knows about the new files to capture.
-
atmchange: On a related note, the file "replay.sh" captures all the CIME commands that are used to setup and run a case from a specific case directory (create_newcase, case.setup, xmlchange, case.build etc.). This file then gets archived. Example from an E3SM case: https://pace.ornl.gov/replaysh/108415 We need to think if we want to capture atmchange commands to that file or capture them another way to aid reproducibility.
Once we get a list of new input configuration files and format (yaml), we will add parsing support for them in PACE.
Capturing provenance is really important but I don't think we should work on this task until we're running at ne1024, which is a higher priority.
I completely agree, just opened an issue so that we don't forget about this.
@sarats , just a heads up that I have not forgotten about this. There's a PR for CIME that moves most model-specific stuff, including provenance, out of CIME. Once we have that in SCREAM's CIME submodule, we will be free to tinker around with how we do provenance.
@sarats, now that provenance code lives in the E3SM (and therefore SCREAM) repo, do you want to take this ticket?
