global-workflow
global-workflow copied to clipboard
Convert staging job to python and yaml
Description
This PR converts the staging job from shell to python and introduces the use of yaml.
Changes in this PR:
- Rename
scripts/exglobal_stage_ic.sh
toscripts/exglobal_stage_ic.py
. - Update
jobs/JGLOBAL_STAGE_IC
to use.py
script extension. MoveCOM*
variable declarations and member loop down into yaml and python respectively. MoveGDATE/gPDY/gcyc
settings up to JJOB from ex-script and replace with newer cycle variables (as done in forecast job). - Create
parm/stage
folder to hold newly createdstage.yaml.j2
, which both mimics forecast-only functionality in existingscripts/exglobal_stage_ic.sh
and adds functionality for cycled mode. - Create
ush/python/pygfs/task/stage.py
to house staging job python functions for call fromscripts/exglobal_stage_ic.py
. - Remove
stage_ic
job rocoto dependencies from xml. Do not need and removes area of duplicate maintenance. - Add cycled staging jobs for gdas and enkf suites.
- Rename
model_data
tomodel
for issue #2686
There will now be distinct stage_ic
jobs for each RUN
: gdasstage_ic
, gfsstage_ic
, enkfgdasstage_ic
, stage_ic
(for gefs).
Related work was done to set up new symlink folder structure under supported platform ICSDIR
folder for use by updated staging job.
Resolves #2475 Resolves #2650 Resolves #2686
Type of change
- New feature (adds functionality)
Change characteristics
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO
How has this been tested?
Ran all of the CI tests (either just staging jobs or end-to-end) on WCOSS2. Ran some of the CI tests on Hera and Hercules.
Ran some control CI tests from develop
and compared outputs against test CI from staging branch. Outputs reproduced between control and test.