pecan icon indicating copy to clipboard operation
pecan copied to clipboard

Refactor workflow to stateless run manifest architecture and SA input design coordination

Open divine7022 opened this issue 2 weeks ago • 0 comments

Description

fixes #3692

  • updated run.write.configs to generate a runs_manifest.csv file containing run metadata (run_id, site_id, pft_name, trait, quantile, type) instead of appending to samples.Rdata.
  • samples.Rdata is now treated as a read-only "Master parameter definition" file.
  • updated write.sa.configs and write.ensemble.configs to return manifest dataframes.

fixes #3693

  • refactored read.sa.output and read.ensemble.output to pass the full vector of requested variables to read.output in a single call.
  • eliminated the inefficient inner loop that opened/closed netcdf files for every individual variable.

Additionally this PR separates input design generation for SA and ensemble runs, fixing dimension mismatch errors in multisite workflows. previously both SA and ensemble used a single input_design matrix, causing failures when:

  • Ensemble needed N runs (e.g. 50)
  • SA needed 1 (median) + (traits * quantiles) runs (e.g. 145)

causes in dimension mismatch errors in input design coordination.

runModule.run.write.configs generates separate ens_input_design and sa_input_design matrices. run.write.configs accepts both input_design_ens and input_design_sa parameters. Old input_design parameter still works with deprecation warning.

Motivation and Context

Review Time Estimate

  • [x] Immediately
  • [ ] Within one week
  • [ ] When possible

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • [ ] My change requires a change to the documentation.
  • [ ] My name is in the list of CITATION.cff
  • [ ] I agree that PEcAn Project may distribute my contribution under any or all of
    • the same license as the existing code,
    • and/or the BSD 3-clause license.
  • [ ] I have updated the CHANGELOG.md.
  • [ ] I have updated the documentation accordingly.
  • [ ] I have read the CONTRIBUTING document.
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.

divine7022 avatar Dec 07 '25 20:12 divine7022