pcmdi_metrics
pcmdi_metrics copied to clipboard
PCMDI
Update MoV code to use xCDAT
Progress note:
- Reproduced demo notebook with new code, and results are consistent.
- Rigorous test: code conducted for all models, and found calendar related issue
- Next step: fix the calendar related issue
Progress note:
- Calendar issue fixed
- Rigorous testing (running for all models) in progress
Progress note: Rigor test -- inconsistency found for PDO
- land masking error for a few models -- found data error (lon inconsistent between sftlf and ts) -- workaround applied
- error in pcor (stat larger than 1) -- need to investigate
Progress note: the error in pcor that became over 1 is caused by doing regrid2 for field filled NaN (e.g., landmask) which results field filled with zero.
Related xcdat issue: https://github.com/xCDAT/xcdat/issues/580 Resolved by: https://github.com/xCDAT/xcdat/pull/533
After updating xcdat, pcor now look more reasonable in a simple case test.
Solution for now: keep xcdat version as its latest main branch Permanent solution: once xcdat new version releases, update pmp dependency version in dev.yml accordingly.
Progress note: essential dependency update: xcdat, 0.6.1 to 0.7.0, due to update of regrid2 for its nan value handling
Progress note: sign flip issue (which was revealed from RMSE for SAM, left in below figure) is now resolved (right).
Bug in centered RMSE calcaulation (left) is resolved (right).
More results: PMP_MoV_xCDAT_rigor_test_20240422.pdf
Progress note: next component that is needed is the custom season capability
@acordonez when you have a chance could you please check if the demo4 notebook is working fine with this PR?
@lee1043 The demo notebook ran successfully. Is it correct that some figures were removed from Section 3.3.2?
@lee1043 The demo notebook ran successfully. Is it correct that some figures were removed from Section 3.3.2?
@acordonez thanks for checking. Yes, that is correct. For AMO I removed monthly example and kept only yearly example which likely be more interest.
@acordonez thanks for reviewing. I also asked Alex (LANL) and Shiheng (LLNL) for testing this code for their analysis. I am holding merge for their result.
Test results from Alex: test_AJ_LANL_20240427.pdf "Everything worked without errors. The results look the same, although they are not identical (the 3rd NAO EOF came out to have a negative center of activity with my old code and a positive one with the new code version. Patterns look the same though."
My comment: Seeing consistent results between the old and new codes is great, although the sign reversion of the EOF3. Unless that reversion would significantly impact your analysis, I don’t worry too much about that as the EOF signs are determined arbitrarily even with a very subtle difference in the input. I suspect the change in building block (CDMS to xCDAT) may have resulted difference in higher decimal numbers. Relevant discussion can be found here.