pyaerocom icon indicating copy to clipboard operation
pyaerocom copied to clipboard

First draft of new climatology config

Open dulte opened this issue 1 year ago • 2 comments

Change Summary

Adds new way of defining how the climatology is defined. Instead of just a bool to turn climatology off and on, a climatology config can now be passed into the colocation setup.

This is a reimplementation of @Ovewh 's PR #1135. As pyaercom has changed as much as it has, it was easier to do everything from scratch.

Related issue number

Aims to fix #1125

Checklist

  • [ ] Start with a draft-PR
  • [ ] The PR title is a good summary of the changes
  • [ ] PR is set to AeroTools and a tentative milestone
  • [ ] Documentation reflects the changes where applicable
  • [ ] Tests for the changes exist where applicable
  • [ ] Tests pass locally
  • [ ] Tests pass on CI
  • [ ] At least 1 reviewer is selected
  • [ ] Make PR ready to review

dulte avatar Sep 30 '24 12:09 dulte

Two tests fail due to different mean obs than expected with new climatology.

dulte avatar Sep 30 '24 12:09 dulte

Two tests fail due to different mean obs than expected with new climatology.

I remember climatology previously was hard coded, in the pyaerocom const to be defined as the mean between 2005- 2015, so if you use different period it might cause the test failing.

The calc_climatology function in helpers.py will set the year of the climatology unless defined (the set_year keyword), to be the middle of the the period which means that unless the year of the model data you want to compare is also that year the collocation will fail. This also causes issues if you like extend the period for which the climatology is defined i.e. from default of 2005-2015 to being 2005 - 2020. Then the time dimension of the gridded data object also would have to update if the collocation is to succeed.

Ovewh avatar Sep 30 '24 14:09 Ovewh

Codecov Report

Attention: Patch coverage is 86.66667% with 6 lines in your changes missing coverage. Please review.

Project coverage is 77.96%. Comparing base (c52cd91) to head (901272c). Report is 762 commits behind head on main-dev.

Files with missing lines Patch % Lines
pyaerocom/stationdata.py 50.00% 3 Missing :warning:
pyaerocom/aeroval/_processing_base.py 50.00% 1 Missing :warning:
pyaerocom/climatology_config.py 94.11% 1 Missing :warning:
pyaerocom/colocation/colocation_setup.py 91.66% 1 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##           main-dev    #1356      +/-   ##
============================================
+ Coverage     77.95%   77.96%   +0.01%     
============================================
  Files           144      145       +1     
  Lines         21563    21602      +39     
============================================
+ Hits          16810    16843      +33     
- Misses         4753     4759       +6     
Flag Coverage Δ
unittests 77.96% <86.66%> (+0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Oct 11 '24 05:10 codecov[bot]

The hours for this effort can be put on Arbeitsordre 285056000 (NFR- INES2 project) according to @MichaelSchulzMETNO

jgriesfeller avatar Jan 15 '25 11:01 jgriesfeller