VII Reader Fixes and VII Composites Updates
Updates the VII reader stack so recent changes are fully covered and exercised. The reader now leaves the solar-zenith correction to the compositor (i.e., correction was previously performed twice), tolerates multiple filename layouts for L1B NetCDF products, and parses a wider range of sensing start/end timestamps attributes inside NetCDF. Adds the new VII daytime-only and night-time composites along with their YAML wiring, refreshes the reader unit tests to match the calibration and timestamp updates.
- [x] Contributes to #3202 (not closing it yet due to pending name change)
- [x] Unit tests for VII readers updated
- [x] AUTHORS.md updated
pre-commit.ci autofix
When I test this with VII_L1B_CrossRef_TDP_V2 it complains about missing key data/calibration_data/band_averaged_solar_irradiance. I can comment that out as a workaround...
However it always crashes with a Segmentation Fault when I try to write an image. Maybe this isn't a fault of the reader, I don't know, but it doesn't crash with any other data. I've tried these tricks and it still crashes:
ulimit -n 4096
export DASK_NUM_WORKERS=1
export OMP_NUM_THREADS=1
As discussed, we should also apply the name change from vii_* to metimage_* (in this PR or another one). We should also update the names here then https://github.com/pytroll/satpy/blob/main/doc/source/examples/vii_l1b_nc.rst.
When I test this with
VII_L1B_CrossRef_TDP_V2it complains about missing keydata/calibration_data/band_averaged_solar_irradiance. I can comment that out as a workaround...However it always crashes with a Segmentation Fault when I try to write an image. Maybe this isn't a fault of the reader, I don't know, but it doesn't crash with any other data. I've tried these tricks and it still crashes:
ulimit -n 4096 export DASK_NUM_WORKERS=1 export OMP_NUM_THREADS=1
Hi @howff , the KeyError is an incompatibility with the currently public test data. We are now adapting the reader so that it will work with real data. The NetCDF issue is a tricky one - let's keep that discussion in the other issue that you found already.
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 96.34%. Comparing base (293e54e) to head (2554db5).
:warning: Report is 36 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #3277 +/- ##
==========================================
+ Coverage 96.32% 96.34% +0.01%
==========================================
Files 463 463
Lines 58896 58916 +20
==========================================
+ Hits 56729 56760 +31
+ Misses 2167 2156 -11
| Flag | Coverage Δ | |
|---|---|---|
| behaviourtests | 3.60% <0.00%> (+<0.01%) |
:arrow_up: |
| unittests | 96.43% <100.00%> (+0.01%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Codecov is marking that we're not testing the ValueError for unrecognised timestamps - that should be an easy addition, could you maybe quickly add it?
EDIT: I just commited this change myself
Pull Request Test Coverage Report for Build 19863014968
Details
- 0 of 0 changed or added relevant lines in 0 files are covered.
- No unchanged relevant lines lost coverage.
- Overall first build on main at 96.415%
| Totals | |
|---|---|
| Change from base Build 19840971577: | 96.4% |
| Covered Lines: | 56611 |
| Relevant Lines: | 58716 |