satpy icon indicating copy to clipboard operation
satpy copied to clipboard

Add reader for FY-4B / GHI data

Open simonrp84 opened this issue 3 years ago • 10 comments

This PR adds support for the Geostationary High-speed Imager aboard FengYun-4B. Overall it's quite similar to the existing AGRI reader but has some specific modifications to deal with the geolocation for GHI (which only samples a small portion of the full disk).

  • [x] Tests added
  • [x] Fully documented

simonrp84 avatar Jun 13 '22 20:06 simonrp84

Codecov Report

Merging #2125 (6cdcb51) into main (fcbc517) will increase coverage by 0.05%. The diff coverage is 99.60%.

@@            Coverage Diff             @@
##             main    #2125      +/-   ##
==========================================
+ Coverage   94.25%   94.30%   +0.05%     
==========================================
  Files         300      306       +6     
  Lines       45638    46039     +401     
==========================================
+ Hits        43014    43416     +402     
+ Misses       2624     2623       -1     
Flag Coverage Δ
behaviourtests 4.62% <0.00%> (-0.04%) :arrow_down:
unittests 94.94% <99.60%> (+0.04%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
satpy/readers/fy4_base.py 98.61% <98.61%> (ø)
satpy/composites/agri.py 100.00% <100.00%> (ø)
satpy/readers/agri_l1.py 100.00% <100.00%> (+1.43%) :arrow_up:
satpy/readers/ghi_l1.py 100.00% <100.00%> (ø)
satpy/tests/compositor_tests/test_agri.py 100.00% <100.00%> (ø)
satpy/tests/reader_tests/test_agri_l1.py 100.00% <100.00%> (ø)
satpy/tests/reader_tests/test_fy4_base.py 100.00% <100.00%> (ø)
satpy/tests/reader_tests/test_ghi_l1.py 100.00% <100.00%> (ø)
satpy/tests/modifier_tests/test_angles.py 100.00% <0.00%> (ø)
satpy/tests/reader_tests/test_hy2_scat_l2b_h5.py 100.00% <0.00%> (ø)
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Jun 13 '22 20:06 codecov[bot]

pre-commit.ci autofix

simonrp84 avatar Jun 13 '22 20:06 simonrp84

Coverage Status

Coverage increased (+0.05%) to 94.891% when pulling 6cdcb51d4b50b01d5a3e7f21a16632b2d56a95b5 on simonrp84:fy4b_ghi into fcbc517578c7b8a3e206ec9b48d2f1cb4239ff87 on pytroll:main.

coveralls avatar Jun 14 '22 00:06 coveralls

pre-commit.ci autofix

simonrp84 avatar Jun 14 '22 15:06 simonrp84

The FY4B document is available online now. The 250m COFF, CFAC, LOFF, and LFC need to be updated.

zxdawn avatar Jun 25 '22 09:06 zxdawn

Thanks. Looks like my guesstimates were not too bad ;-) I'll update the reader early next week.

simonrp84 avatar Jun 25 '22 13:06 simonrp84

pre-commit.ci autofix

simonrp84 avatar Jun 27 '22 12:06 simonrp84

pre-commit.ci autofix

simonrp84 avatar Jun 27 '22 15:06 simonrp84

As mentioned on slack, after talking with @simonrp84 and Kathy Strabala I'm starting to think that it would be better to have two readers as @simonrp84 originally had it. Simon pointed out that a user doing:

scn = Scene(reader='agri_l1', filenames...)
scn.load(["C10"])

and having to know what "C10" they were requesting is too confusing. It should be more explicit than that. Despite what we would have liked as far as instrument naming and design (not changing channel properties for the same channel name), we have what we have and have to live with it. I think being more explicit with two separate readers (with names that match the reader naming conventions) is best. @mraspaud rebuttal?

djhoese avatar Aug 25 '22 15:08 djhoese

I'm not so much against it actually. It's unfortunate, but maybe it's for the best. As long as the file patterns are clearly different to allow find_fles_and_readers to do its job properly, then I'm willing to see what you come up with :)

mraspaud avatar Aug 26 '22 12:08 mraspaud

Any news on this?

mraspaud avatar Nov 09 '22 15:11 mraspaud

Thanks for the comments @djhoese. There certainly is a lot in here - turned into a far bigger job than expected.

Have updated the geolocation and the tests and have confirmed accuracy by comparing output images to coastlines, so hopefully all in order. I know the separation of AGRI on FY4A and FY4B isn't ideal, but can't think of a better solution.

simonrp84 avatar Nov 10 '22 15:11 simonrp84

I haven't finished this, but have run out of time to work on it - can't guarantee I'll be able to come back to it for some time so hopefully someone else can pick it up. Am happy to supply data.

GHI seems pretty much OK. AGRI is causing issues as:

  1. Different channels have different area extents, which is causing the native resampler to fail if reduce_data=True
  2. The areas don't quite match coastlines, particularly for FY-4A/AGRI. I'm unsure if this is due to poor L1 data, poor area definition or both.

simonrp84 avatar Nov 10 '22 22:11 simonrp84

FWIW, my feeling is to edit the LOFF/COFF values supplied by CMA as I don't trust them. If we make those factors of each other (they're presently not) then the native resampler problem at least should go away and, hopefully, it would also improve geolocation.

simonrp84 avatar Nov 10 '22 22:11 simonrp84