satpy icon indicating copy to clipboard operation
satpy copied to clipboard

Orbital Parameters missing when reading Himawari ahi_hsd data

Open guidocioni opened this issue 9 months ago • 3 comments

Describe the bug When loading Himawari data the orbital_parameters attribute is not added, and this causes some issue if one has to for example compute the zenith angle.

To Reproduce

himawari_scene = Scene(
    reader="ahi_hsd",
    filenames=files_himawari,
)
himawari_scene.load(["natural_color"])
scn_himawari_r = himawari_scene.resample(
    "worldeqc3km",
)

get_satellite_zenith_angle(scn_himawari_r["natural_color"]) / 90.0

causes a KeyError

KeyError: "Unable to determine satellite position. Either the reader doesn't provide that information or geolocation datasets were not available."
Image

As workaround one can do

scn_himawari_r['natural_color'].attrs["orbital_parameters"] = {
    "projection_longitude": 140.7,
    "projection_latitude": 0.0,
    "projection_altitude": 35786023.0,
    "satellite_nominal_latitude": 0.0,
    "satellite_nominal_longitude": 140.7,
    "satellite_nominal_altitude": 35786023.4375,
    "yaw_flip": False,
}

Environment Info:

  • Satpy Version: 0.55.0
  • PyResample Version: 1.33.0
  • Readers and writers dependencies (when relevant):
  • Readers ======= abi_l1b: ok abi_l1b_scmi: ok abi_l2_nc: ok acspo: ok agri_fy4a_l1: ok agri_fy4b_l1: ok ahi_hrit: ok ahi_hsd: ok ahi_l1b_gridded_bin: ok ahi_l2_nc: ok ami_l1b: ok amsr2_l1b: ok amsr2_l2: ok amsr2_l2_gaasp: ok amsub_l1c_aapp: ok ascat_l2_soilmoisture_bufr: cannot find module 'satpy.readers.ascat_l2_soilmoisture_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'"))) atms_l1b_nc: ok atms_sdr_hdf5: ok avhrr_l1b_aapp: ok avhrr_l1b_eps: ok avhrr_l1b_gaclac: cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac') avhrr_l1b_hrpt: ok avhrr_l1c_eum_gac_fdr_nc: ok aws1_mwr_l1b_nc: ok aws1_mwr_l1c_nc: ok caliop_l2_cloud: cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf') camel_l3_nc: ok clavrx: cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf') cmsaf-claas2_l2_nc: ok electrol_hrit: ok epic_l1b_h5: ok eps_sterna_mwr_l1b_nc: ok fci_l1c_nc: ok fci_l2_bufr: cannot find module 'satpy.readers.eum_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes) fci_l2_grib: cannot find module 'satpy.readers.eum_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes) fci_l2_nc: ok fy3a_mersi1_l1b: ok fy3b_mersi1_l1b: ok fy3c_mersi1_l1b: ok generic_image: ok geocat: ok gerb_l2_hr_h5: ok ghi_l1: ok ghrsst_l2: ok gld360_ualf2: ok glm_l2: ok gms5-vissr_l1b: cannot find module 'satpy.readers.gms.gms5_vissr_l1b' (No module named 'numba') goci2_l2_nc: ok goes-imager_hrit: ok goes-imager_nc: ok gpm_imerg: ok grib: ok hsaf_grib: ok hsaf_h5: ok hy2_scat_l2b_h5: ok iasi_l2: ok iasi_l2_cdr_nc: ok iasi_l2_so2_bufr: cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n Error: ', ModuleNotFoundError("No module named 'eccodes'"))) ici_l1b_nc: ok insat3d_img_l1b_h5: ok jami_hrit: ok li_l2_nc: ok maia: ok mcd12q1: cannot find module 'satpy.readers.mcd12q1' (No module named 'pyhdf') meris_nc_sen3: ok mersi2_l1b: ok mersi3_l1b: ok mersi_ll_l1b: ok mersi_rm_l1b: ok mhs_l1c_aapp: ok mimicTPW2_comp: ok mirs: ok modis_l1b: cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf') modis_l2: cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf') modis_l3: cannot find module 'satpy.readers.modis_l3' (No module named 'pyhdf') msi_safe: ok msi_safe_l2a: ok msu_gsa_l1b: ok mtsat2-imager_hrit: ok multiple_sensors_isccpng_l1g_nc: ok mviri_l1b_fiduceo_nc: ok mwi_l1b_nc: ok mws_l1b_nc: ok nucaps: ok nwcsaf-geo: ok nwcsaf-msg2013-hdf5: ok nwcsaf-pps_nc: ok oceancolorcci_l3_nc: ok oci_l2_bgc: cannot find module 'satpy.readers.seadas_l2' (No module named 'pyhdf') olci_l1b: ok olci_l2: ok oli_tirs_l1_tif: ok omps_edr: ok osisaf_nc: ok safe_sar_l2_ocn: ok sar-c_safe: ok satpy_cf_nc: ok scatsat1_l2b: cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/Users/guidocioni/miniforge3/envs/default/lib/python3.12/site-packages/satpy/dataset/init.py)) seadas_l2: cannot find module 'satpy.readers.seadas_l2' (No module named 'pyhdf') seviri_l1b_hrit: ok seviri_l1b_icare: cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf') seviri_l1b_native: ok seviri_l1b_nc: ok seviri_l2_bufr: cannot find module 'satpy.readers.eum_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes) seviri_l2_grib: cannot find module 'satpy.readers.eum_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes) sgli_l1b: ok slstr_l1b: ok smos_l2_wind: ok tropomi_l2: ok vii_l1b_nc: ok vii_l2_nc: ok viirs_compact: ok viirs_edr: ok viirs_edr_active_fires: ok viirs_edr_flood: cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf') viirs_l1b: ok viirs_l2: ok viirs_sdr: ok viirs_vgac_l1c_nc: ok virr_l1b: ok

Writers

awips_tiled: ok cf: ok geotiff: ok mitiff: ok ninjogeotiff: ok ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff') simple_image: ok

Versions

platform: macOS-15.4-arm64-arm-64bit python: 3.12.9

cartopy: 0.24.0 dask: 2025.3.0 fsspec: 2025.3.2 gdal: not installed geoviews: not installed h5netcdf: 1.6.1 h5py: 3.13.0 netcdf4: 1.7.2 numpy: 2.2.4 pyhdf: not installed pyproj: 3.6.1 rasterio: 1.4.3 xarray: 2025.3.1

guidocioni avatar Apr 10 '25 18:04 guidocioni

The orbital parameters exist in the individual reader-based DataArrays, but are removed during RGB creation. This happens when metadata is merged, but the merging discovers that the orbital parameters are not equivalent between the various bands and/or segments. There are many issues related to this, but I do not remember the current best practice or future planned decisions to resolve the issues. If I remember correctly it is not a very easy problem to solve. I'm sorry I do not have the time right now to search for all the related issues.

djhoese avatar Apr 10 '25 19:04 djhoese

As @djhoese said, the issue is in the merging of the single channel DataArrays into an RGB DataArray, as the attributes that are not identical between arrays are discarded during the merge. So feel free to investigate more and see if there is a pragmatic solution to the issue!

mraspaud avatar Apr 11 '25 09:04 mraspaud

Here is one of the primary PRs explaining the issue:

https://github.com/pytroll/satpy/pull/2573

Another fun issue on the topic:

https://github.com/pytroll/satpy/issues/2012

The summary is that AHI's satellite position is not consistent between bands and not even consistent between segments of a single band. Satpy merges the orbital parameters of a series of segments when it merges them together into a single DataArray so there is some control there over what is produced as the final satellite position. However, there isn't really anything we can do between bands. I think the case you're running into is where the composite generation is merging the metadata between RGB inputs and finding that the orbital parameters are not equal so not passing any of them along to the final RGB's metadata.

djhoese avatar Apr 11 '25 14:04 djhoese