satpy icon indicating copy to clipboard operation
satpy copied to clipboard

Unexpected floats when reading LI L2 LFL data

Open gerritholl opened this issue 1 year ago • 1 comments

Describe the bug

When reading LI L2 LFL data, several datasets are returned as floats when they should be integers.

To Reproduce

from satpy import Scene
from glob import glob
from satpy.utils import debug_on; debug_on()

fn = "/media/nas/x23352/MTG/LI/L2/2024/07/01/10/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LFL--FD--CHK-BODY---NC4E_C_EUMT_20240701105944_L2PF_OPE_20240701105923_20240701105933_N__T_0066_0057.nc"
sc = Scene(filenames=[fn], reader="li_l2_nc")
sc.load(["flash_id"])
print(sc["flash_id"].dtype)

Expected behavior

I expect an integer dtype.

Actual results

I get float64.

[DEBUG: 2024-07-10 16:42:30 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/li_l2_nc.yaml',)
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.yaml_reader] Assigning to li_l2_nc: ['/media/nas/x23352/MTG/LI/L2/2024/07/01/10/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LFL--FD--CHK-BODY---NC4E_C_EUMT_20240701105944_L2PF_OPE_20240701105923_20240701105933_N__T_0066_0057.nc']
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.li_base_nc] Product type is: 2-LFL
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.li_base_nc] Adding 12 datasets for 2-LFL input product.
[DEBUG: 2024-07-10 16:42:30 : satpy.readers.li_l2_nc] The current product 2-LFL is not an accumulated product so it will not be regridded.
[DEBUG: 2024-07-10 16:42:30 : satpy.composites.config_loader] Looking for composites config file li.yaml
[DEBUG: 2024-07-10 16:42:30 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2024-07-10 16:42:30 : pyorbital.tlefile] Path to the Pyorbital configuration (where e.g. platforms.txt is found): /data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/pyorbital/etc
[DEBUG: 2024-07-10 16:42:31 : satpy.readers.yaml_reader] No coordinates found for DataID(name='longitude', modifiers=())
[DEBUG: 2024-07-10 16:42:31 : satpy.readers.yaml_reader] No coordinates found for DataID(name='latitude', modifiers=())
float64

Environment Info:

  • OS: [e.g. OSX, Windows, Linux]
  • Satpy Version: main (v0.49.0-156-g3a742c297)

Additional context

The same for other integer variables in the dataset.

Sometimes this happens when a fill value is defined in the variable attributes, but that is not the case here:

	uint flash_id(flashes) ;
		flash_id:_Unsigned = "true" ;
		flash_id:long_name = "LI L2 Flash IDs" ;

gerritholl avatar Jul 10 '24 14:07 gerritholl

@ameraner ^

mraspaud avatar Jul 29 '24 08:07 mraspaud

@ameraner @gerritholl this issue happens in this method this line bellow makes the transformation

data_array = (data_array * scale_factor) + add_offset

whith the following type

  • data_array : uint32
  • scale_factor : float32

Or with python the multiplication of a uint32 with a float32 get a float64

I suggest that we add this modification into the apply_use_rescaling method

if data_array.dtype == "uint32":
    data_array = data_array.astype(type(scale_factor))

ClementLaplace avatar Nov 21 '24 14:11 ClementLaplace

Is this also the case for flash_id specifically? I am surprised it has a scale factor.. what is the value of it?

ameraner avatar Nov 21 '24 14:11 ameraner