netcdf4-python
netcdf4-python copied to clipboard
Unidata
netCDF4 unable to open netCDF file that opens as HDF and with Panoply
The version of the software with which you are encountering an issue
h5netcdf version: 1.2.0 libnetcdf version: 4.9.2 netCDF4 version: 1.6.5 xarray version: 2024.2.0
Environmental information (i.e. Operating System, compiler info, java version, python version, etc.)
OS: MacOS 14.3.1 Chip: Apple M3 Pro Python version: 3.11.8
A description of the issue with the steps needed to reproduce it
The following error is raised when attempting to open the file (linked below) via the netCDF4 python library:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[10], line 1
----> 1 nc.Dataset(filepath)
File src[/netCDF4/_netCDF4.pyx:2495](http://localhost:8888/netCDF4/_netCDF4.pyx#line=2494), in netCDF4._netCDF4.Dataset.__init__()
File src[/netCDF4/_netCDF4.pyx:1891](http://localhost:8888/netCDF4/_netCDF4.pyx#line=1890), in netCDF4._netCDF4._get_grps()
File src[/netCDF4/_netCDF4.pyx:3616](http://localhost:8888/netCDF4/_netCDF4.pyx#line=3615), in netCDF4._netCDF4.Group.__init__()
File src[/netCDF4/_netCDF4.pyx:2001](http://localhost:8888/netCDF4/_netCDF4.pyx#line=2000), in netCDF4._netCDF4._get_vars()
AttributeError: 'NoneType' object has no attribute 'dimensions'
Here is where the file can be downloaded. It does require a free NASA Earthdata login account to access, but the file size even when compressed (51 MB) exceeds the GitHub issue limit of 25 MB so I couldn't upload it here.
Interestingly, as an HDF5, the file appears OK: The whole file can be opened without error by the underlying library libhdf5, as well as h5dump, h5netcdf, and the myHDF5 tool. However, the irradiance and irradiance_error variables — which exist in multiple groups, e.g., /frames/band_290_490_nm/ — do raise errors when the HDFView application attempts to open them.
Also, interestingly, the NASA Panoply data viewer opens the file without issue as well.
I'm in a conda environment, which has libnetcdf version 4.9.2 installed.
I can reproduce this problem (on my Mac M2) in Python 3.12:
# Name Version Build Channel
libnetcdf 4.9.2 nompi_h291a7c2_113 conda-forge
netcdf4 1.6.5 nompi_py312h9035142_100 conda-forge
While ncdump -h works without error, this was interesting:
❯ ncdump TEMPO_IRR_L1_V02_20240314T041808Z.nc > /dev/null
NetCDF: Start+count exceeds dimension bound
Location: file ?; fcn ? line 478
ncdump -sh output if anyone is curious:
netcdf TEMPO_IRR_L1_V02_20240314T041808Z {
dimensions:
mirror_step = 1 ;
variables:
int mirror_step(mirror_step) ;
mirror_step:long_name = "scan mirror position index" ;
mirror_step:_Storage = "contiguous" ;
mirror_step:_Endianness = "little" ;
double time(mirror_step) ;
time:standard_name = "time" ;
time:long_name = "exposure start time" ;
time:calendar = "gregorian" ;
time:units = "seconds since 1980-01-06T00:00:00Z" ;
time:comment = "Average of /frames/time" ;
time:_Storage = "contiguous" ;
time:_Endianness = "little" ;
float exposure_time(mirror_step) ;
exposure_time:units = "seconds" ;
exposure_time:long_name = "exposure duration" ;
exposure_time:valid_min = 0.f ;
exposure_time:valid_max = 10.f ;
exposure_time:_FillValue = 9.96921e+36f ;
exposure_time:comment = "Average of /frames/exposure_time" ;
exposure_time:_Storage = "contiguous" ;
exposure_time:_Endianness = "little" ;
float solar_phi(mirror_step) ;
solar_phi:units = "degrees" ;
solar_phi:long_name = "solar boresight azimuthal angle" ;
solar_phi:valid_min = -180.f ;
solar_phi:valid_max = 180.f ;
solar_phi:_FillValue = 9.96921e+36f ;
solar_phi:comment = "Average of /frames/solar_phi" ;
solar_phi:_Storage = "contiguous" ;
solar_phi:_Endianness = "little" ;
float solar_theta(mirror_step) ;
solar_theta:units = "degrees" ;
solar_theta:long_name = "solar boresight polar angle" ;
solar_theta:valid_min = 0.f ;
solar_theta:valid_max = 180.f ;
solar_theta:_FillValue = 9.96921e+36f ;
solar_theta:comment = "Average of /frames/solar_theta" ;
solar_theta:_Storage = "contiguous" ;
solar_theta:_Endianness = "little" ;
float earth_sun_distance ;
earth_sun_distance:units = "m" ;
earth_sun_distance:long_name = "Earth-sun distance" ;
earth_sun_distance:_Storage = "contiguous" ;
earth_sun_distance:_Endianness = "little" ;
// global attributes:
:time_reference = "1980-01-06T00:00:00Z" ;
:format_version = 2 ;
:product_type = "IRR" ;
:processing_level = "1" ;
:sdpc_version = "TEMPO_SDPC_v4.3.0" ;
:processing_version = 2 ;
:production_date_time = "2024-03-14T04:39:37Z" ;
:local_granule_id = "TEMPO_IRR_L1_V02_20240314T041808Z.nc" ;
:version_id = 2 ;
:pge_version = "0.1.0" ;
:shortname = "TEMPO_IRR_L1" ;
:begin_date = "2024-03-14" ;
:begin_time = "04:18:08" ;
:end_date = "2024-03-14" ;
:end_time = "04:18:11" ;
:time_coverage_start = "2024-03-14T04:18:08Z" ;
:time_coverage_end = "2024-03-14T04:18:11Z" ;
:time_coverage_start_since_epoch = 1394425106.5535 ;
:time_coverage_end_since_epoch = 1394425109.21705 ;
:history = "2024-03-14T04:39:44Z: /tempo/nas0/sdpc_soft/install/gnu-rh8/sdpc/v4p3/bin/L0_ccd -vv --Version 2 --config l0_ccd.cfg -i @hk.lis --trend trend_params.nc -d /tempo/nas0/sdpc_archive/temposdpc/ops3/L1/DRK/D16138/20240314T041121Z/TEMPO_DRK_L1_V02_20240314T041121Z.nc -o TEMPO_IRR_L1_V02_20240314T041808Z.nc TEMPO_IRR_L0_V01_20240314T041808Z.nc\n2024-03-14T04:57:25Z: /tempo/nas0/sdpc_soft/install/gnu-rh8/sdpc/v4p3/bin/wavecal_merge --meta --delete -t TEMPO_IRR_L1_V02_20240314T041808Z.nc wavecal_TEMPO_IRR_L1_V02_20240314T041808Z\n" ;
string :input_files = "TEMPO_IRR_L0_V01_20240314T041808Z.nc", "TEMPO_HK_L0_V01_20240314T040946Z.nc", "TEMPO_HK_L0_V01_20240314T041410Z.nc", "TEMPO_HK_L0_V01_20240314T041840Z.nc", "TEMPO_HK_L0_V01_20240314T042310Z.nc", "TEMPO_DRK_L1_V02_20240314T041121Z.nc" ;
:Conventions = "CF-1.6, ACDD-1.3" ;
:day_of_year = 74LL ;
:project = "TEMPO" ;
:platform = "Intelsat 40e" ;
:source = "UV-VIS hyperspectral imaging" ;
:institution = "Smithsonian Astrophysical Observatory" ;
:creator_url = "http://tempo.si.edu" ;
:title = "TEMPO Level 1 irradiance measurement" ;
:collection_shortname = "TEMPO_IRR_L1" ;
:collection_version = 1LL ;
:keywords = "EARTH SCIENCE>ATMOSPHERE>ATMOSPHERIC RADIATION>SOLAR IRRADIANCE" ;
:summary = "Level 1 irradiance files provide solar irradiance measured using the working solar diffuser. Each file includes the measured solar irradiance for all the North-South cross-track pixels. The files are provided in netCDF4 format, and contain information on radiometrically and wavelength calibrated solar irradiance for the UV and visible bands, corresponding noise, parameterized wavelength grid, solar viewing geometry, quality flags and other ancillary information. The product is produced using the L0-1b processor which includes multiple steps: (1) Image processing to produce radiometrically calibrated radiance, and (2) Additional wavelength calibration to improve wavelength registration. Please refer to the ATBD for details." ;
:coremetadata = "\nGROUP = INVENTORYMETADATA\n GROUPTYPE = MASTERGROUP\n\n GROUP = ECSDATAGRANULE\n\n OBJECT = LOCALGRANULEID\n NUM_VAL = 1\n VALUE = "TEMPO_IRR_L1_V02_20240314T041808Z.nc"\n END_OBJECT = LOCALGRANULEID\n\n OBJECT = LOCALVERSIONID\n NUM_VAL = 1\n VALUE = ("RFC1321 MD5 = not yet calculated")\n END_OBJECT = LOCALVERSIONID\n\n OBJECT = PRODUCTIONDATETIME\n NUM_VAL = 1\n VALUE = "2024-03-14T04:39:37Z"\n END_OBJECT = PRODUCTIONDATETIME\n\n END_GROUP = ECSDATAGRANULE\n\n GROUP = COLLECTIONDESCRIPTIONCLASS\n\n OBJECT = SHORTNAME\n NUM_VAL = 1\n VALUE = "TEMPO_IRR_L1"\n END_OBJECT = SHORTNAME\n\n OBJECT = VERSIONID\n NUM_VAL = 1\n VALUE = 2\n END_OBJECT = VERSIONID\n\n END_GROUP = COLLECTIONDESCRIPTIONCLASS\n\n GROUP = INPUTGRANULE\n\n OBJECT = INPUTPOINTER\n NUM_VAL = 6\n VALUE = ("TEMPO_IRR_L0_V01_20240314T041808Z.nc", "TEMPO_HK_L0_V01_20240314T040946Z.nc", "TEMPO_HK_L0_V01_20240314T041410Z.nc", "TEMPO_HK_L0_V01_20240314T041840Z.nc", "TEMPO_HK_L0_V01_20240314T042310Z.nc", "TEMPO_DRK_L1_V02_20240314T041121Z.nc")\n END_OBJECT = INPUTPOINTER\n\n END_GROUP = INPUTGRANULE\n\n GROUP = SPATIALDOMAINCONTAINER\n OBJECT = GRANULESPATIALREPRESENTATION\n CLASS = "1"\n NUM_VAL = 1\n VALUE = "NO_SPATIAL"\n END_OBJECT = GRANULESPATIALREPRESENTATION\n END_GROUP = SPATIALDOMAINCONTAINER\n\n GROUP = RANGEDATETIME\n\n OBJECT = RANGEENDINGDATE\n NUM_VAL = 1\n VALUE = "2024-03-14"\n END_OBJECT = RANGEENDINGDATE\n\n OBJECT = RANGEENDINGTIME\n NUM_VAL = 1\n VALUE = "04:18:11"\n END_OBJECT = RANGEENDINGTIME\n\n OBJECT = RANGEBEGINNINGDATE\n NUM_VAL = 1\n VALUE = "2024-03-14"\n END_OBJECT = RANGEBEGINNINGDATE\n\n OBJECT = RANGEBEGINNINGTIME\n NUM_VAL = 1\n VALUE = "04:18:08"\n END_OBJECT = RANGEBEGINNINGTIME\n\n END_GROUP = RANGEDATETIME\n\n GROUP = PGEVERSIONCLASS\n\n OBJECT = PGEVERSION\n NUM_VAL = 1\n VALUE = "0.1.0"\n END_OBJECT = PGEVERSION\n\n END_GROUP = PGEVERSIONCLASS\n\nEND_GROUP = INVENTORYMETADATA\n\nEND\n" ;
:_NCProperties = "version=1|netcdflibversion=4.6.1|hdf5libversion=1.8.14" ;
:_SuperblockVersion = 0 ;
:_IsNetcdf4 = 1 ;
:_Format = "netCDF-4" ;
group: band_290_490_nm {
dimensions:
xtrack = 2048 ;
spectral_channel = 1028 ;
xtrack = 2048 ;
variables:
int xtrack(xtrack) ;
xtrack:_Storage = "contiguous" ;
xtrack:_Endianness = "little" ;
float irradiance(mirror_step, xtrack, spectral_channel) ;
irradiance:units = "photons/s/cm^2/nm" ;
irradiance:ancillary_variables = "irradiance_error" ;
irradiance:_FillValue = -9.96921e+36f ;
irradiance:comment = "Average of /frames/band_290_490_nm/irradiance" ;
irradiance:_Storage = "chunked" ;
irradiance:_ChunkSizes = 1, 128, 1028 ;
irradiance:_Shuffle = "true" ;
irradiance:_DeflateLevel = 1 ;
irradiance:_Endianness = "little" ;
float irradiance_error(mirror_step, xtrack, spectral_channel) ;
irradiance_error:units = "photons/s/cm^2/nm" ;
irradiance_error:long_name = "irradiance error" ;
irradiance_error:_FillValue = -9.96921e+36f ;
irradiance_error:number_of_significant_digits = 2 ;
irradiance_error:comment = "Root mean square of /frames/band_290_490_nm/irradiance_error" ;
irradiance_error:_Storage = "chunked" ;
irradiance_error:_ChunkSizes = 1, 128, 1028 ;
irradiance_error:_Shuffle = "true" ;
irradiance_error:_DeflateLevel = 1 ;
irradiance_error:_Endianness = "little" ;
float nominal_wavelength(xtrack, spectral_channel) ;
nominal_wavelength:units = "nm" ;
nominal_wavelength:valid_min = -9.96921e+36f ;
nominal_wavelength:valid_max = 9.96921e+36f ;
nominal_wavelength:_FillValue = 9.96921e+36f ;
nominal_wavelength:_Storage = "contiguous" ;
nominal_wavelength:_Endianness = "little" ;
ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;
pixel_quality_flag:long_name = "pixel quality flag" ;
pixel_quality_flag:_FillValue = 65535US ;
pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;
pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;
pixel_quality_flag:comment = "Logical OR of /frames/band_290_490_nm/pixel_quality_flag" ;
pixel_quality_flag:_Storage = "chunked" ;
pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;
pixel_quality_flag:_Shuffle = "true" ;
pixel_quality_flag:_DeflateLevel = 1 ;
pixel_quality_flag:_Endianness = "little" ;
float wavecal_params(mirror_step, xtrack, /xtrack) ;
wavecal_params:num_coefficients = 2 ;
wavecal_params:start_spectral_channel = 0 ;
wavecal_params:num_spectral_channels = 1028 ;
wavecal_params:_Storage = "contiguous" ;
wavecal_params:_Endianness = "little" ;
float sf_hw1e(mirror_step, xtrack, spectral_channel) ;
sf_hw1e:_Storage = "contiguous" ;
sf_hw1e:_Endianness = "little" ;
float sf_shape(mirror_step, xtrack, spectral_channel) ;
sf_shape:_Storage = "contiguous" ;
sf_shape:_Endianness = "little" ;
float sf_asym(mirror_step, xtrack, spectral_channel) ;
sf_asym:_Storage = "contiguous" ;
sf_asym:_Endianness = "little" ;
int wavecal_niter(mirror_step, xtrack) ;
wavecal_niter:_Storage = "contiguous" ;
wavecal_niter:_Endianness = "little" ;
int wavecal_opt_status(mirror_step, xtrack) ;
wavecal_opt_status:_Storage = "contiguous" ;
wavecal_opt_status:_Endianness = "little" ;
// group attributes:
} // group band_290_490_nm
group: band_540_740_nm {
dimensions:
xtrack = 2048 ;
spectral_channel = 1028 ;
mirror_step = 1 ;
variables:
int xtrack(xtrack) ;
xtrack:_Storage = "contiguous" ;
xtrack:_Endianness = "little" ;
float irradiance(mirror_step, xtrack, spectral_channel) ;
irradiance:units = "photons/s/cm^2/nm" ;
irradiance:ancillary_variables = "irradiance_error" ;
irradiance:_FillValue = -9.96921e+36f ;
irradiance:comment = "Average of /frames/band_540_740_nm/irradiance" ;
irradiance:_Storage = "chunked" ;
irradiance:_ChunkSizes = 1, 128, 1028 ;
irradiance:_Shuffle = "true" ;
irradiance:_DeflateLevel = 1 ;
irradiance:_Endianness = "little" ;
float irradiance_error(mirror_step, xtrack, spectral_channel) ;
irradiance_error:units = "photons/s/cm^2/nm" ;
irradiance_error:long_name = "irradiance error" ;
irradiance_error:_FillValue = -9.96921e+36f ;
irradiance_error:number_of_significant_digits = 2 ;
irradiance_error:comment = "Root mean square of /frames/band_540_740_nm/irradiance_error" ;
irradiance_error:_Storage = "chunked" ;
irradiance_error:_ChunkSizes = 1, 128, 1028 ;
irradiance_error:_Shuffle = "true" ;
irradiance_error:_DeflateLevel = 1 ;
irradiance_error:_Endianness = "little" ;
float nominal_wavelength(xtrack, spectral_channel) ;
nominal_wavelength:units = "nm" ;
nominal_wavelength:valid_min = -9.96921e+36f ;
nominal_wavelength:valid_max = 9.96921e+36f ;
nominal_wavelength:_FillValue = 9.96921e+36f ;
nominal_wavelength:_Storage = "contiguous" ;
nominal_wavelength:_Endianness = "little" ;
ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;
pixel_quality_flag:long_name = "pixel quality flag" ;
pixel_quality_flag:_FillValue = 65535US ;
pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;
pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;
pixel_quality_flag:comment = "Logical OR of /frames/band_540_740_nm/pixel_quality_flag" ;
pixel_quality_flag:_Storage = "chunked" ;
pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;
pixel_quality_flag:_Shuffle = "true" ;
pixel_quality_flag:_DeflateLevel = 1 ;
pixel_quality_flag:_Endianness = "little" ;
float wavecal_params(mirror_step, xtrack, /mirror_step) ;
wavecal_params:num_coefficients = 3 ;
wavecal_params:start_spectral_channel = 0 ;
wavecal_params:num_spectral_channels = 1028 ;
wavecal_params:_Storage = "contiguous" ;
wavecal_params:_Endianness = "little" ;
float sf_hw1e(mirror_step, xtrack, spectral_channel) ;
sf_hw1e:_Storage = "contiguous" ;
sf_hw1e:_Endianness = "little" ;
float sf_shape(mirror_step, xtrack, spectral_channel) ;
sf_shape:_Storage = "contiguous" ;
sf_shape:_Endianness = "little" ;
float sf_asym(mirror_step, xtrack, spectral_channel) ;
sf_asym:_Storage = "contiguous" ;
sf_asym:_Endianness = "little" ;
int wavecal_niter(mirror_step, xtrack) ;
wavecal_niter:_Storage = "contiguous" ;
wavecal_niter:_Endianness = "little" ;
int wavecal_opt_status(mirror_step, xtrack) ;
wavecal_opt_status:_Storage = "contiguous" ;
wavecal_opt_status:_Endianness = "little" ;
// group attributes:
} // group band_540_740_nm
group: frames {
dimensions:
mirror_step = 1 ;
variables:
int mirror_step(mirror_step) ;
mirror_step:long_name = "scan mirror position index" ;
mirror_step:_Storage = "contiguous" ;
mirror_step:_Endianness = "little" ;
double time(mirror_step) ;
time:standard_name = "time" ;
time:long_name = "exposure start time" ;
time:calendar = "gregorian" ;
time:units = "seconds since 1980-01-06T00:00:00Z" ;
time:_Storage = "contiguous" ;
time:_Endianness = "little" ;
float exposure_time(mirror_step) ;
exposure_time:units = "seconds" ;
exposure_time:long_name = "exposure duration" ;
exposure_time:valid_min = 0.f ;
exposure_time:valid_max = 10.f ;
exposure_time:_FillValue = 9.96921e+36f ;
exposure_time:_Storage = "contiguous" ;
exposure_time:_Endianness = "little" ;
float solar_phi(mirror_step) ;
solar_phi:units = "degrees" ;
solar_phi:long_name = "solar boresight azimuthal angle" ;
solar_phi:valid_min = -180.f ;
solar_phi:valid_max = 180.f ;
solar_phi:_FillValue = 9.96921e+36f ;
solar_phi:_Storage = "contiguous" ;
solar_phi:_Endianness = "little" ;
float solar_theta(mirror_step) ;
solar_theta:units = "degrees" ;
solar_theta:long_name = "solar boresight polar angle" ;
solar_theta:valid_min = 0.f ;
solar_theta:valid_max = 180.f ;
solar_theta:_FillValue = 9.96921e+36f ;
solar_theta:_Storage = "contiguous" ;
solar_theta:_Endianness = "little" ;
float earth_sun_distance ;
earth_sun_distance:units = "m" ;
earth_sun_distance:long_name = "Earth-sun distance" ;
earth_sun_distance:_Storage = "contiguous" ;
earth_sun_distance:_Endianness = "little" ;
// group attributes:
:time_reference = "1980-01-06T00:00:00Z" ;
:Conventions = "CF-1.6, ACDD-1.3" ;
:format_version = 2 ;
:product_type = "IRR" ;
:processing_level = "1" ;
:sdpc_version = "TEMPO_SDPC_v4.3.0" ;
:time_coverage_start = "2024-03-14T04:18:08Z" ;
:time_coverage_end = "2024-03-14T04:18:11Z" ;
:processing_version = 2 ;
:time_coverage_start_since_epoch = 1394425106.5535 ;
:time_coverage_end_since_epoch = 1394425109.21705 ;
group: band_290_490_nm {
dimensions:
xtrack = 2048 ;
spectral_channel = 1028 ;
variables:
int xtrack(xtrack) ;
xtrack:_Storage = "contiguous" ;
xtrack:_Endianness = "little" ;
float irradiance(mirror_step, xtrack, spectral_channel) ;
irradiance:units = "photons/s/cm^2/nm" ;
irradiance:ancillary_variables = "irradiance_error" ;
irradiance:_FillValue = -9.96921e+36f ;
irradiance:_Storage = "chunked" ;
irradiance:_ChunkSizes = 1, 128, 1028 ;
irradiance:_Shuffle = "true" ;
irradiance:_DeflateLevel = 1 ;
irradiance:_Endianness = "little" ;
float irradiance_error(mirror_step, xtrack, spectral_channel) ;
irradiance_error:units = "photons/s/cm^2/nm" ;
irradiance_error:long_name = "irradiance error" ;
irradiance_error:_FillValue = -9.96921e+36f ;
irradiance_error:number_of_significant_digits = 2 ;
irradiance_error:_Storage = "chunked" ;
irradiance_error:_ChunkSizes = 1, 128, 1028 ;
irradiance_error:_Shuffle = "true" ;
irradiance_error:_DeflateLevel = 1 ;
irradiance_error:_Endianness = "little" ;
float nominal_wavelength(xtrack, spectral_channel) ;
nominal_wavelength:units = "nm" ;
nominal_wavelength:valid_min = -9.96921e+36f ;
nominal_wavelength:valid_max = 9.96921e+36f ;
nominal_wavelength:_FillValue = 9.96921e+36f ;
nominal_wavelength:_Storage = "contiguous" ;
nominal_wavelength:_Endianness = "little" ;
ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;
pixel_quality_flag:long_name = "pixel quality flag" ;
pixel_quality_flag:_FillValue = 65535US ;
pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;
pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;
pixel_quality_flag:_Storage = "chunked" ;
pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;
pixel_quality_flag:_Shuffle = "true" ;
pixel_quality_flag:_DeflateLevel = 1 ;
pixel_quality_flag:_Endianness = "little" ;
// group attributes:
} // group band_290_490_nm
group: band_540_740_nm {
dimensions:
xtrack = 2048 ;
spectral_channel = 1028 ;
variables:
int xtrack(xtrack) ;
xtrack:_Storage = "contiguous" ;
xtrack:_Endianness = "little" ;
float irradiance(mirror_step, xtrack, spectral_channel) ;
irradiance:units = "photons/s/cm^2/nm" ;
irradiance:ancillary_variables = "irradiance_error" ;
irradiance:_FillValue = -9.96921e+36f ;
irradiance:_Storage = "chunked" ;
irradiance:_ChunkSizes = 1, 128, 1028 ;
irradiance:_Shuffle = "true" ;
irradiance:_DeflateLevel = 1 ;
irradiance:_Endianness = "little" ;
float irradiance_error(mirror_step, xtrack, spectral_channel) ;
irradiance_error:units = "photons/s/cm^2/nm" ;
irradiance_error:long_name = "irradiance error" ;
irradiance_error:_FillValue = -9.96921e+36f ;
irradiance_error:number_of_significant_digits = 2 ;
irradiance_error:_Storage = "chunked" ;
irradiance_error:_ChunkSizes = 1, 128, 1028 ;
irradiance_error:_Shuffle = "true" ;
irradiance_error:_DeflateLevel = 1 ;
irradiance_error:_Endianness = "little" ;
float nominal_wavelength(xtrack, spectral_channel) ;
nominal_wavelength:units = "nm" ;
nominal_wavelength:valid_min = -9.96921e+36f ;
nominal_wavelength:valid_max = 9.96921e+36f ;
nominal_wavelength:_FillValue = 9.96921e+36f ;
nominal_wavelength:_Storage = "contiguous" ;
nominal_wavelength:_Endianness = "little" ;
ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;
pixel_quality_flag:long_name = "pixel quality flag" ;
pixel_quality_flag:_FillValue = 65535US ;
pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;
pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;
pixel_quality_flag:_Storage = "chunked" ;
pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;
pixel_quality_flag:_Shuffle = "true" ;
pixel_quality_flag:_DeflateLevel = 1 ;
pixel_quality_flag:_Endianness = "little" ;
// group attributes:
} // group band_540_740_nm
} // group frames
}
Thanks @dopplershift!
Hi @ajelenak, does the error message that @dopplershift encountered in his comment above —
NetCDF: Start+count exceeds dimension bound
Location: file ?; fcn ? line 478
— signify anything further to you about what might be causing the netCDF opening issue?
Sounds to me like the dimension information was perhaps recorded wrong somehow? Though I have no idea right now what next to check.
Wild guess because dimension bounds are mentioned... The spectral_channel netCDF dimensions in the file are HDF5 datasets without any data stored but declared as 1D arrays of certain size. This is valid HDF5, and valid for netCDF dimensions as far as I understand. Reading data from these HDF5 datasets will give all fill values (zeroes in this case).
I tried this file with a h5dump --enable-error-stack=2 ... command to check at the HDF5 level and there were no errors printed.
Thanks @ajelenak.
The Location: file ?; fcn ? line 478 message is cryptic. I wonder if anyone here knows what that refers to — perhaps @DennisHeimbigner — e.g., line 478 in which file? And is there any way to determine which variable or dimension might be driving that error, so we can see whether it is the spectral_channel?
Thanks, I'll take a look. This is not the first time something like this (with this error message) has come up.
After some checking, the error report appears to be coming from the function print_rows in the file ncdump/vardata.c line 478.
I am still poking at this to get specifics, thanks for your patience. I was able to use netCDF v4.6.1 to dump the binary file to .cdl, and then use ncgen to re-generate the netCDF binary file; this new, regenerated file does not produce the same error.