go-native-netcdf icon indicating copy to clipboard operation
go-native-netcdf copied to clipboard

Fletcher checksum failure when reading netcdf-4 file.

Open harel opened this issue 7 months ago • 1 comments

Hi!

I'm trying to read a .nc file, which is identified as NetCDF-4. I can read it in R, h5dump etc. The file is valid. The data is grouped, and it does contain Fletcher32 checksum. The dataset i'm interested in is in the /PRODUCT group.

> h5dump -pH filename.nc | grep -A 10 'latitude'

DATASET "latitude" {
         DATATYPE  H5T_IEEE_F32LE
         DATASPACE  SIMPLE { ( 1, 373, 450 ) / ( 1, 373, 450 ) }
         STORAGE_LAYOUT {
            CHUNKED ( 1, 373, 450 )
            SIZE 300032 (2.238:1 COMPRESSION)
         }
         FILTERS {
            CHECKSUM FLETCHER32
            PREPROCESSING SHUFFLE
            COMPRESSION DEFLATE { LEVEL 3 }

Here is a sample code I'm using to try to read the variables off the file.

ncf, err := netcdf.Open(path)
if err != nil {
	panic(err)
}
defer ncf.Close()

grps := ncf.ListSubgroups()
fmt.Println("Groups are", grps)  // lists the groups

nc, _ := ncf.GetGroup("/PRODUCT")

vars := nc.ListVariables()
fmt.Println("Vars are", vars) // lists valid vars

vr, err := nc.GetVariable("latitude")
fmt.Println("LATITUDE IS ", vr, err)   

Here it fails with "fletcher checksum failure". I've tried different ways to access the variable but could not progress beyond that.

I can go around it by cloning the repo, replacing it in my go.mod to use the local copy and commenting out the actual checksum checks, but it feels like a hacky thing to do.

You can find a sample file that exhibits this issue here

Thank you

harel avatar May 17 '25 00:05 harel

I don't have access to the sample file.

batchatco avatar Oct 19 '25 18:10 batchatco