go-native-netcdf icon indicating copy to clipboard operation
go-native-netcdf copied to clipboard

Get wrong value when read netcdf(bad case)

Open ziyunhai opened this issue 3 years ago • 5 comments

thanks you for your project but I have a problem when read netcdf file(abc.nc)。The data can be read correctly by using netcdf-java library。But this library cannot be read correctly。

vr, err := nc.GetVariable("SSI")
vals, ok:= vr.Values.([][]float32)  //2-D Array Value Order Error, Duplicate Read。

ziyunhai avatar Jan 20 '22 11:01 ziyunhai

Hi, I was not able to reproduce the problem.

Can you please try the following code and see if it works for you? It works for me. My go version is 1.17.

package main

import (
	"fmt"

	"github.com/batchatco/go-native-netcdf/netcdf"
)

func main() {
	nc, err := netcdf.Open("abc.nc")
	if err != nil {
		panic(err)
	}
	vr, err := nc.GetVariable("SSI")
	if err != nil {
		panic(err)
	}
	vals, ok := vr.Values.([][]float32)
	if !ok {
		panic("SSI not found")
	}
	fmt.Println("len(SSI)=", len(vals))
}

batchatco avatar Jan 24 '22 07:01 batchatco

Hi,My go version is 1.17.6.I output the results to a file.

package main

import (
	"bytes"
	"fmt"
	"github.com/batchatco/go-native-netcdf/netcdf"
	"io/ioutil"
	"strconv"
)


func main() {
	nc, err := netcdf.Open("abc.nc")
	if err != nil {
		panic(err)
	}
	vr, err := nc.GetVariable("SSI")
	if err != nil {
		panic(err)
	}
	vals, ok := vr.Values.([][]float32)
	if !ok {
		panic("SSI not found")
	}
	fmt.Println("len(SSI)=", len(vals))


	var width = 1600
	var height = 1300


	var buffer bytes.Buffer
	for i := 0; i < height; i++ {
		for j := 0; j < width; j++ {
			v := vals[i][j]
			buffer.WriteString(strconv.FormatFloat(float64(v), 'g', 30, 32))
			buffer.WriteString(" ")
		}
		buffer.WriteString("\n")
	}
	err = ioutil.WriteFile("print_by_go.dat", buffer.Bytes(), 0777)

}

Two different output files for the same file.

incorrect result file. print_by_go.zip

netcdf-java library output correct file. print_by_java.zip

ziyunhai avatar Feb 09 '22 06:02 ziyunhai

Thanks, there is a bug with chunking that I need to fix.

In the mean time, you can convert the chunked format to contiguous as follows:

$ h5repack -i abc.nc -o abc2.nc -l CONTI $ h5repack -i abc2.nc -o abc3.nc -f GZIP=4

(you will need to install hdf5 to get the h5repack command)

batchatco avatar Feb 15 '22 04:02 batchatco

Should be fixed with 9c11d5351929dfb7ecb60fd3300e93045f20af6b

batchatco avatar Mar 13 '22 21:03 batchatco

I am not certain yet it is fixed. Re-opening.

batchatco avatar Mar 14 '22 07:03 batchatco