YAXArrays.jl
YAXArrays.jl copied to clipboard
make mask and scale optional
I was wondering if there would be an easy way to circumvent automatic masking and scaling when opening a (mf)dataset.
specific case would be below the concatenation of multiple nc files where i just want to store them in a more favourable chunking.
import Pkg
Pkg.activate(".")
using Zarr, NetCDF, Dates, Glob, Zarr, YAXArrays
using YAXArrays: YAXArrays as YAX
using DimensionalData: DimensionalData as DD
filelist = glob("*/*.nc", "/.../")
basenames = sort([basename(f) for f in filelist])
order = Dict(name => i for (i, name) in enumerate(basenames))
filelist = sort(filelist, by = f -> order[basename(f)])
dates_read = [DateTime(match(r"\d{8}T\d{6}Z", b).match, dateformat"yyyymmddTHHMMSSZ") for b in basenames]
ds = open_mfdataset(DD.DimArray(filelist, YAX.time(dates_read)), skip_keys=["geostationary"])
ds = readcubedata(ds)
ds = setchunks(ds,(lon=100,lat=100,time=96))
compression = Zarr.BloscCompressor(; clevel=5)
savedataset(ds, path="/../2020_01_01.zarr", driver=:zarr, compressor=compression)
best, Markus
ah, and @lazarusA told me to tag @meggart :)