YAXArrays.jl icon indicating copy to clipboard operation
YAXArrays.jl copied to clipboard

make mask and scale optional

Open MarkusZehner opened this issue 2 months ago • 0 comments

I was wondering if there would be an easy way to circumvent automatic masking and scaling when opening a (mf)dataset.

specific case would be below the concatenation of multiple nc files where i just want to store them in a more favourable chunking.

import Pkg

Pkg.activate(".")
using Zarr, NetCDF, Dates, Glob, Zarr, YAXArrays
using YAXArrays: YAXArrays as YAX
using DimensionalData: DimensionalData as DD

filelist = glob("*/*.nc", "/.../")
basenames = sort([basename(f) for f in filelist])
order = Dict(name => i for (i, name) in enumerate(basenames))
filelist = sort(filelist, by = f -> order[basename(f)])
dates_read = [DateTime(match(r"\d{8}T\d{6}Z", b).match, dateformat"yyyymmddTHHMMSSZ") for b in basenames]
ds = open_mfdataset(DD.DimArray(filelist, YAX.time(dates_read)), skip_keys=["geostationary"])

ds = readcubedata(ds)

ds = setchunks(ds,(lon=100,lat=100,time=96))


compression = Zarr.BloscCompressor(; clevel=5)

savedataset(ds, path="/../2020_01_01.zarr", driver=:zarr, compressor=compression)

best, Markus

ah, and @lazarusA told me to tag @meggart :)

MarkusZehner avatar Sep 19 '25 12:09 MarkusZehner