YAXArrays.jl icon indicating copy to clipboard operation
YAXArrays.jl copied to clipboard

incorrect length of Time dim

Open kongdd opened this issue 9 months ago • 1 comments

using Zarr, YAXArrays
using DimensionalData
using CFTime
# chunksize(c) = Cubes.cubechunks(c)

store ="gs://cmip6/CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/3hr/tas/gn/v20190710/"
@time g = open_dataset(zopen(store, consolidated=true))
c = g.tas

@time kelvinCube = c[time=Between(DateTime("2030-01-01"), DateTime("2030-04-01"))]
@time savecube(kelvinCube, "temp-02.zarr")
julia> kelvinCube
384×192×721 YAXArray{Float32,3} with dimensions: 
  Dim{:lon} Sampled{Float64} 0.0:0.9375:359.0625 ForwardOrdered Regular Points,
  Dim{:lat} Sampled{Float64} Float64[-89.28422753251364, -88.35700351866494, …, 88.35700351866494, 89.28422753251364] ForwardOrdered Irregular Points,
  Ti Sampled{DateTime} DateTime[2030-01-01T00:00:00, …, 2030-04-01T00:00:00] ForwardOrdered Irregular Points
units: K
name: tas
Total size: 202.78 MB

Ti should has a length of 721. However, the length of time dimension in the zarr file is 880

ds = Cube("temp-02.zarr") # correct if open by YAXArrays
ds.Time

julia> # length of time is 880
       times = zopen("temp-02.zarr/Time")[:]
       CFTime.timedecode(times, "days since 1980-01-01", "standard")
880-element Vector{DateTime}:
 2029-12-12T03:00:00
 2029-12-12T06:00:00
 2029-12-12T09:00:00
 2029-12-12T12:00:00
 2029-12-12T15:00:00
 2029-12-12T18:00:00
 2029-12-12T21:00:00
 2029-12-13T00:00:00
 2029-12-13T03:00:00
 2029-12-13T06:00:00
 2029-12-13T09:00:00
 2029-12-13T12:00:00
 2029-12-13T15:00:00
 ⋮
 2030-03-30T12:00:00
 2030-03-30T15:00:00
 2030-03-30T18:00:00
 2030-03-30T21:00:00
 2030-03-31T00:00:00
 2030-03-31T03:00:00
 2030-03-31T06:00:00
 2030-03-31T09:00:00
 2030-03-31T12:00:00
 2030-03-31T15:00:00
 2030-03-31T18:00:00
 2030-03-31T21:00:00
 2030-04-01T00:00:00

Double check in Python

import xarray as xr

ds = xr.open_dataset("./temp-02.zarr", engine = "zarr")
ds

image

kongdd avatar Oct 07 '23 07:10 kongdd