YAXArrays.jl icon indicating copy to clipboard operation
YAXArrays.jl copied to clipboard

Can't save dataset with multiple time axes

Open felixcremer opened this issue 8 months ago • 1 comments

I am trying to extract some data with the same spatial axes but different time axes. It seems to me as if the savedataset function tries to use the first time axis for all cubes and therefore fails at the second cube in constructing the data cube due to a dimension mismatch.

  :A059 => 15000×15000×290 GMWOPResult{Union{Missing, Float64}, 3, GMDWop{3, Tuple{InputArray{DiskArrayStack{Union{Missing, Float64}, 3, CFDiskArray{Union{Missing, …
  :D080 => 15000×15000×250 GMWOPResult{Union{Missing, Float64}, 3, GMDWop{3, Tuple{InputArray{DiskArrayStack{Union{Missing, Float64}, 3, CFDiskArray{Union{Missing, …
  :D152 => 15000×15000×1 GMWOPResult{Union{Missing, Float64}, 3, GMDWop{3, Tuple{InputArray{DiskArrayStack{Union{Missing, Float64}, 3, CFDiskArray{Union{Missing, Fl…
  :D153 => 15000×15000×9 GMWOPResult{Union{Missing, Float64}, 3, GMDWop{3, Tuple{InputArray{DiskArrayStack{Union{Missing, Float64}, 3, CFDiskArray{Union{Missing, Fl…
  :A161 => 15000×15000×293 GMWOPResult{Union{Missing, Float64}, 3, GMDWop{3, Tuple{InputArray{DiskArrayStack{Union{Missing, Float64}, 3, CFDiskArray{Union{Missing, …


julia> ds = Dataset(;relorbitcubes...)
YAXArray Dataset
Shared Axes: 
  (↓ X Sampled{Float64} 4.5e6:20.0:4.79998e6 ForwardOrdered Regular Points,
  → Y Sampled{Float64} 5.7e6:-20.0:5.40002e6 ReverseOrdered Regular Points)

Variables with additional axes:
  Additional Axes: 
  (↓ Ti Sampled{DateTime} [2015-01-15T04:48:10, …, 2025-03-11T04:49:01] ForwardOrdered Irregular Points)
  Variables: 
  D080

  Additional Axes: 
  (↓ Ti Sampled{DateTime} [2015-04-14T17:19:00, …, 2025-03-04T17:19:48] ForwardOrdered Irregular Points)
  Variables: 
  A161

  Additional Axes: 
  (↓ Ti Sampled{DateTime} [DateTime("2015-01-15T04:48:35")] ForwardOrdered Irregular Points)
  Variables: 
  D152

  Additional Axes: 
  (↓ Ti Sampled{DateTime} [2015-04-07T17:27:20, …, 2025-03-09T17:28:09] ForwardOrdered Irregular Points)
  Variables: 
  A059

  Additional Axes: 
  (↓ Ti Sampled{DateTime} [2017-03-04T04:56:08, …, 2025-02-08T04:57:39] ForwardOrdered Irregular Points)
  Variables: 
  D153




julia> savedataset(ds, path="data/fluxtestdata_relorbitsplit.zarr/")
ERROR: ArgumentError: Can not construct YAXArray, supplied data size is (15000, 15000, 250) while axis lengths are (15000, 15000, 290)
Stacktrace:
  [1] YAXArray(axes::Tuple{…}, data::DiskArrayTools.CFDiskArray{…}, properties::Dict{…}, chunks::DiskArrays.GridChunks{…}, cleaner::Vector{…})
    @ YAXArrays.Cubes /mnt/felix1/worldmap/dev/YAXArrays/src/Cubes/Cubes.jl:114
  [2] #YAXArray#5
    @ /mnt/felix1/worldmap/dev/YAXArrays/src/Cubes/Cubes.jl:136 [inlined]
  [3] collectfromhandle(e::@NamedTuple{…}, dshandle::ZarrExt.ZarrDataset, cleaner::Vector{…})
    @ YAXArrays.Datasets /mnt/felix1/worldmap/dev/YAXArrays/src/DatasetAPI/Datasets.jl:544
  [4] #115
    @ /mnt/felix1/worldmap/dev/YAXArrays/src/DatasetAPI/Datasets.jl:706 [inlined]
  [5] iterate
    @ ./generator.jl:48 [inlined]
  [6] _collect(c::Vector{…}, itr::Base.Generator{…}, ::Base.EltypeUnknown, isz::Base.HasShape{…})
    @ Base ./array.jl:811
  [7] collect_similar(cont::Vector{@NamedTuple{…}}, itr::Base.Generator{Vector{…}, YAXArrays.Datasets.var"#115#121"{…}})
    @ Base ./array.jl:720
  [8] map(f::Function, A::Vector{@NamedTuple{…}})
    @ Base ./abstractarray.jl:3371
  [9] savedataset(ds::Dataset; path::String, persist::Nothing, overwrite::Bool, append::Bool, skeleton::Bool, backend::Symbol, driver::Symbol, max_cache::Float64, writefac::Float64, kwargs::@Kwargs{})
    @ YAXArrays.Datasets /mnt/felix1/worldmap/dev/YAXArrays/src/DatasetAPI/Datasets.jl:706
 [10] top-level scope
    @ REPL[63]:1
Some type information was truncated. Use `show(err)` to see complete types.

felixcremer avatar Mar 12 '25 10:03 felixcremer

You should use two different names for the time axis! i.e, time time_2 😄

lazarusA avatar Mar 12 '25 10:03 lazarusA