Where selection for Netcdf raster slower than for in Memory raster with same dimensions

[felixcremer]

When I am doing a Where Selector it seems to be quite slow compared to other selection methods. I expect that the Where selector is slower because it has to check the function for every element of the axis. But in my case the selection takes 2 seconds while the selection function takes 30 nanoseconds and when I multiply it with the number of elements I get 10 microseconds. This selection is much faster when I do it with random data that I generate compared to the selection on a netcdf based Raster. When I construct a Raster with the same dimensions as ras but a random array the timing is similar to the fully random based Raster.

# Selection on random data 
julia> r = Raster(rand(800,800, 289), (X(1:800), Y(1:800), Ti(DateTime(1995):Month(1):DateTime(2019))))
800×800×289 Raster{Float64,3} with dimensions: 
  X Sampled{Int64} 1:800 ForwardOrdered Regular Points,
  Y Sampled{Int64} 1:800 ForwardOrdered Regular Points,
  Ti Sampled{DateTime} DateTime("1995-01-01T00:00:00"):Month(1):DateTime("2019-01-01T00:00:00") ForwardOrdered Regular Points
extent: Extent(X = (1, 800), Y = (1, 800), Ti = (DateTime("1995-01-01T00:00:00"), DateTime("2019-01-01T00:00:00")))
missingval: missing
parent:


julia> @btime r[Ti=Where(x->year(x)==1995)]
  29.809 ms (19 allocations: 58.60 MiB)
800×800×12 Raster{Float64,3} with dimensions: 
  X Sampled{Int64} 1:800 ForwardOrdered Regular Points,
  Y Sampled{Int64} 1:800 ForwardOrdered Regular Points,
  Ti Sampled{DateTime} DateTime[1995-01-01T00:00:00, …, 1995-12-01T00:00:00] ForwardOrdered Irregular Points
extent: Extent(X = (1, 800), Y = (1, 800), Ti = (DateTime("1995-01-01T00:00:00"), DateTime("1995-12-01T00:00:00")))
missingval: missing
parent:


# results of a NetCDF backed raster
julia> @btime ras[Ti=Where(x->year(x)==1995)]
  1.994 s (643532 allocations: 240.25 MiB)
848×824×12 Raster{Union{Missing, Float32},3} with dimensions: 
  Dim{:rlon} Sampled{Float64} -28.4029998779297:0.0550011811599821:18.18300056457514 ForwardOrdered Regular Points,
  Dim{:rlat} Sampled{Float64} -23.4029998779297:0.0550012159753107:21.863000869751005 ForwardOrdered Regular Points,
  Ti Sampled{DateTime} DateTime[1995-01-16T12:00:00, …, 1995-12-16T12:00:00] ForwardOrdered Irregular Points
extent: Extent(rlon = (-28.4029998779297, 18.18300056457514), rlat = (-23.4029998779297, 21.863000869751005), Ti = (DateTime("1995-01-16T12:00:00"), DateTime("1995-12-16T12:00:00")))
missingval: missing

# Results of a random array with the same dimensions as ras 
ulia> ra = Raster(rand(size(ras)...), dims(ras))
848×824×288 Raster{Float64,3} with dimensions: 
  Dim{:rlon} Sampled{Float64} -28.4029998779297:0.0550011811599821:18.18300056457514 ForwardOrdered Regular Points,
  Dim{:rlat} Sampled{Float64} -23.4029998779297:0.0550012159753107:21.863000869751005 ForwardOrdered Regular Points,
  Ti Sampled{DateTime} DateTime[1995-01-16T12:00:00, …, 2018-12-16T12:00:00] ForwardOrdered Irregular Points
extent: Extent(rlon = (-28.4029998779297, 18.18300056457514), rlat = (-23.4029998779297, 21.863000869751005), Ti = (DateTime("1995-01-16T12:00:00"), DateTime("2018-12-16T12:00:00")))
missingval: missing


julia> @btime ra[Ti=Where(x->year(x)==1995)]
  32.535 ms (19 allocations: 63.97 MiB)
848×824×12 Raster{Float64,3} with dimensions: 
  Dim{:rlon} Sampled{Float64} -28.4029998779297:0.0550011811599821:18.18300056457514 ForwardOrdered Regular Points,
  Dim{:rlat} Sampled{Float64} -23.4029998779297:0.0550012159753107:21.863000869751005 ForwardOrdered Regular Points,
  Ti Sampled{DateTime} DateTime[1995-01-16T12:00:00, …, 1995-12-16T12:00:00] ForwardOrdered Irregular Points
extent: Extent(rlon = (-28.4029998779297, 18.18300056457514), rlat = (-23.4029998779297, 21.863000869751005), Ti = (DateTime("1995-01-16T12:00:00"), DateTime("1995-12-16T12:00:00")))
missingval: missing

[and 11 more slices...]