Rasters.jl icon indicating copy to clipboard operation
Rasters.jl copied to clipboard

make zonal open stacks of length < 5

Open asinghvi17 opened this issue 8 months ago • 8 comments

not sure if we want this size or not but that's easy to configure

asinghvi17 avatar Apr 22 '25 18:04 asinghvi17

Probably needs some form of test as well? If creating a stack with 800 layers and saving it to NC makes sense I can do that

asinghvi17 avatar Apr 22 '25 18:04 asinghvi17

Hmm I'm going to change how open works pretty soon (non closure version), so we can open more like 300

Yes nc it's just one object so the number of layers doesn't matter as much.

But let's wait and do it properly. We also need to fix the type stability of multy layer stacks, the scaling isn't good currently

rafaqz avatar Apr 22 '25 18:04 rafaqz

Well some code like this will be needed anyway right?

Or do we actually want to change the way this works, and "chunk" the files into groups that we open one at a time? Horrific I know, but if it's for that sweet sweet performance....

asinghvi17 avatar Apr 22 '25 19:04 asinghvi17

I will try and just write it properly soon. Maybe we can merge this as a stopgap but not worth doing anything complicated. Have you tried it with 32 layers already? How is the compile time?

rafaqz avatar Apr 23 '25 08:04 rafaqz

Haven't tried with 32 layers but IIRC that's the recommended limit for static arrays, so I figure it's probably fine here too

asinghvi17 avatar Apr 23 '25 14:04 asinghvi17

Different problem here... It's 32 deep closures!

rafaqz avatar Apr 23 '25 15:04 rafaqz

Ah yeah it would do that...in that case the limit is 5 ish I believe

asinghvi17 avatar Apr 23 '25 15:04 asinghvi17

Looks like I changed it to 5.

asinghvi17 avatar Sep 09 '25 09:09 asinghvi17