Anshul Singhvi
Anshul Singhvi
If you want to use filter on a GeoJSON feature collection, you can use [TableOperations.jl](https://github.com/JuliaData/TableOperations.jl), which is implementation-agnostic - though it will probably be slower than DataFrames!
Aha, didn't realize that case was also a thing. Too used to working with Kerchunk I guess :D We probably need a lot more documentation around `lazy`, `open` and `read`...
Isn't that what `IDataset` is for in ArchGDAL, for instance? It has an attached finalizer. Any ArchGDAL/GeoDataFrames-read geometry is in a similar position...
> relying on people using `close` sorry to belabour this point - but isn't the point of a finalizer that you don't need to use `close`? The first GC sweep...
Anything interactive is where I think `oras = open(ras)` could be really useful. Currently you have to do `open(ras) do R; ...; end` as opposed to directly working with an...
So I guess we now have four levels of lazy? - `true`: file based only - `:open`: open the dataset, add a finalizer (?) or require user to `close` -...
Multithreading speeds it up significantly as well - 22s to 3s (7.3x speedup with 8 threads) ```julia-repl julia> chunked_result = @time _spherical_cellarea_chunked_threaded(ras |> dims) 3.060461 seconds (15.53 k allocations: 2.569...
Well the tiling is just a nice way to do threading. We could just materialize the whole array of transformed points in memory but it's nice to have the ability...
We can do it manually with Iterators.partition, this was just easier. The reason for tilediteration was just to get the correct chunk indices. Maybe we can use DiskArrays eachchunk for...
Yeah DiskArrays works, so let's do that. ```julia julia> DiskArrays.GridChunks(rand(100, 102), (10, 10)) 10×11 DiskArrays.GridChunks{2, Tuple{DiskArrays.RegularChunks, DiskArrays.RegularChunks}}: (1:10, 1:10) (1:10, 11:20) (1:10, 21:30) (1:10, 31:40) (1:10, 41:50) (1:10, 51:60) (1:10,...