spatialdata icon indicating copy to clipboard operation
spatialdata copied to clipboard

Lazy loading of points and spatial index

Open kevinyamauchi opened this issue 3 years ago • 2 comments

@LucaMarconato has observed that load lots (1E6) points is slow. Thus, we should discuss lazy loading and using some sort of spatial index for query.

A few points:

  • we need to define the access patterns we want to support and prioritize which should be fastest
  • we need to determine if delayed loading is sufficient or if we need a spatial index
  • we should separately consider analysis and rendering cases
  • we need to decide if spatial index should be able to be saved/loaded

kevinyamauchi avatar Oct 26 '22 14:10 kevinyamauchi

Lazy loading is now supported. Spatial index could be achieved using Dask GeoDataFrame for points. We are still considering this.

LucaMarconato avatar Mar 15 '23 16:03 LucaMarconato

  • [ ] after this is implemented we should explain how to use it in the docs.

LucaMarconato avatar Mar 30 '24 15:03 LucaMarconato

Since #359 contains the information of the remainder of the work I will close this issue.

melonora avatar Jul 09 '24 13:07 melonora