When to use GeoArrowScatterplotLayer vs. ScatterplotLayer + GeoParquet loader?

[rmx90210]

Hello, and thank you for this exciting project! 🗺

I have a use case where I'd like to render 8 million points in the browser from a GeoParquet file. I'm interested in using deck.gl as opposed to MVT + MapLibre, but I'm having some trouble understanding the solution space. My understanding is there are two high-level approaches:

1. Use a vanilla deck.gl ScatterplotLayer + the loaders.gl GeoParquetLoader
2. Use GeoArrowScatterplotLayer via the recipe found in this repo

I'm not sure I fully grasp the difference between these two. @kylebarron is there any chance you might be able to give me a quick heuristic on when to use which? Is one method preferred over the other for loading very large datasets?

[rmx90210]

Realizing this is mostly answered by #134! If I'm understanding correctly:

• GeoParquetLoader loads data into a custom, deck.gl-specific binary format which entails more overhead than...
• @geoarrow/deck.gl-layers which uses GeoArrow to store features in memory

[kylebarron]

I don't know the latest state of the upstream deck.gl code. If you're interested in benchmarking the two approaches, that would be super interesting. My hunch is that @geoarrow/deck.gl-layers should still be slightly faster, but I don't have hard evidence to support that

[majid-saeed]

parquet-wasm with loaders.gl and custom geometry processing works fine but takes a lot of memory. duckdb-wasm with viewport based loading and conversions to geojson is also fine if you dont want to load the data to its extents. geoparquet with geoarrow and deckgl geoarrow its most efficient for now, it works fine if you dont have too many attributes present. It also only support a single geoarrow table.