cuspatial icon indicating copy to clipboard operation
cuspatial copied to clipboard

[FEA]: Implement Arrow PyCapsule Interface

Open kylebarron opened this issue 1 year ago • 4 comments

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Medium

Please provide a clear description of problem you would like to solve.

I have an interest in growing the GeoArrow ecosystem and making projects interoperable. I'm also developing lonboard, a Python library that uses GeoArrow with deck.gl for visualization of millions of geometries in a Jupyter notebook. I think that would complement cuspatial well, as cuspatial already uses GeoArrow and does not implement its own visualization.

Arrow recently made a PyCapsule Interface spec, where a consumer is able to call an __arrow_c_stream__ method and construct a table without knowing anything about the producer. This feature request is for cuspatial to implement the PyCapsule spec. So as an example in lonboard, you could pass a cuspatial.GeoDataFrame into lonboard.viz and it would just work because of the __arrow_c_stream__ method.

Describe any alternatives you have considered

Right now it looks like to_arrow only exists on the GeoSeries object but not the GeoDataFrame object? Maybe a first approach is to implement https://github.com/rapidsai/cuspatial/issues/1288 and create the public dunder of __arrow_c_stream__ after that?

I'm not sure how the interfacing with GPU memory works; I suppose __arrow_c_stream__ could call table.to_geoarrow_pyarrow() and finish by calling the pyarrow __arrow_c_stream__ method?

Additional context

See also the discussion and links in pyarrow: https://github.com/apache/arrow/issues/39195 and GeoPandas https://github.com/geopandas/geopandas/issues/3156

I don't have an NVIDIA GPU so I'm unable to test solutions :/

kylebarron avatar Jan 30 '24 21:01 kylebarron