spatialpandas icon indicating copy to clipboard operation
spatialpandas copied to clipboard

Published 20 hours ago • verified publisher icon holoviz

compat: Ensure that pandas dtype matches dask when loading data from parquet

Open hoxbro opened this issue 1 year ago • 1 comments

It makes it so that the following will give the same types before it returned (string[pyarrow], object).

import dask
import spatialpandas.io as sio

dask.config.set({"dataframe.convert-string": True})

# http://s3.amazonaws.com/datashader-data/nyc_buildings.parq.zip
ddf = sio.read_parquet_dask("./data/nyc_buildings.parq")
ddf["type"].dtype, ddf["type"].compute().dtype

Together with https://github.com/holoviz/holoviews/pull/6362 should make it possible to run the NYC Buildings example.

hoxbro avatar Aug 17 '24 17:08 hoxbro