arquero icon indicating copy to clipboard operation
arquero copied to clipboard

Feather format written by PyArrow 7.0.0 cannot be opened in Arquero

Open josesho opened this issue 2 years ago • 1 comments

Hi,

with pyarrow v7.0.0:

import seaborn as sns
import pyarrow as arrow
iris = sns.load_dataset("iris")
pf.write_feather(iris, 'iris.feather')

The resultant file is then loaded into arquero in an Observable notebook. The table has the correct columnar structure but the internal data is in a different format.

Screenshot 2022-05-20 at 10 16 57

I think this is related to #270? And hopefully resolved in #277? Hope it can get PRed soon!

PS I use pyarrow to (lightly) munge and convert a 4GB dataset to a smaller 36MB feather file, and I'd like to start building a dashboard with it, using arquero as the analytics engine.

Thanks for all the hard work!

josesho avatar May 20 '22 02:05 josesho

I see that with arquero>=5.0, this issue is resolved, with the caveat that only uncompressed Feather files can be read in for the time being!

josesho avatar Jun 15 '22 03:06 josesho