petastorm icon indicating copy to clipboard operation
petastorm copied to clipboard
verified publisher icon uber

Petastorm break with pyarrow 13.0 or newer. Stable version of pyarrow is at 16.0 now.

Open LauritsDixen opened this issue 1 year ago • 2 comments

Several parts of pyarrow has changed and deprecated.

Running with 12.0 gives FutureWarnings of using objects that has deprecated since 5.0.0!

Here is such a warning. /python3.11/site-packages/petastorm/etl/dataset_metadata.py:288: FutureWarning: ParquetDatasetPiece is deprecated as of pyarrow 5.0.0 and will be removed in a future version

Are there any plans of updating?

LauritsDixen avatar May 01 '24 07:05 LauritsDixen

I do not actively maintain the project. Would be happy to review a PR if you can put it up.

selitvin avatar May 03 '24 00:05 selitvin

https://github.com/uber/petastorm/pull/754 this CR is a work-in-progress without implementing new dataset interface

westfly avatar Jul 09 '24 03:07 westfly