Antoine Pitrou
Antoine Pitrou
> > Or will this require that all Parquet implementations have some baseline level of geospatial support? > > I think the minimum would be "ability to access logical type...
Started a discussion about extension types here: https://lists.apache.org/thread/9xo3mp4n23p8psmrhso5t9q899vxwfjt
That is great! Thanks a lot to all those who made this happen :)
It seems like a similar issue is remaining in pytest-cython 0.3.1: https://github.com/ursacomputing/crossbow/actions/runs/12296705269/job/34316261656#step:7:5932 ``` + pytest -r s -m hypothesis -W error --pyargs pyarrow /opt/conda/envs/arrow/lib/python3.11/site-packages/pytest_cython/__init__.py:2: DeprecationWarning: pkg_resources is deprecated as an...
> FWIW I'd be very interested to see how far we can push the current data structures with approaches like [apache/arrow-rs#5775](https://github.com/apache/arrow-rs/issues/5775), before reaching for format changes. At first sight this...
Hmm. If `SchemaElementV3` is an issue, we might further decouple things I suppose. Though I'm not sure how one would look up columns by names without decoding all the schema...
> The same optimisation could be done in C++, borrows are just pointers with compiler enforced lifetimes This assumes the Thrift C++ APIs allow this.
I added a "PAR3 without legacy metadata" variation for the distant future.
@alkis @JFinis and others, just a quick note that you've convinced me that this proposal is suboptimal for footer access latency, and something better is required. I hope we can...
> @pitrou can we close this now? Yes, we should! Thanks for the reminder.