erddap icon indicating copy to clipboard operation
erddap copied to clipboard

Add EDDTableFromParquet reader?

Open ChrisJohnNOAA opened this issue 5 months ago • 1 comments

This needs more investigation. If already available integrations (jdbc) can read we should document that. Otherwise it may be worth a new dataset type to support parquet.

Discussed in https://github.com/ERDDAP/erddap/discussions/194

Originally posted by MathewBiddle August 30, 2024 I'm continually seeing more and more groups using the Parquet Data File format (https://parquet.apache.org/docs/). Looking through the docs, I see there is an apache/parquet-java package (https://github.com/apache/parquet-java) which could lend itself to the java backend of ERDDAP. I'm curious if others have explored Parquet more in depth and if there is a need for ERDDAP to have the capacity to read from that format.

I haven't seen any discussion on the topic yet. https://github.com/search?q=org%3AERDDAP%20parquet&type=code https://groups.google.com/g/erddap/search?q=parquet

ChrisJohnNOAA avatar Aug 30 '24 19:08 ChrisJohnNOAA