parquet2

parquet2 copied to clipboard

Improved bitpacking

2

This PR ports https://github.com/apache/arrow-rs/pull/2278 to parquet2. Credit to the design and implementation of the packing path go to @tustvold - it is 5-10% faster than the bitpacking crate 🚀 Additionally,...

jorgecarleitao

Deserialisation Error for Nested Types

2

Hi while deserialising the parquet with nested types facing error, do we have the implementation for the following code snippet (got from the examples section) Below code executes when page.descriptor.max_rep_level...

sundeepks

Support async reading without file's content length

8

Currently, any async reading using `parquet2` requires knowing the content length of the remote resource, such as: https://github.com/jorgecarleitao/parquet2/blob/7be3cd6e14a8b7c444cb608ee664c9798c194d82/examples/s3/src/main.rs#L21-L22 However, for any API that follows the [`Range` HTTP request header spec](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range),...

kylebarron

Add support for reading files with modular encryption

2

See https://github.com/apache/parquet-format/blob/master/Encryption.md for motivation and technical details.

jorgecarleitao

See https://github.com/apache/parquet-format/blob/master/Encryption.md for motivation and technical details.

jorgecarleitao

write readme and example

3

Hi, The documentation describes well what can be done for reading. However, I am writing a crate to convert specific measurement file into parquet format ; There is no explanation...

ratal

Consider removing parquet-tools from this repo?

1

I am unable to maintain it and it has never been released, cc @elferherrera .

jorgecarleitao

Appending to a parquet file

4

Hi ! I was wondering if it would be possible to add support for appending data at the end of a parquet file ? It would probably mean to truncate...

GrandChaman

Reading pages with large statistics with an IndexedPageReader fails

1

`IndexedPageReader` puts a hard limit of 1MB on the size of the page headers it can deserialize https://github.com/jorgecarleitao/parquet2/blob/main/src/read/page/indexed_reader.rs#L63 If a page contains a value larger than 512KB and is written...

tjwilson90

Improved API to read column chunks

1

jorgecarleitao