parquet-java icon indicating copy to clipboard operation
parquet-java copied to clipboard

Add Brotli compression to Parquet

Open asfimport opened this issue 9 years ago • 1 comments

Hi,

Google release a new compression algorithm called Brotli. A new compression algorithm for the internet, as they claim. Currently Firefox and Chrome (both developer editions) already support Brotli, as such as e.g. ngingx. http://google-opensource.blogspot.de/2015/09/introducing-brotli-new-compression.html

I'm the author of jbrotli - the Java bindings for brotli.

I'm curios and doing a survey, if jBrotli is an option for parquet? Feedback is welcome.

Thanks in advance Martin

Environment: all Reporter: Martin W. Kirst Assignee: Ryan Blue / @rdblue

Related issues:

PRs and other links:

Note: This issue was originally created as PARQUET-521. Please see the migration documentation for further details.

asfimport avatar Feb 10 '16 23:02 asfimport

Ryan Blue / @rdblue: [~marki], thanks for opening this issue! I've actually been testing out Brotli lately with jbrotli and didn't realize there was already an issue for it. I just submitted HADOOP-13126 to add a codec to Hadoop and I'm going to post a PR for this issue soon.

asfimport avatar May 10 '16 18:05 asfimport