parquet-go
parquet-go copied to clipboard
fraugster
Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Prest...
Hi! Do you have the function of merging parquet files? I can't find it :(
It would be nice to know the release version / commit the parquet-tool binary was built with
**Describe the bug** larger files are only split if both the `--row-group-size` and `--file-size` parameters are set, setting the latter alone won't result in files being split. I think if...
The [SchemaWriter](https://github.com/fraugster/parquet-go/blob/c28e3072a9721f25d59193b9d74c642d20800fe8/schema.go#L987-L994) interface gives functionality for adding row level data via the [AddData](https://github.com/fraugster/parquet-go/blob/c28e3072a9721f25d59193b9d74c642d20800fe8/schema.go#L990 ) method. This method accepts the row information in the form of `map[string]interface{}` which allows the caller...
We have files that we are processing that have nil values inside of a list somewhere, that when casted via ```` value.([]byte) ```` Causes a panic. This change will instead...
Hi @fraugster @akrennmair @panamafrancis, I would like to know how to read the following schema with parquet-go? ``` message spark_schema { optional group foo (LIST) { repeated group list {...
This PR fixes the issue here: https://github.com/fraugster/parquet-go/issues/98
**Describe the bug** I have a schema ``` message test { optional group a { optional group foo (MAP) { repeated group key_value { required binary key (STRING); optional binary...
**Describe the bug** AWS Kinesis produces maps in an old way, most notably using `map` instead of `key_value` example: ``` optional group new (MAP) { repeated group map (MAP_KEY_VALUE) {...