parquet-go
parquet-go copied to clipboard
fraugster
Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Prest...
By design, the byte array store, is limited to accept only `[]byte` and not string, so if the `map[string]interface{}` contains the string, then the library returns an error indicated that...
It would be neat to have richer support for struct tags for auto-generated schema definitions. I added this feature to a branch off my forked repo and am happy to...
**Describe the bug** I uncovered a small bug in schema.go, the `recursiveFix` func overwrites the passed in `colPath` by using it in the `append` function to add the column name...
**Describe the bug** I am trying to load multiple concurrent parquet files into memory and try to read them row by row. I am facing OOM issue while I read...
Hi, How can I read the entire data into a slice call Foo? Is there a better option than for loop with fr.Next() as you do it here https://github.com/fraugster/parquet-go/blob/master/examples/read-low-level/main.go#L44? ```...
**Describe the bug** Using `floor` struct writer/reader: an empty slice field (e.g. []string{}) written as a list cannot be read back, resulting in error `sub-group list or bag not found`....
**Describe the bug** This library provides no protection against untrusted inputs. It is trivial to write inputs that will cause a server to OOM or crash from another issue. This...
As a user i would like a command to aid in debugging parquet files. For instance I would like to obtain the following file stats in a single command: -...
This PR introduces Go 1.18 generics to generalize the separate int32/int64/float32/float64 implementation into a single number type with small specializations. This deduplicates a lot of similar code across these types....
