parquet4s

parquet4s copied to clipboard

Review issues found by Codacy and fix them.

mjakubowski84

Parquet 1.12.3 introduced `In` and `NotIn` filter predicates. - Let's replace our own implementation of `in` backed by `UserDefinedPredicate` - Let's introduce `notIn` operator

mjakubowski84

partitionBy to do not include partition key into filesystem path

1

Hi, I use parquet4s with Akka Streams. Lib allows you to partition data via `partitionBy()` method. Any way to do not include specified partition key into path name? In example...

s1rc0

Support for custom metadata?

3

Hello, Is there any effort underway (or interest in) to allow users to write additional file-level metadata, e.g. for provenance of how data contained were generated? Something like for the...

vreuter

Feature request: InputFile support

1

Title says it all. I would love to try the library out but think I require `ParquetReader`'s `InputFile` constructor. From looking around, it seems like only `Path` is supported.

nkconnor

mjakubowski84

Output path's for files in partition

1

Hi! I'm write partitioned parquet via fs2 stream. Is there a way to get the full output path (with file name included) after writing records batch? I found several options...

MaksGS09

[Question] get a listing of parquet files?

4

Hello, it’s me again! I was wondering if it’s possible to get a listing of all parquet files (and their partition info) in a partitioned read? To add some context:...

calvinlfer

[Question] Is there a mechanism to detect when the `rotatingWriter` finishes writing to a file and to be notified of the file that was written?

1

I was looking at the `postWriteHandler` mechanism on `viaParquet`, I see that there's a flush mechanism but I'm a bit confused on the semantics. * It seems like `postWriteHandler` gets...

calvinlfer

[RFC] Refactor timestamp codecs

2

For now, `LocalDateTime` is a "base" type for INT96 timestamp encoding/decoding: conversion for a `Timestamp` and `Instant` goes via `LocalDateTime`. I see a little mismatch here: - If I'm not...

flipp5b