Jim Cipar

This change adds two new configuration options for controlling datalake support as part of the Iceberg project. One option, at the cluster level, can be used to globally enable or...

Allow directory of payload files

1

If the payload file is a diretory, load all immediate child files as payloads for the workload.

This is the first step towards outputting Parquet files to support Iceberg. It adds a class, `arrow_writing_consumer`, that can be used with `record_batch_reader::consume` to write a log to a Parquet...

Splitting up https://github.com/redpanda-data/redpanda/pull/18313 into multiple PRs. This is a self-contained PR for the Protobuf to Arrow conversion code. I have also made a slight refactoring: there was some duplicate code...

datalake: iceberg to arrow schema translation

2

This is the first part of the Arrow writer that will translate abstract Iceberg types (iceberg::field_type and iceberg::value) to the Arrow and ultimately Parquet data. This PR adds schema translation...

## Backports Required - [ ] none - not a bug fix - [ ] none - this is a backport - [ ] none - issue does not exist...