iceberg-rust icon indicating copy to clipboard operation
iceberg-rust copied to clipboard

Support for `_file` metadata column

Open vustef opened this issue 2 months ago • 0 comments

Is your feature request related to a problem or challenge?

The spec defines metadata columns, one of which is the _file column. This column seems instrumental for write operations positional deletes in future, as well as for having some ways for users to identify rows (combination of filepath + position)

Describe the solution you'd like

The solution should allow building a TableScan with option to return metadata columns (in this case _file). When that happens, the library should return batches that include this extra column. The column should be encoded in such a way that it doesn't add much memory overhead. Since arrow doesn't support constant vectors, perhaps RLE could be used.

Willingness to contribute

I can contribute to this feature independently

vustef avatar Oct 20 '25 09:10 vustef