Handle tz, datetime & int96 rebase configurations in native_iceberg_compat

Open parthchandra opened this issue 8 months ago • 2 comments

Describe the bug

CometScan passes in configuration flags for tz, datetime, and int96 rebase into the NativeBatchReader used by native_iceberg_compat. These flags should be honored the native implementation

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

Apr 10 '25 17:04 parthchandra

Can we fallback to Spark (or another reader) for now if any of these configs are set to values that we do not yet honor?

Apr 10 '25 20:04 andygrove

This issue is based on the TODO comment in NativeBatchReader. Turns out that the original BatchReader that NativeBatchReader was derived from also has the same comment. All tests appear to pass as well, so perhaps we need to leave this issue to investigate and make sure this does not need additional work.

Apr 10 '25 20:04 parthchandra