duckdb_iceberg
duckdb_iceberg copied to clipboard
IOException when querying table with a list<int> column.
Hello, thanks for your great work on duckdb-iceberg!
I'm encountering an issue where attempting to query an iceberg table with iceberg_scan
fails if the table has a column which is a list of ints (list<int>
).
table_with_lists # a pyiceberg.table.Table object
table_without_lists # another pyiceberg.table.Table object
duckdb.sql("INSTALL ICEBERG; LOAD ICEBERG;")
# querying the table that doesn't feature any lists works fine
duckdb.sql(f"SELECT * FROM iceberg_scan('{table_without_lists.metadata_location}') LIMIT 10")
# querying the table that *does* have lists throws an error
duckdb.sql(f"SELECT * FROM iceberg_scan('{table_with_lists.metadata_location}') LIMIT 10")
# IOException: IO Error: Invalid field found while parsing field: type
This seems to be related to an error parsing the table schema. I was wondering if there maybe was no logic for complex types in the duckdb-iceberg extension, but it appears that there is.