Parquet output with uint8 can't be read by some other tools

[philrz]

tl;dr

The file generated by the following can be read back in with Zed tools but not some other tools.

$ echo '{device_floor: 3 (uint8)}' | zq -f parquet -o repro-uint8.parquet -

Details

Repro is with Zed commit b05e70b.

The Parquet file generated is readable by zq itself.

$ zq -version
Version: v1.18.0-18-gb05e70bd

$ echo '{device_floor: 3 (uint8)}' | zq -f parquet -o repro-uint8.parquet -

$ zq -z -i parquet repro-uint8.parquet
{device_floor:3(uint8)}

However, If I try to read the file back with DuckDB, it fails.

$ duckdb --version
v1.1.1 af39bd0dcf

$ duckdb -c 'SELECT * FROM "repro-uint8.parquet"'
Invalid Input Error: Failed to cast value: Type UINT32 with value 4294967295 can't be cast because the value is out of range for the destination type UINT8

Likewise with Tad, which doesn't show any kind of error, just a blank screen.

https://github.com/user-attachments/assets/8d2e32b4-85cc-4484-bab2-ab4f04f3da60

As implied by the DuckDB error message, it seems the uint8 value is the root cause, since if I stick to the default integer, the problem doesn't appear.

$ echo '{device_floor: 3}' | zq -f parquet -o repro-default.parquet -

$ duckdb -c 'SELECT * FROM "repro-default.parquet"'
┌──────────────┐
│ device_floor │
│    int64     │
├──────────────┤
│            3 │
└──────────────┘

https://github.com/user-attachments/assets/9a8bf6a1-2c8e-44c4-963a-3f415de8ba0d

Of course, even as I can show these multiple tools choking on it, the reader at https://parquetreader.com loads the one with the uint8 without complaint, so it's hard for me to guess if this a bug or if some tools just have spotty type coverage.

That said, DuckDB has no problem with its own Parquet-that-contains-uint8.

$ duckdb my.db
v1.1.1 af39bd0dcf
Enter ".help" for usage hints.
D CREATE TABLE repro (device_floor UTINYINT);
D INSERT INTO repro (device_floor) values(3);
D select * from repro;
┌──────────────┐
│ device_floor │
│    uint8     │
├──────────────┤
│            3 │
└──────────────┘
D copy repro to "uint8-from-duckdb.parquet" (FORMAT PARQUET);

$ duckdb -c 'SELECT * FROM "uint8-from-duckdb.parquet"'
┌──────────────┐
│ device_floor │
│    uint8     │
├──────────────┤
│            3 │
└──────────────┘

Tad loads that one ok too.