iceberg
iceberg copied to clipboard
zorder does not work with sub fields
Apache Iceberg version
1.4.2
Query engine
Spark
Please describe the bug 🐞
The rewrite_data_files
with zorder does not work on sub-fields.
CALL users.system.rewrite_data_files(
table => 'users.jcc.flow',
options => map('max-concurrent-file-group-rewrites', '20',
'partial-progress.enabled', 'true',
'rewrite-all', 'true'),
strategy => 'sort',
sort_order => 'zorder(SRC_IP.v4, DST_IP.v4)',
where => "END_TIME >= TIMESTAMP '2024-02-12'
AND END_TIME < TIMESTAMP '2024-02-12' + INTERVAL 1 DAY"
)
I get the error java.lang.IllegalArgumentException: SRC_IP.v4 does not exist
schema of the table is SRC_IP: struct<v4:bigint,v6:binary>, DST_IP: struct<v4:bigint,v6:binary>