iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

zorder does not work with sub fields

Open cccs-jc opened this issue 4 months ago • 3 comments

Apache Iceberg version

1.4.2

Query engine

Spark

Please describe the bug 🐞

The rewrite_data_files with zorder does not work on sub-fields.

CALL users.system.rewrite_data_files(
        table => 'users.jcc.flow',
        options => map('max-concurrent-file-group-rewrites', '20',
                       'partial-progress.enabled', 'true',
                       'rewrite-all', 'true'),
        strategy => 'sort',
        sort_order => 'zorder(SRC_IP.v4, DST_IP.v4)',
        where => "END_TIME >= TIMESTAMP '2024-02-12'
            AND END_TIME < TIMESTAMP '2024-02-12' + INTERVAL 1 DAY"
        )

I get the error java.lang.IllegalArgumentException: SRC_IP.v4 does not exist

schema of the table is SRC_IP: struct<v4:bigint,v6:binary>, DST_IP: struct<v4:bigint,v6:binary>

cccs-jc avatar Mar 21 '24 18:03 cccs-jc