iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard
verified publisher icon apache

Spark3.1: Backport #3728 (Binary/Fixed literals) / #3983 (Zorder strategy) to Spark3.1

Open xloya opened this issue 3 years ago • 1 comments

Port zorder strategy (https://github.com/apache/iceberg/pull/3983) and generate binary/fixed literals (https://github.com/apache/iceberg/pull/3728) to Spark 3.1, the reason for the merged port is that Zorder's tests rely on the fix in patch #3728

xloya avatar Sep 22 '22 05:09 xloya

cc @RussellSpitzer @rdblue @xiaotianzhang01 @kbendick

xloya avatar Sep 22 '22 07:09 xloya

@xloya, can you separate backports into individual PRs, please?

rdblue avatar Sep 24 '22 00:09 rdblue

@xloya, can you separate backports into individual PRs, please?

Sure, but the unit tests of the patch of #3983 (Zorder strategy) would not pass until the patch of #3728 (Generate Binary/Fixed literals) is merged. When #3728 is merged, I will rebase to the backport patch of #3983

xloya avatar Sep 24 '22 13:09 xloya

@rdblue Already seperate them to #5847 and #5848, please review again when you have time, thx!

xloya avatar Sep 24 '22 16:09 xloya

Closing this PR as spark-3.1 module is removed from master code.

ajantha-bhat avatar Sep 28 '23 15:09 ajantha-bhat