iceberg Spark: support rewrite on specified target branch

why the change

currently, rewrite only happens on main branch. with the branch functionality it is also useful to conduct rewrite related tests on branches

what is changed

added target branch to specify the rewrite target

how is the change tested

added new unit tests to cover the new cases

Oct 11 '23 07:10 zinking

can you also please update the respective docs?

Oct 11 '23 07:10 nastra

can you also please update the respective docs?

do you mean the branch & tagging doc? anywhere else?

Oct 11 '23 08:10 zinking

can you also please update the respective docs?

do you mean the branch & tagging doc? anywhere else?

I mean in the Spark procedures section: https://github.com/apache/iceberg/blob/e5ad5ce4c09d1efcd26e3cf0bd27bcf8c06c2400/docs/spark-procedures.md#rewrite_data_files

Oct 11 '23 09:10 nastra

#8762 associating the original issue here.

Oct 11 '23 10:10 zinking

Just checking on the state of this PR, @zinking is there anything pending to move this forward? @ajantha-bhat @nastra any remaining concerns? With branches and merge-on-read deletes now fixed in branches I think the functionality to rewrite data files is necessary. Thanks everyone!

Feb 27 '24 17:02 edgarRd

let me update the PR, and @nastra can have another look.

Feb 28 '24 03:02 zinking

any updates on this ?? is this something thats going to be merged soon

May 20 '24 10:05 amit-cloudinary

ping @nastra for another review.

May 21 '24 03:05 zinking

@zinking - if i can assist with this please let me know (would love to contribute if possible)

May 27 '24 08:05 amit-cloudinary

@zinking - if i can assist with this please let me know (would love to contribute if possible)

feel free to pick it up here.

May 27 '24 08:05 zinking

@zinking - if i can assist with this please let me know (would love to contribute if possible)

feel free to pick it up here.

Ill give it a try :) ,thanks

May 27 '24 08:05 amit-cloudinary