[Managed Iceberg] Cleanup SerializedDataFile; use ByteBuffer instead

[ahmedabu98]

We were using a workaround because of an issue in ByteBuddy (#32701). This has been fixed so we can now use ByteBuffer normally

Note that this is an update compatible change because byte[] and ByteBuffer types will both map to the same BYTES Schema type

[github-actions[bot]]

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

[ahmedabu98]

assign set of reviewers

[github-actions[bot]]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java. R: @Abacn for label build. R: @damondouglas for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[github-actions[bot]]

Reminder, please take a look at this pr: @kennknowles @Abacn @damondouglas

[Abacn]

LGTM, however I saw one of the fixes was reverted: https://github.com/apache/beam/issues/32701#issuecomment-2437929579, feel free to merge if get confirm that the original issue indeed fixed and stablelized

[github-actions[bot]]

Reminder, please take a look at this pr: @kennknowles @Abacn @damondouglas

[github-actions[bot]]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java. R: @damccorm for label build. R: @chamikaramj for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions[bot]]

Reminder, please take a look at this pr: @robertwb @damccorm @chamikaramj

[damccorm]

What is the current status here? It looks like we have a LGTM but checks are failing

[ahmedabu98]

The PR that fixed the original issue was reverted, so we'll wait until a future fix gets in. This change isn't urgent in any way, so will turn it into a draft until then

[github-actions[bot]]

Reminder, please take a look at this pr: @robertwb @damccorm @chamikaramj

[ahmedabu98]

Waiting on author

[github-actions[bot]]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java. R: @Abacn for label build. R: @damondouglas for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[kennknowles]

On your other PR I basically suggested what you did here. What was the issue?

[kennknowles]

Update-incompatible?

[ahmedabu98]

I checked with some folks and it should be update-compatible because ByteBuffer gets resolved to byte[] for serialization.

The problem is that our current ByteBuddy utils don't fully acknowledge Map parameterized types (more details in issue #32701). This was going to be fixed as a byproduct of this PR #32757, but the PR was later reverted in favor of #32081.

The latter PR doesn't fix this issue. I tried delving in a couple of times recently to fix this again but I couldn't get something substantive quickly enough. I'm not familiar with this realm of Schema inference.