Initial Commit for AvroPayloadSerializer

[talatuyarer]

Please add a meaningful description for your change here

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• [ ] Choose reviewer(s) and mention them in a comment (R: @username).
• [ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• [ ] Update CHANGES.md with noteworthy changes.
• [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[talatuyarer]

R: @TheNeuralBit Could you review this ? This is initial commit. I will continue develop based on your feedbacks. Thanks

[github-actions[bot]]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 74.34%. Comparing base (d54841c) to head (bfa5999). Report is 6632 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #22348      +/-   ##
==========================================
+ Coverage   74.24%   74.34%   +0.10%     
==========================================
  Files         702      703       +1     
  Lines       92999    93627     +628     
==========================================
+ Hits        69045    69610     +565     
- Misses      22687    22750      +63     
  Partials     1267     1267              

Flag	Coverage Δ
python	83.67% <ø> (+0.06%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[steveniemitz]

I'd be interested to see the performance of this vs FastReaderBuilder.createDatumReader and wrapping the resulting object in a Row.

[TheNeuralBit]

I'd be interested to see the performance of this vs FastReaderBuilder.createDatumReader and wrapping the resulting object in a Row.

That documentation is sparse, is there more somewhere?

[talatuyarer]

I'd be interested to see the performance of this vs FastReaderBuilder.createDatumReader and wrapping the resulting object in a Row.

Thank you for reviewing my pr @steveniemitz Let me write a JMH test and share result. I will reply your other comments after JMH test result sounds good ?

[steveniemitz]

That documentation is sparse, is there more somewhere?

I'm not sure actually, I'd stumbled across it a year or two ago because I was doing something very similar to what's being done here. https://issues.apache.org/jira/browse/AVRO-2247 is the most detail I can find on the work being done to improve the performance.

I realize now after writing that comment though that this wasn't in the java avro library until 1.10, and beam is still on 1.8 (or maybe 1.9?), so it might not be particularly relevant anyways.

[TheNeuralBit]

That documentation is sparse, is there more somewhere?

I'm not sure actually, I'd stumbled across it a year or two ago because I was doing something very similar to what's being done here. https://issues.apache.org/jira/browse/AVRO-2247 is the most detail I can find on the work being done to improve the performance.

I realize now after writing that comment though that this wasn't in the java avro library until 1.10, and beam is still on 1.8 (or maybe 1.9?), so it might not be particularly relevant anyways.

Ah ok. If we run some benchmarks now and it looks good we could file an issue to consider switching to it when/if we upgrade Avro. For reference #19969 is the only avro upgrade issue I can find.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[damondouglas]

Good day, @talatuyarer. Thank you again for making this contribution! Would you like to still work on this PR or should I close it? We appreciate what you do for the Beam community!

[talatuyarer]

Hi @damondouglas Let me work on it. I want to merge this MR to BEAM. It makes Beam's AVRO serialization more than 3x faster.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.