[BEAM-14378] [CdapIO] SparkReceiverIO Read via SDF

[Amar3tto]

Resolves #21680

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• [ ] Choose reviewer(s) and mention them in a comment (R: @username).
• [ ] Add a link to the appropriate issue in your description, if applicable. This will automatically link the pull request to the issue.
• [ ] Update CHANGES.md with noteworthy changes.
• [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[asf-ci]

Can one of the admins verify this patch?

[asf-ci]

Can one of the admins verify this patch?

[asf-ci]

Can one of the admins verify this patch?

[codecov[bot]]

Codecov Report

Merging #17828 (b84ab8c) into master (aa574f5) will decrease coverage by 0.17%. The diff coverage is n/a.

:exclamation: Current head b84ab8c differs from pull request most recent head 18ac5cc. Consider uploading reports for the commit 18ac5cc to get more accurate results

@@            Coverage Diff             @@
##           master   #17828      +/-   ##
==========================================
- Coverage   74.17%   74.00%   -0.18%     
==========================================
  Files         711      704       -7     
  Lines       93605    93250     -355     
==========================================
- Hits        69433    69007     -426     
- Misses      22895    22977      +82     
+ Partials     1277     1266      -11     

Flag	Coverage Δ
python	83.54% <ø> (-0.02%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ython/apache_beam/io/gcp/bigquery_read_internal.py	53.36% <0.00%> (-4.33%)	:arrow_down:
sdks/go/pkg/beam/io/filesystem/memfs/memory.go	92.00% <0.00%> (-4.16%)	:arrow_down:
sdks/python/apache_beam/io/gcp/bigquery.py	70.37% <0.00%> (-3.88%)	:arrow_down:
...ks/go/pkg/beam/runners/dataflow/dataflowlib/job.go	21.55% <0.00%> (-1.13%)	:arrow_down:
sdks/go/pkg/beam/core/runtime/exec/input.go	50.96% <0.00%> (-1.00%)	:arrow_down:
sdks/go/pkg/beam/core/graph/coder/row_encoder.go	67.52% <0.00%> (-0.55%)	:arrow_down:
sdks/python/apache_beam/runners/direct/executor.py	96.46% <0.00%> (-0.55%)	:arrow_down:
sdks/python/apache_beam/dataframe/io.py	88.78% <0.00%> (-0.49%)	:arrow_down:
sdks/python/apache_beam/typehints/row_type.py	96.55% <0.00%> (-0.47%)	:arrow_down:
sdks/python/apache_beam/pipeline.py	92.42% <0.00%> (-0.42%)	:arrow_down:
... and 61 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[Amar3tto]

Run Java PreCommit

[Amar3tto]

Run SQL_Java17 PreCommit

[Amar3tto]

Run Java PreCommit

[Amar3tto]

Run Java PreCommit

[elizaveta-lomteva]

@aromanenko-dev Alexey, we've prepared the SparkReceiverIO connector Read interface. This PR is ready to be reviewed. Thanks for your attention!

[aromanenko-dev]

Thanks! I'm sorry for a delay with review, I'll try to take a look as soon as I can.

[Amar3tto]

@aromanenko-dev Kind ping on this. Did you have time to review this PR?

[aromanenko-dev]

Please, avoid merge commits and use git rebase instead

[aromanenko-dev]

Thanks for addressing review comments. It looks ok in general for me.

I'll be on PTO for the next 2 weeks, so I'd kindly ask @chamikaramj or @lukecwik to take a look on SDF part.

[chamikaramj]

Ack. Will take a look. Thanks @aromanenko-dev

[Amar3tto]

Run Java PreCommit

[chamikaramj]

Any updates ?

[elizaveta-lomteva]

@chamikaramj All your comments have been answered. The PR is ready to be merged. Thank you.

[Abacn]

New test flake seen on SparkReceiverIO test: #23449