beam icon indicating copy to clipboard operation
beam copied to clipboard

Implement Web API connector interfaces

Open damondouglas opened this issue 1 year ago • 9 comments

This PR addresses #30423 with the Web API connector interfaces. A future PR will create the actual transform that processes Web API requests into a response PCollection.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • [x] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • [ ] Update CHANGES.md with noteworthy changes.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels Python tests Java tests Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

damondouglas avatar Mar 31 '24 00:03 damondouglas

Codecov Report

Attention: Patch coverage is 60.97561% with 32 lines in your changes are missing coverage. Please review.

Project coverage is 70.72%. Comparing base (fc5df6f) to head (b8e8747).

Files Patch % Lines
sdks/go/pkg/beam/io/webapi/webapi.go 60.97% 25 Missing and 7 partials :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #30815      +/-   ##
============================================
- Coverage     70.73%   70.72%   -0.02%     
  Complexity     4468     4468              
============================================
  Files          1256     1257       +1     
  Lines        140774   140856      +82     
  Branches       4306     4306              
============================================
+ Hits          99581    99622      +41     
- Misses        37714    37745      +31     
- Partials       3479     3489      +10     
Flag Coverage Δ
go 54.34% <60.97%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Mar 31 '24 00:03 codecov[bot]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @lostluck for label go. R: @Abacn for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions[bot] avatar Mar 31 '24 17:03 github-actions[bot]

Reminder, please take a look at this pr: @lostluck @Abacn

github-actions[bot] avatar Apr 08 '24 12:04 github-actions[bot]

Codecov Report

Attention: Patch coverage is 60.97561% with 32 lines in your changes are missing coverage. Please review.

Project coverage is 70.72%. Comparing base (b34cf54) to head (b8e8747). Report is 289 commits behind head on master.

:exclamation: Current head b8e8747 differs from pull request most recent head 4239007

Please upload reports for the commit 4239007 to get more accurate results.

Files Patch % Lines
sdks/go/pkg/beam/io/webapi/webapi.go 60.97% 25 Missing and 7 partials :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #30815      +/-   ##
============================================
- Coverage     71.44%   70.72%   -0.73%     
- Complexity     1474     4468    +2994     
============================================
  Files           906     1257     +351     
  Lines        113271   140856   +27585     
  Branches       1076     4306    +3230     
============================================
+ Hits          80931    99622   +18691     
- Misses        30327    37745    +7418     
- Partials       2013     3489    +1476     
Flag Coverage Δ
go 54.34% <60.97%> (+<0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Apr 08 '24 12:04 codecov-commenter

Reminder, please take a look at this pr: @lostluck @Abacn

github-actions[bot] avatar Apr 16 '24 12:04 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @jrmccluskey for label go. R: @chamikaramj for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Apr 19 '24 12:04 github-actions[bot]

@lostluck This is ready but not urgent.

damondouglas avatar May 03 '24 17:05 damondouglas

Reminder, please take a look at this pr: @jrmccluskey @chamikaramj

github-actions[bot] avatar May 11 '24 12:05 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @riteshghorse for label go. R: @ahmedabu98 for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar May 15 '24 12:05 github-actions[bot]

Reminder, please take a look at this pr: @riteshghorse @ahmedabu98

github-actions[bot] avatar May 28 '24 12:05 github-actions[bot]

Closing this after talking with @lostluck . Summary what I got out of the conversation is that the best strategy to expand I/O coverage is to submit an end-to-end I/O using the Go SDK (non-xlang) before contributing any abstractions like this PR attempts. This PR follows what already exists in Java and Python. For Java, it's RequestRequestIO eliminated a lot of boilerplate but this may not necessarily be needed in Go.

damondouglas avatar May 30 '24 16:05 damondouglas