DataflowTemplates icon indicating copy to clipboard operation
DataflowTemplates copied to clipboard

Allow custom endpoints for Spanner ITs

Open Polber opened this issue 10 months ago • 2 comments

Allows custom endpoints to be specified in Spanner IT's and adds a suite of tests that use a staged spanner instance.

Polber avatar Apr 22 '24 22:04 Polber

Codecov Report

Attention: Patch coverage is 28.57143% with 5 lines in your changes missing coverage. Please review.

Project coverage is 40.93%. Comparing base (5838d4e) to head (c7fc9da). Report is 36 commits behind head on main.

:exclamation: Current head c7fc9da differs from pull request most recent head 6420846

Please upload reports for the commit 6420846 to get more accurate results.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1465      +/-   ##
============================================
- Coverage     41.28%   40.93%   -0.35%     
+ Complexity     2929     2814     -115     
============================================
  Files           769      740      -29     
  Lines         44568    42838    -1730     
  Branches       4768     4582     -186     
============================================
- Hits          18399    17536     -863     
+ Misses        24621    23802     -819     
+ Partials       1548     1500      -48     
Components Coverage Δ
spanner-templates 59.81% <ø> (-1.60%) :arrow_down:
spanner-import-export 65.61% <ø> (+1.16%) :arrow_up:
spanner-live-forward-migration 73.58% <ø> (-0.30%) :arrow_down:
spanner-live-reverse-replication 48.51% <ø> (-1.20%) :arrow_down:
spanner-bulk-migration 77.69% <ø> (-4.38%) :arrow_down:
Files Coverage Δ
...he/beam/it/gcp/spanner/SpannerResourceManager.java 73.29% <28.57%> (-1.44%) :arrow_down:

... and 195 files with indirect coverage changes

codecov[bot] avatar Apr 22 '24 23:04 codecov[bot]

Could you also make the same changes to the DatastreamToSpanner template IT as well? https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/main/v2/datastream-to-spanner/src/main/java/com/google/cloud/teleport/v2/templates/DataStreamToSpanner.java

This template had the staging tests present in the internal python based UIF tests. It will be good to have parity! Thanks!

That is on my agenda of things to accomplish this week, but it will require a bit more effort since Datastream tests aren't stable yet, so that will be in a separate PR. This PR migrates all Spanner tests to use staging other than the Datastream suite if I'm not mistaken

Polber avatar Apr 23 '24 17:04 Polber

Test failures unrelated

Polber avatar Jun 06 '24 21:06 Polber