DataflowTemplates
DataflowTemplates copied to clipboard
Session file support for bulk migration using JSON node
Contains the session file support in the bulk template using JSON node as the standard format. If we decide to finalise avro, we will update this PR with avro support.
This PR supports:
- Name transformations, synth PK handling and dropped columns using session file.
- Almost all mysql data types except MySQL YEAR and BIT.
This PR does not support:
- Sharding support (shard id population)
- Transformation context
Codecov Report
Merging #1400 (fe123de) into main (f8653ef) will increase coverage by
0.07%. Report is 11 commits behind head on main. The diff coverage is100.00%.
Additional details and impacted files
@@ Coverage Diff @@
## main #1400 +/- ##
============================================
+ Coverage 38.92% 38.99% +0.07%
+ Complexity 2761 2759 -2
============================================
Files 743 745 +2
Lines 41939 41992 +53
Branches 4516 4527 +11
============================================
+ Hits 16324 16375 +51
Misses 24127 24127
- Partials 1488 1490 +2
| Components | Coverage Δ | |
|---|---|---|
| spanner-templates | 52.46% <100.00%> (+0.28%) |
:arrow_up: |
| spanner-import-export | 65.22% <ø> (-0.03%) |
:arrow_down: |
| spanner-live-forward-migration | 57.20% <100.00%> (+0.97%) |
:arrow_up: |
| spanner-live-reverse-replication | 38.44% <100.00%> (+0.91%) |
:arrow_up: |
| spanner-bulk-migration | 59.40% <100.00%> (+1.27%) |
:arrow_up: |
| Files | Coverage Δ | |
|---|---|---|
| ...eport/v2/spanner/migrations/schema/LazyMapper.java | 100.00% <100.00%> (ø) |
|
| .../spanner/migrations/schema/SessionBasedMapper.java | 100.00% <100.00%> (ø) |