Added API to check if 2 tables are co-located
Addresses #11788.
Approach: For each partition get the set of servers on which the partition resides for each table and verify that the 2 server sets are equal
Edge case: Consider table0 (with replication factor: 2) and table1 (replication factor: 1) server0: segment0_table0_replica_0, segment0_table1_replica_0 server1: segment0_table0_replia_1
In the above case the API will return false i.e tables aren't co-located (because the server sets must match exactly).
Notes: The reason I am returning a string response ("true" or "false") is so that we can decide in the UI how to display it to the user
Testing: Tested both positive and negative scenarios
Codecov Report
Attention: Patch coverage is 0% with 25 lines in your changes missing coverage. Please review.
Project coverage is 61.84%. Comparing base (
59551e4) to head (ccbd8be). Report is 632 commits behind head on master.
| Files | Patch % | Lines |
|---|---|---|
| ...oller/api/resources/PinotTableRestletResource.java | 0.00% | 25 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## master #13403 +/- ##
============================================
+ Coverage 61.75% 61.84% +0.09%
+ Complexity 207 198 -9
============================================
Files 2436 2550 +114
Lines 133233 140311 +7078
Branches 20636 21812 +1176
============================================
+ Hits 82274 86777 +4503
- Misses 44911 46924 +2013
- Partials 6048 6610 +562
| Flag | Coverage Δ | |
|---|---|---|
| custom-integration1 | <0.01% <0.00%> (-0.01%) |
:arrow_down: |
| integration | <0.01% <0.00%> (-0.01%) |
:arrow_down: |
| integration1 | <0.01% <0.00%> (-0.01%) |
:arrow_down: |
| integration2 | 0.00% <0.00%> (ø) |
|
| java-11 | 27.65% <0.00%> (-34.06%) |
:arrow_down: |
| java-21 | 61.84% <0.00%> (+0.21%) |
:arrow_up: |
| skip-bytebuffers-false | 61.82% <0.00%> (+0.07%) |
:arrow_up: |
| skip-bytebuffers-true | 61.82% <0.00%> (+34.10%) |
:arrow_up: |
| temurin | 61.84% <0.00%> (+0.09%) |
:arrow_up: |
| unittests | 61.84% <0.00%> (+0.09%) |
:arrow_up: |
| unittests1 | 46.38% <ø> (-0.51%) |
:arrow_down: |
| unittests2 | 27.65% <0.00%> (-0.08%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
On second thought these is something wrong with the implementation - we shouldn't be getting segment names and comparing them. My implementation works locally because
IdealState table2IdealState = _pinotHelixResourceManager.getHelixAdmin()
.getResourceIdealState(_pinotHelixResourceManager.getHelixClusterName(), table2);
table1IdealState.getPartitionSet() contains eventData_OFFLINE_1717648729000_1717883732000_0 and table2IdealState.getPartitionSet() contains eventData_OFFLINE_1717648729000_1717883732000_0, transcript_OFFLINE_1570863600000_1572418800000_0
for some reason table2IdealState.getPartitionSet() contains eventData_OFFLINE_1717648729000_1717883732000_0 and transcript_OFFLINE_1570863600000_1572418800000_0 which I think is a bug.
@xiangfu0 @Jackie-Jiang Do you think table2IdealState.getPartitionSet() returning a segment from another table is a bug? Also, what approach would you suggest to implement this? Thanks!