[ARCTIC-910]Ignore committing error to HMS when committing Hive format table transaction
…saction
Why are the changes needed?
Close #910 .
Brief change log
- Ignore committing error to HMS when committing Hive format table transaction
How was this patch tested?
-
[x] Add some test cases that check the changes thoroughly including negative and positive cases if possible
-
[ ] Add screenshots for manual tests if appropriate
-
[ ] Run test locally before making a pull request
Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not documented)
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
5eb153f) 32.64% compared to head (92c2f16) 32.59%.
Additional details and impacted files
@@ Coverage Diff @@
## master #2482 +/- ##
============================================
- Coverage 32.64% 32.59% -0.05%
+ Complexity 4473 4469 -4
============================================
Files 598 598
Lines 50276 50280 +4
Branches 6676 6676
============================================
- Hits 16411 16389 -22
- Misses 32550 32587 +37
+ Partials 1315 1304 -11
| Flag | Coverage Δ | |
|---|---|---|
| core | 30.75% <100.00%> (+<0.01%) |
:arrow_up: |
| trino | 50.38% <ø> (-0.58%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Can you share the exception stack for the scenario addressed by this PR? I'm not sure when this issue occurs. @hameizi
I also noticed that it seems like the try-catch block was forgotten to be added here com.netease.arctic.hive.op.ReplaceHivePartitions#commit
@Override
public void commit() {
if (!addFiles.isEmpty()) {
List<DataFile> dataFiles =
HiveCommitUtil.commitConsistentWriteFiles(this.addFiles, table.io(), table.spec());
this.addFiles.clear();
this.addFiles.addAll(dataFiles);
this.addFiles.forEach(delegate::addFile);
commitTimestamp = (int) (System.currentTimeMillis() / 1000);
if (table.spec().isUnpartitioned()) {
generateUnpartitionTableLocation();
} else {
applyHivePartitions();
}
delegate.commit();
commitPartitionProperties();
if (!insideTransaction) {
transaction.commitTransaction();
}
// here the try-catch is missing
if (table.spec().isUnpartitioned()) {
commitUnPartitionedTable();
} else {
commitPartitionedTable();
}
}
}
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.