amoro
amoro copied to clipboard
[AMORO-2273]: Fix highly memory usage when orphan file clean is enabled with too many manifest list file
Why are the changes needed?
Close #2273.
Brief change log
- Remove in-memory Content file cache for mixed-format Table Maintainer.
- Refactor logic of fetch all manifest file info from a snapshot when doing orphan file clean.
How was this patch tested?
-
[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
-
[ ] Add screenshots for manual tests if appropriate
-
[X] Run test locally before making a pull request
Documentation
- Does this pull request introduce a new feature? (no)
Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.
Comparison is base (
38c4986) 53.46% compared to head (798ca0a) 53.53%. Report is 1 commits behind head on master.
| Files | Patch % | Lines |
|---|---|---|
| .../netease/arctic/server/utils/IcebergTableUtil.java | 90.00% | 2 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## master #2346 +/- ##
============================================
+ Coverage 53.46% 53.53% +0.07%
- Complexity 4358 4371 +13
============================================
Files 517 517
Lines 29861 29870 +9
Branches 2908 2909 +1
============================================
+ Hits 15964 15992 +28
+ Misses 12608 12590 -18
+ Partials 1289 1288 -1
| Flag | Coverage Δ | |
|---|---|---|
| core | 54.02% <93.93%> (+0.09%) |
:arrow_up: |
| trino | 50.93% <ø> (ø) |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Thanks @baiyangtx for contributions, I left some comment,BTW the ci is unpass
CI unpass