amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[AMORO-2553] Parallelize reading of EQ delete files and cache them on optimizers

Open zhongqishang opened this issue 1 year ago • 1 comments

Why are the changes needed?

Close #2553.

Brief change log

How was this patch tested?

  • [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • [ ] Add screenshots for manual tests if appropriate

  • [ ] Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

zhongqishang avatar Feb 28 '24 10:02 zhongqishang

Codecov Report

Attention: Patch coverage is 55.60976% with 91 lines in your changes are missing coverage. Please review.

Project coverage is 34.08%. Comparing base (92730de) to head (371a390). Report is 5 commits behind head on master.

Files Patch % Lines
...tease/arctic/io/reader/OptimizerExecutorCache.java 5.17% 55 Missing :warning:
...ase/arctic/io/reader/CombinedBaseDeleteLoader.java 76.11% 13 Missing and 3 partials :warning:
...n/java/com/netease/arctic/utils/AmoroTypeUtil.java 26.31% 13 Missing and 1 partial :warning:
...netease/arctic/io/reader/CombinedDeleteFilter.java 84.00% 1 Missing and 3 partials :warning:
...ic/io/reader/GenericCombinedIcebergDataReader.java 75.00% 1 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2584      +/-   ##
============================================
- Coverage     34.71%   34.08%   -0.64%     
+ Complexity     4521     4397     -124     
============================================
  Files           608      608              
  Lines         50980    50919      -61     
  Branches       6686     6678       -8     
============================================
- Hits          17700    17354     -346     
- Misses        31827    32159     +332     
+ Partials       1453     1406      -47     
Flag Coverage Δ
core 32.36% <55.60%> (-0.71%) :arrow_down:
trino 50.93% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 28 '24 10:02 codecov[bot]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions[bot] avatar Aug 22 '24 00:08 github-actions[bot]

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

github-actions[bot] avatar Aug 30 '24 00:08 github-actions[bot]