pinot icon indicating copy to clipboard operation
pinot copied to clipboard

Enforce continue on error flag during batch segment ingestion as well

Open KKcorps opened this issue 1 year ago • 1 comments

This will allow us to ingest data even when some records are corrupt in a file

KKcorps avatar Oct 25 '24 12:10 KKcorps

Codecov Report

Attention: Patch coverage is 46.42857% with 15 lines in your changes missing coverage. Please review.

Project coverage is 63.74%. Comparing base (59551e4) to head (54ae10c). Report is 1246 commits behind head on master.

Files with missing lines Patch % Lines
.../core/segment/processing/mapper/SegmentMapper.java 33.33% 8 Missing and 2 partials :warning:
...creator/RecordReaderSegmentCreationDataSource.java 61.53% 4 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14309      +/-   ##
============================================
+ Coverage     61.75%   63.74%   +1.99%     
- Complexity      207     1555    +1348     
============================================
  Files          2436     2659     +223     
  Lines        133233   145469   +12236     
  Branches      20636    22225    +1589     
============================================
+ Hits          82274    92730   +10456     
- Misses        44911    45892     +981     
- Partials       6048     6847     +799     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) :arrow_up:
integration 100.00% <ø> (+99.99%) :arrow_up:
integration1 100.00% <ø> (+99.99%) :arrow_up:
integration2 0.00% <ø> (ø)
java-11 63.72% <46.42%> (+2.01%) :arrow_up:
java-21 63.61% <46.42%> (+1.99%) :arrow_up:
skip-bytebuffers-false 63.74% <46.42%> (+1.99%) :arrow_up:
skip-bytebuffers-true 63.59% <46.42%> (+35.87%) :arrow_up:
temurin 63.74% <46.42%> (+1.99%) :arrow_up:
unittests 63.74% <46.42%> (+1.99%) :arrow_up:
unittests1 55.39% <46.42%> (+8.50%) :arrow_up:
unittests2 34.23% <28.57%> (+6.50%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Oct 25 '24 13:10 codecov-commenter