Fix: Handle kafka topic not found exception in fetchPartitionCount fu…
Summary
Resolves Issue 17045
This PR fixes an issue where an exception in fetchPartitionCount, often caused by a deleted topic, would halt ingestion for an entire multi-topic table. By catching this exception, the failure is now isolated. Partitions for the deleted topic will get stuck in a "consuming" state, but ingestion from all other valid topics will continue unaffected.
Testing
Tested on a pinot table. Validated single incorrect topic doesn't stop table ingestion completely. This table contains 2 topics - adaptive-authn-gateway & adaptive-authn-gatewy. adaptive-authn-gatewy is a invalid kafka topic.
Validated logs are still getting ingested for adaptive-authn-gateway.
Warning message is logged for topic - adaptive-authn-gatewy
Codecov Report
:x: Patch coverage is 23.07692% with 10 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 33.58%. Comparing base (51632a5) to head (9999d5e).
:warning: Report is 98 commits behind head on master.
:white_check_mark: All tests successful. No failed tests found.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| ...pache/pinot/spi/stream/StreamMetadataProvider.java | 23.07% | 10 Missing :warning: |
:exclamation: There is a different number of reports uploaded between BASE (51632a5) and HEAD (9999d5e). Click for more details.
HEAD has 6 uploads less than BASE
Flag BASE (51632a5) HEAD (9999d5e) unittests1 1 0 unittests 3 1 java-11 4 3 temurin 8 7 unittests2 2 1
Additional details and impacted files
@@ Coverage Diff @@
## master #17088 +/- ##
=============================================
- Coverage 63.17% 33.58% -29.60%
+ Complexity 1421 726 -695
=============================================
Files 3104 3104
Lines 183247 183366 +119
Branches 28088 28098 +10
=============================================
- Hits 115774 61590 -54184
- Misses 58502 116692 +58190
+ Partials 8971 5084 -3887
| Flag | Coverage Δ | |
|---|---|---|
| custom-integration1 | 100.00% <ø> (ø) |
|
| integration | 100.00% <ø> (ø) |
|
| integration1 | 100.00% <ø> (ø) |
|
| integration2 | 0.00% <ø> (ø) |
|
| java-11 | 100.00% <ø> (+36.82%) |
:arrow_up: |
| java-21 | 33.58% <23.07%> (+0.09%) |
:arrow_up: |
| temurin | 33.58% <23.07%> (-29.60%) |
:arrow_down: |
| unittests | 33.58% <23.07%> (-29.60%) |
:arrow_down: |
| unittests1 | ? |
|
| unittests2 | 33.58% <23.07%> (+0.09%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.