Subbu Subramaniam
Subbu Subramaniam
Pinot recovers automatically from this scenario. You can run the RealitmeSegmentValidationManager manually if urgent fix is needed. Alternatively, you can configure to run this periodically (say, every 30m) to fix...
@npawar the consumer on the server side should experience a (permanent) exception, which will cause the segment to go OFFLINE, and then the validator will fix it automatically. At least,...
If pinot broker returns the requestId and the brokerId along with the response/exception, will that help?
I believe the brokerId is also logged in the server log for a successful query. We may need to make sure that these are also logged whenever an error/warning pertaining...
We can't really use forceCommit, right? We will need to do in 3 steps: - Pause - Change schema - Resume this will ensure that the new schema is pulled...
How many rows would the new consuming segment be setup to consume? (Currently that is decided when the prev segment is committed).
Changing the idealstate is a much more complex solution, and will have the number of rows to consume as an unresolved point. Today, we compute the new number of rows...
IMO adding a new segment on the same server is very localized change and easy to implement. I strongly suggest we enter the feature with that approach, and then branch...
Yes, syncing up sounds good.