reactor
Drain SinkManyEmitterProcessor buffer after cancel
The internal buffer/queue in SinkManyEmitterProcessor will be drained after all the subscriptions are canceled.
As explained here the queue/buffer in SinkManyEmitterProcessor is not drained properly after the last subscriber canceled the subscription. This was happening due to the WIP marker is left in an unclean state. This PR fixes the issue by updating the WIP marker. I am not sure if this is the ideal approach though.
Fixes #3715
@chemicL I addressed your comments and also added a JCStress test. But I am not sure whether the test case is correct.
Hey. Thank you for the effort. Your PR has allowed to unravel some hidden complexities and is pushing the research into this space forward. Unfortunately, I'm not able to accept this in the current form. There are multiple issues here and it would require quite an effort on my part to help correct these. The issues I can see are:
- Consider reviewing how the concept of WIP (work-in-progress) is used in the codebase -> this is a mechanism to signal a Thread that is currently draining that more work needs to be considered and also as means to guard access to the critical section in a lock-free fashion
- in case of cancellation, the return value can't be "OK", but rather "FAIL_CANCELLED"
- the JCStress test should not consider an item stuck in the queue forever as "acceptable, interesting" but rather fail the evaluation
In general, I think the SinkManyEmitterProcessor is a super critical piece in the Sinks API and should be handled with care. The attempt to fix it should consider a broad set of circumstances and would require means to atomically represent the cancelled state and a way to reject new additions and ensure proper cleanup in the face of high concurrency. With that, I'm sorry to reject the PR.
