kafka-go icon indicating copy to clipboard operation
kafka-go copied to clipboard

Fix: Closing kafka Writer during WriteMessages causes a potential hang #1307

Open iamgoroot opened this issue 1 year ago • 0 comments

Fixes issue #1307 with closing Writer while WriteMessages() is ongoing

https://github.com/segmentio/kafka-go/issues/1307

Cause of the issue:

  • Close takes the lock and goes through writers created in batchMessages closing them in a loop
  • Close hangs in wait while ongoing messages are processed
  • meanwhile in WriteMessages there's only one check for writer being closed. If WriteMessages is past that check while closing - new writers are created that will not be released in Close since it's already in a wait

Solution: WriteMessages should fail with error in case the writer is closed upon acquiring mutex second time and before creating new writers (or map of writers) that won't be closed.

iamgoroot avatar Jul 23 '24 23:07 iamgoroot