clickhouse-go icon indicating copy to clipboard operation
clickhouse-go copied to clipboard

wait until processImpl completes before closing stream chan

Open nmerkulov opened this issue 3 months ago • 2 comments

Fix race condition in query processing

Problem

There was a race condition between closing the stream channel and writing to it from a background goroutine in conn_query.go.

Race condition scenario:

  1. conn_query.go starts a goroutine that calls c.process(ctx, onProcess) (line 45)
  2. c.process() spawns another goroutine internally that calls c.processImpl() (line 106 in conn_process.go)
  3. c.processImpl() reads data from the server and calls on.data(block) for each data block (line 170 in conn_process.go)
  4. When the context is cancelled, c.process() returns with ctx.Err()
  5. The outer goroutine in conn_query.go immediately closes the stream channel (line 54)
  6. However, the inner goroutine from step 2 may still be running and attempting to write to stream via on.data(block)
  7. Writing to a closed channel causes a panic: panic: send on closed channel

Stack trace example:

goroutine 1: close(stream)
goroutine 2: stream <- b  // panic!

Solution

Modified conn_process.go to ensure that the process() function waits for the internal goroutine to complete before returning, even when the context is cancelled.

Changes:

  • When ctx.Done() is triggered, after calling c.cancel(), we now wait for the goroutine to finish by selecting on errCh or doneCh
  • This guarantees that on.data will not be called after process() returns
  • The stream channel can now be safely closed in conn_query.go without race conditions

Impact

  • Fixes panic when cancelling queries that are actively receiving data
  • Ensures proper cleanup and resource management
  • No breaking changes to public API
  • Minimal performance impact (only adds synchronization on context cancellation path)

nmerkulov avatar Oct 06 '25 14:10 nmerkulov

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Oct 06 '25 14:10 CLAassistant

@nmerkulov thanks for the PR :) do you mind adding a test case to lock the behavior?

kavirajk avatar Oct 15 '25 10:10 kavirajk