pulsar-client-go icon indicating copy to clipboard operation
pulsar-client-go copied to clipboard

concurrent map iteration and map write

Open maolinHu opened this issue 3 years ago • 8 comments

Expected behavior

企业微信截图_16652163562552

Actual behavior

Tell us what happens instead

Steps to reproduce

How can we reproduce the issue

System configuration

Pulsar version: 0.8.1

maolinHu avatar Oct 08 '22 08:10 maolinHu

Can you describe the steps to reproduce this bug?

labuladong avatar Oct 09 '22 02:10 labuladong

There is a new feature in Go 1.8 to detect concurrent edit and access to a map. It's described at https://tip.golang.org/doc/go1.8#mapiter The problem was reported (or panicked) by go map. This should have been reported by a race detector. I will do a PR to fix it, probably use mutex to protect the map access.

zzzming avatar Oct 12 '22 15:10 zzzming

properties := make(map[string]string) for _, data := range Arr { msg := &pulsar.ProducerMessage{ Payload: data, Properties: properties, } p.sendAsync(ctx, msg, fn) }

I guess it is caused by multiplexing map(pulsar.ProducerMessage.Properties) when send asyn, This is an occasional bug.

maolinHu avatar Oct 13 '22 02:10 maolinHu

In Go, a map is a reference type. Properties is a map. A producer application can still update the map while it's passed to the go client. This might be where the problem is. The original stack trace is from internalSend that is a Send(). I guess the problem should exist in both Send and SendAsync. The producer application should not concurrently update the properties.

zzzming avatar Oct 16 '22 03:10 zzzming