Burrow
Burrow copied to clipboard
inmemory deleteTopic is not thread-safe
We're seeing 'fatal error: concurrent map iteration and map write' errors from burrow. deleteTopic should be locking the clusterMap.consumer before iterating on it similarly to what fetchConsumerList does
https://bitbucket.citadelgroup.com/projects/PS/repos/burrow/browse/core/internal/storage/inmemory.go#647,717,722
Here is the stack trace we're getting
fatal error: concurrent map iteration and map write
goroutine 220 [running]: runtime.throw({0xc0cc01, 0x415147}) /usr/local/go/src/runtime/panic.go:1198 +0x71 fp=0xc0002fb9f8 sp=0xc0002fb9c8 pc=0x437ad1 runtime.mapiternext(0xc028b36cd8) /usr/local/go/src/runtime/map.go:858 +0x4eb fp=0xc0002fba68 sp=0xc0002fb9f8 pc=0x411aab bitbucket.citadelgroup.com/ps/burrow/core/internal/storage.(*InMemoryStorage).deleteTopic(0xa, 0xc043c8ee00, 0xc03d193500) /burrow/src/core/internal/storage/inmemory.go:647 +0x12f fp=0xc0002fbb28 sp=0xc0002fba68 pc=0x98cb2f bitbucket.citadelgroup.com/ps/burrow/core/internal/storage.(*InMemoryStorage).deleteTopic-fm(0xc0003800c0, 0xc03a6f7680) /burrow/src/core/internal/storage/inmemory.go:639 +0x2f fp=0xc0002fbb50 sp=0xc0002fbb28 pc=0x98f02f bitbucket.citadelgroup.com/ps/burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0002ee780, 0x12, 0x0) /burrow/src/core/internal/storage/inmemory.go:237 +0x10d1 fp=0xc0002fbfb8 sp=0xc0002fbb50 pc=0x9898b1 bitbucket.citadelgroup.com/ps/burrow/core/internal/storage.(*InMemoryStorage).Start<C2><B7>dwrap<C2><B7>3() /burrow/src/core/internal/storage/inmemory.go:191 +0x2e fp=0xc0002fbfe0 sp=0xc0002fbfb8 pc=0x9886ce runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc0002fbfe8 sp=0xc0002fbfe0 pc=0x468261 created by bitbucket.citadelgroup.com/ps/burrow/core/internal/storage.(*InMemoryStorage).Start /burrow/src/core/internal/storage/inmemory.go:191 +0x20c
+1 this happens to us as well