pushgateway icon indicating copy to clipboard operation
pushgateway copied to clipboard

Replace RLock with Lock in cleanupStaleValues

Open matijavizintin opened this issue 1 year ago • 0 comments

cleanupStaleValues function needs a read-write lock because it's deleting values from dms.metricGroups.

Using a read lock causes a panic fatal error: concurrent map iteration and map write

goroutine 422 [running]:
runtime.throw(0xa817fa, 0x26)
        /usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc01efd2f10 sp=0xc01efd2ee0 pc=0x438132
runtime.mapiternext(0xc01efd3170)
        /usr/local/go/src/runtime/map.go:858 +0x54c fp=0xc01efd2f90 sp=0xc01efd2f10 pc=0x410cac
github.com/prometheus/pushgateway/storage.(*DiskMetricStore).GetMetricFamilies(0xc000380ba0, 0x0, 0x0, 0x0)
        /root/workspace/src/github.com/prometheus/pushgateway/storage/diskmetricstore.go:148 +0xf2f fp=0xc01efd3300 sp=0xc01efd2f90 pc=0x93484f
main.main.func1(0x10, 0x50, 0x39, 0x9e3779b185ebca87, 0xc01efd36a0)
        /root/workspace/src/github.com/prometheus/pushgateway/main.go:107 +0x2a fp=0xc01efd3330 sp=0xc01efd3300 pc=0x95598a
goroutine 59 [runnable]:
github.com/prometheus/pushgateway/storage.(*DiskMetricStore).cleanupStaleValues(0xc000380ba0, 0x45d964b800)
        /root/workspace/src/github.com/prometheus/pushgateway/storage/diskmetricstore.go:473 +0x10b
github.com/prometheus/pushgateway/storage.(*DiskMetricStore).doCleanUpInReguarInterval(0xc000380ba0, 0x45d964b800)
        /root/workspace/src/github.com/prometheus/pushgateway/storage/diskmetricstore.go:460 +0x3a
created by github.com/prometheus/pushgateway/storage.NewDiskMetricStore
        /root/workspace/src/github.com/prometheus/pushgateway/storage/diskmetricstore.go:105 +0x215

matijavizintin avatar Mar 01 '24 12:03 matijavizintin