parca icon indicating copy to clipboard operation
parca copied to clipboard

Data races in parca

Open javierhonduco opened this issue 3 years ago • 1 comments

Lately, we've been debugging some issues that were rooted in race conditions (https://github.com/parca-dev/parca-agent/pull/444, https://github.com/parca-dev/parca/pull/1136). After several hours of running, we no longer store broken debug files, but I wanted to make sure with the race detector enabled:

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -34,7 +34,7 @@ go/deps:
 .PHONY: go/bin
 go/bin: go/deps
        mkdir -p ./bin
-       go build -o bin/ ./cmd/parca
+       go build -race -o bin/ ./cmd/parca

And it reported some data races (most of them are dups), all of them seem unrelated to the debug path (good!), but we should look into them. Some of them happen in parquet-go, but not sure if it's because of not using them with the correct locking on our side or if they are bugs in FrostDB / parquet-go. Let's split this issue accordingly as we troubleshoot them :)

(Will add a build flag to easily add the race detector)

Raw logs: http://sprunge.us/oUsfdr based off https://github.com/parca-dev/parca/commit/bcf2171c4374f29211901526ea1db51aff843d24

javierhonduco avatar Jun 21 '22 13:06 javierhonduco

The race in the logs looks like what we fixed in this one https://github.com/polarsignals/frostdb/pull/96

We should update frostdb and try again.

brancz avatar Jun 21 '22 16:06 brancz