parca icon indicating copy to clipboard operation
parca copied to clipboard
verified publisher icon parca-dev

DWARF-based symbolization panic

Open javierhonduco opened this issue 2 years ago • 6 comments

Running on 286993c6be561cfa97133cb5dfaf43859aa7ba88 this panic happens after some minutes of running Parca

[36mooooooooo.[0m
[36m`888   `Y88.[0m
[36m 888   .d88'  .oooo.   oooo d8b  .ooooo.   .oooo.[0m
[36m 888ooo88P'  `P  )88b  `888""8P d88' `"Y8 `P  )88b[0m
[36m 888          .oP"888   888     888        .oP"888[0m
[36m 888         d8(  888   888     888   .o8 d8(  888[0m
[36mo888o        `Y888""8o d888b    `Y8bod8P' `Y888""8o[0m
[36m[0m
[36m[0m
[36m[0m
level=info name=parca ts=2023-03-20T15:34:26.62548982Z caller=main.go:57 msg="maxprocs: Leaving GOMAXPROCS=12: CPU quota undefined"
level=info name=parca ts=2023-03-20T15:34:26.626113299Z caller=factory.go:52 msg="loading bucket configuration"
level=info name=parca ts=2023-03-20T15:34:26.62885896Z caller=badger.go:53 msg="Set nextTxnTs to 0"
level=info name=parca ts=2023-03-20T15:34:26.631314691Z caller=server.go:93 msg="starting server" addr=:7070
level=error name=parca ts=2023-03-20T15:35:06.399221546Z caller=logger.go:38 protocol=grpc grpc.component=server grpc.service=parca.query.v1alpha1.QueryService grpc.method=Query grpc.method_type=unary peer.address=127.0.0.1:42412 grpc.start_time=2023-03-20T15:35:06Z grpc.code=Unknown grpc.error="expected column stacktrace, got 0 columns" grpc.time_ms=0.606 finishedcall=(MISSING)
level=error name=parca ts=2023-03-20T15:35:17.404275707Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-1512456994: no such file or directory"
level=error name=parca ts=2023-03-20T15:35:27.293622199Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-3655194792: no such file or directory"
level=error name=parca ts=2023-03-20T15:35:37.070067244Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-3716938657: no such file or directory"
level=error name=parca ts=2023-03-20T15:35:56.683578412Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-395879669: no such file or directory"
level=error name=parca ts=2023-03-20T15:35:56.6836137Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-1421001896: no such file or directory"
level=error name=parca ts=2023-03-20T15:35:56.683631221Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-4132867600: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:06.706965686Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-2676808588: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:26.764199782Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-2492351849: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:36.706374543Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-2584284251: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:47.070680819Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-3387410815: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:58.151078124Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-642134578: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:58.151106667Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-1316066457: no such file or directory"
level=error name=parca ts=2023-03-20T15:36:58.151118988Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-2527153577: no such file or directory"
level=error name=parca ts=2023-03-20T15:37:06.704386551Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-3536254930: no such file or directory"
level=error name=parca ts=2023-03-20T15:37:06.704423169Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-4080699571: no such file or directory"
level=error name=parca ts=2023-03-20T15:37:06.704451463Z caller=symbolizer.go:315 component=symbolizer msg="failed to remove liner file" err="remove /tmp/parca-symbolizer-1881661482: no such file or directory"
recovered stack trace:
 goroutine 284 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x7a
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines.func1()
	/home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:107 +0x55
panic({0x4e7fcc0, 0xc010cda1b0})
	/usr/local/go/src/runtime/panic.go:890 +0x262
github.com/parca-dev/parca/pkg/symbol/elfutils.(*debugInfoFile).SourceLines(0xc00830f5f0, 0x135477)
	/home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:113 +0xda5
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines(0xc02945fb80, 0x135477)
	/home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:112 +0xd4
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).pcToLines(0xc00010ef70, {0x672e5a8, 0xc02945fb80}, {0xc02056a600, 0x28}, 0x135477)
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:574 +0xac
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).symbolizeLocationsForMapping(0xc00010ef70, {0x672dd90, 0xc001405cc0}, 0xc00e90c600, {0xc0074d5000, 0x88, 0x100})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:517 +0x32fb
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Symbolize(0xc00010ef70, {0x672dd90, 0xc001405cc0}, {0xc02700a000, 0x1ee, 0x1ee})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:297 +0xeb7
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).runSymbolizationCycle(0xc00010ef70, {0x672dd90, 0xc001405cc0})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:221 +0x5bc
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run.func1()
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:190 +0x15f
github.com/parca-dev/parca/pkg/runutil.Repeat(0x2540be400, 0xc001806060, 0xc019d87ed8)
	/home/javierhonduco/code/parca/pkg/runutil/runutil.go:38 +0xef
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run(0xc00010ef70, {0x672dd90, 0xc001405cc0}, 0x2540be400)
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:188 +0xbb
github.com/parca-dev/parca/pkg/parca.Run.func3()
	/home/javierhonduco/code/parca/pkg/parca/parca.go:423 +0x53
github.com/oklog/run.(*Group).Run.func1({0xc0000d1700, 0xc0000d1720})
	/home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x31
created by github.com/oklog/run.(*Group).Run
	/home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0x219

recovered stack trace:
 goroutine 284 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x7a
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines.func1()
	/home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:107 +0x55
panic({0x4e7fcc0, 0xc010f20930})
	/usr/local/go/src/runtime/panic.go:890 +0x262
github.com/parca-dev/parca/pkg/symbol/elfutils.(*debugInfoFile).SourceLines(0xc00830f5f0, 0x104777)
	/home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:113 +0xda5
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines(0xc02945fb80, 0x104777)
	/home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:112 +0xd4
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).pcToLines(0xc00010ef70, {0x672e5a8, 0xc02945fb80}, {0xc02056a600, 0x28}, 0x104777)
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:574 +0xac
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).symbolizeLocationsForMapping(0xc00010ef70, {0x672dd90, 0xc001405cc0}, 0xc00e90c600, {0xc0074d5000, 0x88, 0x100})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:517 +0x32fb
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Symbolize(0xc00010ef70, {0x672dd90, 0xc001405cc0}, {0xc02700a000, 0x1ee, 0x1ee})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:297 +0xeb7
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).runSymbolizationCycle(0xc00010ef70, {0x672dd90, 0xc001405cc0})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:221 +0x5bc
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run.func1()
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:190 +0x15f
github.com/parca-dev/parca/pkg/runutil.Repeat(0x2540be400, 0xc001806060, 0xc019d87ed8)
	/home/javierhonduco/code/parca/pkg/runutil/runutil.go:38 +0xef
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run(0xc00010ef70, {0x672dd90, 0xc001405cc0}, 0x2540be400)
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:188 +0xbb
github.com/parca-dev/parca/pkg/parca.Run.func3()
	/home/javierhonduco/code/parca/pkg/parca/parca.go:423 +0x53
github.com/oklog/run.(*Group).Run.func1({0xc0000d1700, 0xc0000d1720})
	/home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x31
created by github.com/oklog/run.(*Group).Run
	/home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0x219

recovered stack trace:
 goroutine 284 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x7a
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines.func1()
	/home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:107 +0x55
panic({0x4e7fcc0, 0xc011031b60})
	/usr/local/go/src/runtime/panic.go:890 +0x262
github.com/parca-dev/parca/pkg/symbol/elfutils.(*debugInfoFile).SourceLines(0xc00830f5f0, 0x124065)
	/home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:113 +0xda5
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines(0xc02945fb80, 0x124065)
	/home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:112 +0xd4
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).pcToLines(0xc00010ef70, {0x672e5a8, 0xc02945fb80}, {0xc02056a600, 0x28}, 0x124065)
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:574 +0xac
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).symbolizeLocationsForMapping(0xc00010ef70, {0x672dd90, 0xc001405cc0}, 0xc00e90c600, {0xc0074d5000, 0x88, 0x100})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:517 +0x32fb
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Symbolize(0xc00010ef70, {0x672dd90, 0xc001405cc0}, {0xc02700a000, 0x1ee, 0x1ee})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:297 +0xeb7
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).runSymbolizationCycle(0xc00010ef70, {0x672dd90, 0xc001405cc0})
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:221 +0x5bc
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run.func1()
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:190 +0x15f
github.com/parca-dev/parca/pkg/runutil.Repeat(0x2540be400, 0xc001806060, 0xc019d87ed8)
	/home/javierhonduco/code/parca/pkg/runutil/runutil.go:38 +0xef
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run(0xc00010ef70, {0x672dd90, 0xc001405cc0}, 0x2540be400)
	/home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:188 +0xbb
github.com/parca-dev/parca/pkg/parca.Run.func3()
	/home/javierhonduco/code/parca/pkg/parca/parca.go:423 +0x53
github.com/oklog/run.(*Group).Run.func1({0xc0000d1700, 0xc0000d1720})
	/home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x31
created by github.com/oklog/run.(*Group).Run
	/home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0x219
[... and thousands and thousands more lines]

javierhonduco avatar Mar 20 '23 15:03 javierhonduco

Still happens in recent main:

Based of 4180992ef3acdf9409bd:

recovered stack trace:
 goroutine 250 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x7a
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines.func1()
        /home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:107 +0x55
panic({0x5236d60, 0xc007ccc6f0})
        /usr/local/go/src/runtime/panic.go:890 +0x262
github.com/parca-dev/parca/pkg/symbol/elfutils.getFunctionName(0xc007cbf7a0)
        /home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:236 +0x17d
github.com/parca-dev/parca/pkg/symbol/elfutils.(*debugInfoFile).SourceLines(0xc00857bb60, 0x27c0b)
        /home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:114 +0xa66
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines(0xc007ee0180, 0x27c0b)
        /home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:112 +0xd4
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).pcToLines(0xc00021c0d0, {0x6cbe978, 0xc007ee0180}, {0xc00ae16810, 0x28}, 0x27c0b)
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:574 +0xac
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).symbolizeLocationsForMapping(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, 0xc00ad11c80, {0xc00ad2e000, 0x47, 0x80})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:517 +0x32fe
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Symbolize(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, {0xc00a08a000, 0x3b8, 0x3b8})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:297 +0xeb7
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).runSymbolizationCycle(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:221 +0x5bc
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run.func1()
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:190 +0x15f
github.com/parca-dev/parca/pkg/runutil.Repeat(0x2540be400, 0xc0006da0c0, 0xc00ad39d60)
        /home/javierhonduco/code/parca/pkg/runutil/runutil.go:38 +0xef
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, 0x2540be400)
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:188 +0xbb
github.com/parca-dev/parca/pkg/parca.Run.func5.1({0x6cbe198, 0xc00132b4d0})
        /home/javierhonduco/code/parca/pkg/parca/parca.go:503 +0x49
runtime/pprof.Do({0x6cbe198, 0xc00132b4d0}, {{0xc00013a080, 0x1, 0x1}}, 0xc0007a4f10)
        /usr/local/go/src/runtime/pprof/runtime.go:44 +0x13a
github.com/parca-dev/parca/pkg/parca.Run.func5()
        /home/javierhonduco/code/parca/pkg/parca/parca.go:502 +0x16c
github.com/oklog/run.(*Group).Run.func1({0xc001597120, 0xc001597140})
        /home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x31
created by github.com/oklog/run.(*Group).Run
        /home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0x219

recovered stack trace:
 goroutine 250 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x7a
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines.func1()
        /home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:107 +0x55
panic({0x5236d60, 0xc01d035ce0})
        /usr/local/go/src/runtime/panic.go:890 +0x262
github.com/parca-dev/parca/pkg/symbol/elfutils.getFunctionName(0xc007cbf7a0)
        /home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:236 +0x17d
github.com/parca-dev/parca/pkg/symbol/elfutils.(*debugInfoFile).SourceLines(0xc00857bb60, 0x27c0b)
        /home/javierhonduco/code/parca/pkg/symbol/elfutils/debuginfofile.go:114 +0xa66
github.com/parca-dev/parca/pkg/symbol/addr2line.(*DwarfLiner).PCToLines(0xc007ee0180, 0x27c0b)
        /home/javierhonduco/code/parca/pkg/symbol/addr2line/dwarf.go:112 +0xd4
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).pcToLines(0xc00021c0d0, {0x6cbe978, 0xc007ee0180}, {0xc01cfbbfb0, 0x28}, 0x27c0b)
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:574 +0xac
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).symbolizeLocationsForMapping(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, 0xc012595980, {0xc00b70c000, 0x69, 0x80})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:517 +0x32fe
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Symbolize(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, {0xc0013d2000, 0x457, 0x457})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:297 +0xeb7
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).runSymbolizationCycle(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:221 +0x5bc
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run.func1()
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:190 +0x15f
github.com/parca-dev/parca/pkg/runutil.Repeat(0x2540be400, 0xc0006da0c0, 0xc00ad39d60)
        /home/javierhonduco/code/parca/pkg/runutil/runutil.go:38 +0xef
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, 0x2540be400)
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:188 +0xbb
github.com/parca-dev/parca/pkg/parca.Run.func5.1({0x6cbe198, 0xc00132b4d0})
        /home/javierhonduco/code/parca/pkg/parca/parca.go:503 +0x49
runtime/pprof.Do({0x6cbe198, 0xc00132b4d0}, {{0xc00013a080, 0x1, 0x1}}, 0xc0007a4f10)
        /usr/local/go/src/runtime/pprof/runtime.go:44 +0x13a
github.com/parca-dev/parca/pkg/parca.Run.func5()
        /home/javierhonduco/code/parca/pkg/parca/parca.go:502 +0x16c
github.com/oklog/run.(*Group).Run.func1({0xc001597120, 0xc001597140})
        /home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x31
created by github.com/oklog/run.(*Group).Run
        /home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0x219

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x6b9718]

goroutine 250 [running]:
github.com/go-kit/log.(*context).Log(0xc020ed5810, {0xc007bbcde0, 0x6, 0x6})
        /home/javierhonduco/go/pkg/mod/github.com/go-kit/[email protected]/log.go:168 +0x678
github.com/parca-dev/parca/pkg/debuginfo.(*DebuginfodClientObjectStorageCache).getAndCache(0xc001813680, {0x6cbe198, 0xc01e7da330}, {0xc01e79d710, 0x28})
        /home/javierhonduco/code/parca/pkg/debuginfo/debuginfod.go:248 +0x593
github.com/parca-dev/parca/pkg/debuginfo.(*DebuginfodClientObjectStorageCache).Get(0xc001813680, {0x6cbe198, 0xc01e7da330}, {0xc01e79d710, 0x28})
        /home/javierhonduco/code/parca/pkg/debuginfo/debuginfod.go:231 +0x1d8
github.com/parca-dev/parca/pkg/debuginfo.(*DebuginfodExistsClientCache).Get(0xc001a058a8, {0x6cbe198, 0xc01e7da330}, {0xc01e79d710, 0x28})
        /home/javierhonduco/code/parca/pkg/debuginfo/debuginfod.go:382 +0xa2
github.com/parca-dev/parca/pkg/debuginfo.(*DebuginfodTracingClient).Get(0xc001596fa0, {0x6cbe198, 0xc01e7da330}, {0xc01e79d710, 0x28})
        /home/javierhonduco/code/parca/pkg/debuginfo/debuginfod.go:430 +0x386
github.com/parca-dev/parca/pkg/debuginfo.(*ParallelDebuginfodClients).Get(0xc001596fc0, {0x6cbe198, 0xc00132b4d0}, {0xc01e7aa780, 0x17}, {0xc01e79d710, 0x28})
        /home/javierhonduco/code/parca/pkg/debuginfo/debuginfod.go:146 +0x153
github.com/parca-dev/parca/pkg/debuginfo.(*Fetcher).fetchFromDebuginfod(0xc001597080, {0x6cbe198, 0xc00132b4d0}, 0xc00036c9a0)
        /home/javierhonduco/code/parca/pkg/debuginfo/fetcher.go:69 +0x14d
github.com/parca-dev/parca/pkg/debuginfo.(*Fetcher).FetchDebuginfo(0xc001597080, {0x6cbe198, 0xc00132b4d0}, 0xc00036c9a0)
        /home/javierhonduco/code/parca/pkg/debuginfo/fetcher.go:51 +0x185
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).symbolizeLocationsForMapping(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, 0xc01259e200, {0xc01cfda460, 0x3, 0x4})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:429 +0xd4d
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Symbolize(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, {0xc0013d2000, 0x457, 0x457})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:297 +0xeb7
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).runSymbolizationCycle(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0})
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:221 +0x5bc
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run.func1()
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:190 +0x15f
github.com/parca-dev/parca/pkg/runutil.Repeat(0x2540be400, 0xc0006da0c0, 0xc00ad39d60)
        /home/javierhonduco/code/parca/pkg/runutil/runutil.go:38 +0xef
github.com/parca-dev/parca/pkg/symbolizer.(*Symbolizer).Run(0xc00021c0d0, {0x6cbe198, 0xc00132b4d0}, 0x2540be400)
        /home/javierhonduco/code/parca/pkg/symbolizer/symbolizer.go:188 +0xbb
github.com/parca-dev/parca/pkg/parca.Run.func5.1({0x6cbe198, 0xc00132b4d0})
        /home/javierhonduco/code/parca/pkg/parca/parca.go:503 +0x49
runtime/pprof.Do({0x6cbe198, 0xc00132b4d0}, {{0xc00013a080, 0x1, 0x1}}, 0xc0007a4f10)
        /usr/local/go/src/runtime/pprof/runtime.go:44 +0x13a
github.com/parca-dev/parca/pkg/parca.Run.func5()
        /home/javierhonduco/code/parca/pkg/parca/parca.go:502 +0x16c
github.com/oklog/run.(*Group).Run.func1({0xc001597120, 0xc001597140})
        /home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x31
created by github.com/oklog/run.(*Group).Run
        /home/javierhonduco/go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0x219

javierhonduco avatar Aug 18 '23 10:08 javierhonduco

I can have a look at it next week.

kakkoyun avatar Aug 18 '23 11:08 kakkoyun

I can have a look at it next week.

I guess I've dropped the ball on this one...

I'll try to priorities

kakkoyun avatar Dec 05 '23 16:12 kakkoyun

Just met the same problem with recent main

dreamerlzl avatar Jan 31 '24 02:01 dreamerlzl

Seeing this with version v0.20.0

TomHellier avatar Apr 24 '24 07:04 TomHellier

A bit of an update, we’ve landed the new revamped symbolizer infrastructure in Parca and we should not be able to fix this bug. Hopefully the next version of Parca will have this fixed!

brancz avatar May 12 '24 08:05 brancz