go runtime: unexpected return pc for runtime cacheSpan [...]

Go version

go1.23.2

Output of `go env` in your module/workspace:

Docker alpine go1.23.2 on amd64

What did you do?

Ran out project as usual (?)

What did you see happen?

	fatal error: unexpected signal during runtime execution
	[signal SIGSEGV: segmentation violation code=0x1 addr=0x5f pc=0x41847f]
	 goroutine 5340061 gp=0xc000df7180 m=25 mp=0xc000583108 [running]:
	 runtime.throw({0x1b1bd01?, 0x20?})
	 	runtime/panic.go:1067 +0x48 fp=0xc005efa5a8 sp=0xc005efa578 pc=0x476e28
	 runtime.sigpanic()
	 	runtime/signal_unix.go:884 +0x3c9 fp=0xc005efa608 sp=0xc005efa5a8 pc=0x4790c9
	 runtime.(*mcentral).cacheSpan(0x417d53)
	 	runtime/mcentral.go:184 +0x15f fp=0xc005efa680 sp=0xc005efa608 pc=0x41847f
	 runtime: g 5340061: unexpected return pc for runtime.(*mcentral).cacheSpan called from 0xc005efa6b8
	 stack: frame={sp:0xc005efa608, fp:0xc005efa680} stack=[0xc005ef8000,0xc005efc000)
	 0x000000c005efa508:  0x00007fb023b0d688  0x0000000000000020
	 0x000000c005efa518:  0x0000000000000017  0x000000000047d4d2 <runtime.systemstack+0x0000000000000032>
	 0x000000c005efa528:  0x000000c005efa568  0x000000000043d1d8 <runtime.fatalthrow+0x0000000000000058>
	 0x000000c005efa538:  0x000000c005efa548  0x000000c000df7180
	 0x000000c005efa548:  0x000000000043d200 <runtime.fatalthrow.func1+0x0000000000000000>  0x000000c000df7180
	 0x000000c005efa558:  0x0000000000476e28 <runtime.throw+0x0000000000000048>  0x000000c005efa578
	 0x000000c005efa568:  0x000000c005efa598  0x0000000000476e28 <runtime.throw+0x0000000000000048>
	 0x000000c005efa578:  0x000000c005efa580  0x000000000043cda0 <runtime.throw.func1+0x0000000000000000>
	 0x000000c005efa588:  0x0000000001b1bd01  0x000000000000002a
	 0x000000c005efa598:  0x000000c005efa5f8  0x00000000004790c9 <runtime.sigpanic+0x00000000000003c9>
	 0x000000c005efa5a8:  0x0000000001b1bd01  0x0000000000000020
	 0x000000c005efa5b8:  0x0000000000000020  0x000000000178e740
	 0x000000c005efa5c8:  0x000000c0125879a0  0x000000c005efa648
	 0x000000c005efa5d8:  0x000000c000df7180  0x000000000178e740
	 0x000000c005efa5e8:  0x000000c005efa650  0x0000000000436d2a <runtime.(*spanSet).push+0x000000000000002a>
	 0x000000c005efa5f8:  0x000000c005efa678  0x000000000041847f <runtime.(*mcentral).cacheSpan+0x000000000000015f>
	 0x000000c005efa608: <0x000000000041843f <runtime.(*mcentral).cacheSpan+0x000000000000011f>  0x0000060405efa648
	 0x000000c005efa618:  0x00000000005d04e5 <github.com/ethereum/go-ethereum/rlp.makeListDecoder.func3+0x0000000000000025>  0x0000000000000197
	 0x000000c005efa628:  0x0001c90900000000  0x000000c012587988
	 0x000000c005efa638:  0x000000000178e740  0x000000c0125879a0
	 0x000000c005efa648:  0x000000c005efa7e8  0x00007fb0c8602cf8
	 0x000000c005efa658:  0x0000000000000005  0x0000000002eee528
	 0x000000c005efa668:  0x00007fb0a5f4e440  0x000000000178f9c0
	 0x000000c005efa678:  0x000000c005efa6b8 >0x0000000000417d53 <runtime.(*mcache).refill+0x0000000000000153>
	 0x000000c005efa688:  0x0000000002eee4a8  0x0000000000000038
	 0x000000c005efa698:  0x000000c005efa6f0  0x0000000000000008
	 0x000000c005efa6a8:  0x00007fb0a5f4e440  0x000000000000001a
	 0x000000c005efa6b8:  0x000000c005efa6f0  0x00000000004119e5 <runtime.(*mcache).nextFree+0x0000000000000085>
	 0x000000c005efa6c8:  0x00007fb113c911d8  0x00000000004be91a <reflect.NewAt+0x000000000000003a>
	 0x000000c005efa6d8:  0x002d0000002d0d80  0x00007fb0a5f4e440
	 0x000000c005efa6e8:  0x000000000000001a  0x000000c005efa790
	 0x000000c005efa6f8:  0x000000000047176d <runtime.mallocgc+0x00000000000004cd>  0x00007fb113c911d8
	 0x000000c005efa708:  0x000000000000001a  0x000000c005efa730
	 0x000000c005efa718:  0x0000000000477ff3 <reflect.resolveTypeOff+0x0000000000000013>  0x0000000000000199
	 0x000000c005efa728:  0x00000000001f0d80  0x000000c005efa7b8
	 0x000000c005efa738:  0x00000000004c3c7f <reflect.(*rtype).ptrTo+0x000000000000009f>  0x0000000000000016
	 0x000000c005efa748:  0x00007fb113c911d8  0x00000000000000b0
	 0x000000c005efa758:  0x00000000018b0d80  0x000000c005efa758
	 0x000000c005efa768:  0x000000c000583108  0x000000c005efa7e8
	 0x000000c005efa778:  0x00000000004bab72 <reflect.Value.Set+0x0000000000000132>

What did you expect to see?

No big boom :)

Oct 16 '24 06:10 karalabe

Related Issues and Documentation

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

Oct 16 '24 06:10 gabyhelp

CC @golang/runtime

Oct 16 '24 12:10 ianlancetaylor

The span pointer in the variable is definitely bogus, but I don't see how this is possible. This code path has not changed recently. This looks like memory corruption of the stack, especially given that the unwinder crashes trying to print a stack trace.

I don't think we'll be able to make much progress without a reproducer. But here are some thoughts/questions.

Maybe give GODEBUG=asyncpreemptoff=1 a try. It's gotten far less buggy over the years but who knows.
Are you using unsafe or cgo widely? Have you made any changes to your codebase related to these recently?
I see you're running on Alpine; have you upgraded the image recently? There could be some incompatibility with a new version of musl. If it's a pure Go program, you can possibly try building with CGO_ENABLED=0 to avoid using libc on Linux to rule this out.

Thanks.

Oct 16 '24 13:10 mknyszek

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

Nov 23 '24 20:11 gopherbot

runtime: unexpected return pc for runtime cacheSpan [...]

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

Output of `go env` in your module/workspace: