runtime: fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted [mipsle]

[bradfitz]

We just got a bug report from a user that AllThreadsSyscall6 (via a call to syscall.Setuid without cgo) crashed.

Does AllThreadsSyscall not work on mipsle?

Go 1.19.2 GOARCH=mipsle (https://openwrt.org/toh/hwdata/gehua/gehua_ghl-r-001) CGO_ENABLED=0

trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted
trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted
trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted
trap:4081, a123456=[1,30851488,0,0,0,0]
results: got {r1=0,r2=4081,errno=0}, want {r1=0,r2=30851488,errno=0
fatal error: AllThreadsSyscall6 results differ between threads; runtime corrupted

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c53d08, 0x1c53d88, 0x1c005a0)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c53d08, 0x1c53d88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

goroutine 1 [running]:
        goroutine running on other thread; stack unavailable

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c5bd08, 0x1c5bd88, 0x1c00780)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c5bd08, 0x1c5bd88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c8bd08, 0x1c8bd88, 0x1c820f0)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c8bd08, 0x1c8bd88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

goroutine 0 [idle]:
runtime.fatal({0xb8af82, 0x44})
        runtime/panic.go:1066 +0x54
runtime.runPerThreadSyscall()
        runtime/os_linux.go:884 +0x3e0
runtime.sighandler(0x21, 0x1c43d08, 0x1c43d88, 0x1c002d0)
        runtime/signal_unix.go:647 +0x710
runtime.sigtrampgo(0x21, 0x1c43d08, 0x1c43d88)
        runtime/signal_unix.go:479 +0x208
runtime.sigtramp()
        runtime/sys_linux_mipsx.s:325 +0x40

(from https://github.com/tailscale/tailscale/issues/6060)

/cc @hdhoang @maisem @ianlancetaylor @aclements

[bradfitz]

I don't have this hardware but I could try to reproduce it on qemu at least if that'd help and this isn't a known dup?

[bcmills]

This might be related to #51192? (CC @golang/runtime)

[bcmills]

Hmm, https://man7.org/linux/man-pages/man2/syscall.2.html#NOTES seems to indicate that the syscall ABI for MIPS does return two values. 🤔

[bcmills]

(attn @golang/mips)

[prattmic]

We've previously fixed this for ppc64 (#42178) with https://go.dev/cl/266202 and later https://go.dev/cl/385796. The latter is probably what we want if mips really doesn't use r2.

[mknyszek]

Since we have a good idea of what this might be and the fix is straightforward, assigning to @golang/mips to follow up.

Feel free to toss it back if you run into issues.

[prattmic]

@cherrymui pointed out that this register may only be used for a few system calls. But if the kernel does not clobber it, it may be best to have Syscall6 zero the register prior to the SYSCALL instruction.

[gopherbot]

Change https://go.dev/cl/445695 mentions this issue: runtime: add missing closing curly brace in runtime corruption error message