cmd/compile: internal compiler error: bad live variable at entry

[mkumatag]

What version of Go are you using (go version)?

$ go version
tip commit - 1eeb257b88

Does this issue reproduce with the latest release?

yes, master branch

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

$ git clone https://github.com/kubernetes/kubernetes.git
$ cd kubernetes
$ make test

What did you expect to see?

Tests should be successful

What did you see instead?

+ make test
+++ [0920 02:22:47] Building go targets for linux/ppc64le
    k8s.io/kubernetes/hack/make-rules/helpers/go2make (non-static)
+++ [0920 02:23:44] Building go targets for linux/ppc64le
    k8s.io/code-generator/cmd/prerelease-lifecycle-gen (non-static)
+++ [0920 02:24:19] Generating prerelease lifecycle code for 28 targets
+++ [0920 02:24:32] Building go targets for linux/ppc64le
    k8s.io/code-generator/cmd/deepcopy-gen (non-static)
+++ [0920 02:24:49] Generating deepcopy code for 243 targets
+++ [0920 02:25:24] Building go targets for linux/ppc64le
    k8s.io/code-generator/cmd/defaulter-gen (non-static)
+++ [0920 02:25:35] Generating defaulter code for 96 targets
+++ [0920 02:26:18] Building go targets for linux/ppc64le
    k8s.io/code-generator/cmd/conversion-gen (non-static)
+++ [0920 02:26:29] Generating conversion code for 133 targets
+++ [0920 02:28:03] Building go targets for linux/ppc64le
    k8s.io/kube-openapi/cmd/openapi-gen (non-static)
+++ [0920 02:29:26] Generating openapi code for KUBE
+++ [0920 02:30:35] Generating openapi code for AGGREGATOR
+++ [0920 02:30:43] Generating openapi code for APIEXTENSIONS
+++ [0920 02:30:50] Generating openapi code for CODEGEN
+++ [0920 02:30:59] Generating openapi code for SAMPLEAPISERVER
+++ [0920 02:31:15] Running tests without code coverage and with -race
# k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/core/v1
vendor/k8s.io/client-go/kubernetes/typed/core/v1/event_expansion.go:120:2: internal compiler error: bad live variable at entry of (*events).Search: stringRefUID (type string)

goroutine 1 [running]:
runtime/debug.Stack()
	/root/go/src/runtime/debug/stack.go:24 +0x6c
cmd/compile/internal/base.FatalfAt({0x1e66c80?, 0xc0?}, {0x9317b8, 0x24}, {0xc002d350d8, 0x2, 0x2})
	/root/go/src/cmd/compile/internal/base/print.go:227 +0x270
cmd/compile/internal/liveness.(*liveness).epilogue(0xc001e66c80)
	/root/go/src/cmd/compile/internal/liveness/plive.go:830 +0xa2c
cmd/compile/internal/liveness.Compute(0xc001b5a8c0, 0xc001d77860, 0x258e10?, 0x259408?)
	/root/go/src/cmd/compile/internal/liveness/plive.go:1340 +0x90
cmd/compile/internal/ssagen.genssa(0xc001d77860, 0xc001e00e00)
	/root/go/src/cmd/compile/internal/ssagen/ssa.go:6906 +0xac
cmd/compile/internal/ssagen.Compile(0xc001b5a8c0, 0xc000076800?)
	/root/go/src/cmd/compile/internal/ssagen/pgen.go:197 +0x278
cmd/compile/internal/gc.compileFunctions.func4.1(0x1?)
	/root/go/src/cmd/compile/internal/gc/compile.go:153 +0x4c
cmd/compile/internal/gc.compileFunctions.func2(0x7fff8b7a0a68?)
	/root/go/src/cmd/compile/internal/gc/compile.go:125 +0x3c
cmd/compile/internal/gc.compileFunctions.func4({0xc002e73000, 0x1c5, 0x200?})
	/root/go/src/cmd/compile/internal/gc/compile.go:152 +0x7c
cmd/compile/internal/gc.compileFunctions()
	/root/go/src/cmd/compile/internal/gc/compile.go:163 +0x188
cmd/compile/internal/gc.Main(0x948570)
	/root/go/src/cmd/compile/internal/gc/main.go:301 +0x13a4
main.main()
	/root/go/src/cmd/compile/main.go:57 +0xfc

[laboger]

This happens on ppc64le.

[cuonglm]

This happens on ppc64le.

Are you sure https://github.com/golang/go/commit/00bee6d9a4c3ed6168350fc6551043ff7a1895f2 is the first bad commit?

[mkumatag]

Last known successful was with https://github.com/golang/go/commit/7db923fe565465f292d3e62d6c7ded86e724062d

[mknyszek]

The same build failure in k8s: https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/ci-build-and-push-k8s-at-golang-tip/1572196947165450240

CC @erifan @cherrymui @randall77

[cherrymui]

https://github.com/golang/go/issues/55269#issue-1379533143 mentions commit cf53990 or 4c414c7 . It is unlikely the former, but the latter is possible, which changes scheduling.

Is there a way we can do the build ourselves? Preferably in a cross-compile environment without running the make file. Thanks.

[cuonglm]

ppc64le

I can reproduce on linux-amd64 as well:

$ git clone https://github.com/kubernetes/client-go.git
$ cd client-go/kubernetes/typed/core/v1
$ go build
go: downloading k8s.io/api v0.0.0-20220920024110-052d63f042d1
go: downloading k8s.io/apimachinery v0.0.0-20220920023906-f8159af4957e
go: downloading github.com/google/gofuzz v1.1.0
go: downloading k8s.io/klog/v2 v2.80.1
go: downloading k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed
go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8
go: downloading k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1
go: downloading sigs.k8s.io/yaml v1.2.0
go: downloading github.com/google/gnostic v0.5.7-v3refs
go: downloading github.com/go-openapi/swag v0.19.14
go: downloading github.com/go-openapi/jsonreference v0.19.5
go: downloading github.com/PuerkitoBio/purell v1.1.1
go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
go: downloading github.com/mailru/easyjson v0.7.6
# k8s.io/client-go/kubernetes/typed/core/v1
./event_expansion.go:120:2: internal compiler error: bad live variable at entry of (*events).Search: stringRefUID (type string)

goroutine 158 [running]:
runtime/debug.Stack()
	/home/cuonglm/sources/go/src/runtime/debug/stack.go:24 +0x65
cmd/compile/internal/base.FatalfAt({0x1dc0280?, 0xc0?}, {0xd937c8, 0x24}, {0xc0023e37a0, 0x2, 0x2})
	/home/cuonglm/sources/go/src/cmd/compile/internal/base/print.go:227 +0x1d7
cmd/compile/internal/liveness.(*liveness).epilogue(0xc001dc0280)
	/home/cuonglm/sources/go/src/cmd/compile/internal/liveness/plive.go:830 +0xb4e
cmd/compile/internal/liveness.Compute(0xc000fae8c0, 0xc00181b040, 0x0?, 0x0?)
	/home/cuonglm/sources/go/src/cmd/compile/internal/liveness/plive.go:1340 +0x8f
cmd/compile/internal/ssagen.genssa(0xc00181b040, 0xc001c09f10)
	/home/cuonglm/sources/go/src/cmd/compile/internal/ssagen/ssa.go:6906 +0xa5
cmd/compile/internal/ssagen.Compile(0xc000fae8c0, 0xc0004b1f90?)
	/home/cuonglm/sources/go/src/cmd/compile/internal/ssagen/pgen.go:197 +0x26f
cmd/compile/internal/gc.compileFunctions.func4.1(0x0?)
	/home/cuonglm/sources/go/src/cmd/compile/internal/gc/compile.go:153 +0x3a
cmd/compile/internal/gc.compileFunctions.func3.1()
	/home/cuonglm/sources/go/src/cmd/compile/internal/gc/compile.go:140 +0x4d
created by cmd/compile/internal/gc.compileFunctions.func3
	/home/cuonglm/sources/go/src/cmd/compile/internal/gc/compile.go:138 +0x78

[cuonglm]

Revert 4c414c7673af6b2aedee276d2e62cb2910eb19f3 makes the build success.

[randall77]

Created revert, CL 432196

[erifan]

Sorry for introducing this issue, and thanks @randall77 for reverting 4c414c7673. I will find out the cause of this issue and resubmit.

[gopherbot]

Change https://go.dev/cl/432275 mentions this issue: cmd/compile: enable carry chain scheduling for arm64

[erifan]

The problem here is that CL 424907 schedules the address fetching operation (MOVDaddr) of a local variable before the variable definition operation (VarDef). Please see v127 and v124 in the image below.

But in fact the scheduling logic is correct or expected. A simpler fix is to adjust the priority of ScoreCarryChainTail. But I think it might be a better fix if the address fetching operation depends on the variable's definition operation, like op LocalAddr .

[cherrymui]

The culprit CL has been reverted. I think we can close this.

[cherrymui]

But I think it might be a better fix if the address fetching operation depends on the variable's definition operation, like op LocalAddr

We thought about this before. But the problem is that introducing memory ordering for MOVDaddr may make it not able to be CSE'd or rematerialized. We didn't choose that path. I think it may be possible to make the schedule more careful about this, maybe schedule VarDef early (it doesn't generate code).

[erifan]

Yeah, CL 432275 does just that. It's just that this looks a little fragile.