`fatal error: invalid function symbol table` when using cgo starting in go 1.21

[dzbarsky]

What version of rules_go are you using?

0.50.1

What version of gazelle are you using?

0.38

What version of Bazel are you using?

7.2.1

Does this issue reproduce with the latest releases of all the above?

yes

What did you do?

See https://github.com/DavidZbarsky-at/cgo-repro for repro

This was not an issue with go 1.20.1 but is an issue starting in 1.21 See https://github.com/google/skia/blob/main/.bazelrc#L41-L45 for another example in the wild

What did you expect to see?

No-op binary runs correctly

What did you see instead?

runtime: pcHeader: magic= 0xfffffff1 pad1= 0 pad2= 0 minLC= 4 ptrSize= 8 pcHeader.textStart= 0xab280 text= 0xaaaacbbab280 pluginpath= 
fatal error: invalid function symbol table
runtime: panic before malloc heap initialized

runtime stack:
runtime.throw({0xaaaacbb2439a?, 0x0?})
	GOROOT/src/runtime/panic.go:1023 +0x40 fp=0xffffc70b8870 sp=0xffffc70b8840 pc=0xaaaacbbde990
runtime.moduledataverify1(0xaaaacbcaaac0?)
	GOROOT/src/runtime/symtab.go:532 +0x654 fp=0xffffc70b89a0 sp=0xffffc70b8870 pc=0xaaaacbbfcc84
runtime.moduledataverify(...)
	GOROOT/src/runtime/symtab.go:518
runtime.schedinit()
	GOROOT/src/runtime/proc.go:783 +0x68 fp=0xffffc70b8a20 sp=0xffffc70b89a0 pc=0xaaaacbbe2538
runtime.rt0_go()
	src/runtime/asm_arm64.s:86 +0xa4 fp=0xffffc70b8a50 sp=0xffffc70b8a20 pc=0xaaaacbc0ddd4

[darkrift]

To add to the issue, I've also had to workaround this using toolchains_musl and a combination of the pure setting.

[DavidZbarsky-at]

I thought this might be fixed by https://github.com/bazelbuild/rules_go/issues/3691 but no luck. I did notice that passing --linkopt=-no-pie fixes things. @siddharthab @fmeum in case you have thoughts.

[siddharthab]

Thanks for the repro. This is actually a combination of two things happening.

1. Go versions before 1.21 did not work properly with external linkers (see #3692 or this comment). So your sysroot was not being used anyway.
2. Looks like when providing a sysroot to the link command in the Go toolchain, the path is being prefixed with __GO_BAZEL_CC_PLACEHOLDER__, which makes the sysroot non-existent and the check for whether the linker supports -nopie fails, thereby keeping the default pie behavior in clang frontend (reason for #3691). The unwanted prefix is coming from a bug in rules_go, which is setting this placeholder when building the Go stdlib. When you build your own binary, the Go toolchain picks up CGo directives from other objects being linked in, in this case, the stdlib objects like net.a and cgo.a.

[siddharthab]

If the placeholder mechanism needs to stay, then I think the only solution is for the linker to also have a wrapper like cc.go.

I am not current in the codebase and I won't have the time to implement a proper solution here, so would appreciate if someone else takes over.

[siddharthab]

Also filed https://github.com/golang/go/issues/69350 to eventually fix the issue upstream.

[fmeum]

Cc @AlessandroPatti for the wrapper

[siddharthab]

Issue has been fixed upstream. Go 1.24 should not have this issue.

[ashi009]

um, I'm seeing the same issue with:

• go 1.24.2
• rules_go 0.54.0
• toolchains_llvm 1.2.0

  lvm_toolchain(
     name = "llvm_toolchain",
     # TODO(xiaoyishi): Remove this once we no longer build on amazon linux.
     # amazonlinux's clang/gcc recognize it self as aarch64-amazon-linux, but
     # tools not from their distro recognize it as aarch64-unknown-linux-gnu.
     # which causes clang failed to find gcc installation.
     gcc_install_dir = {
         "linux-aarch64": "/usr/lib/gcc/aarch64-amazon-linux/11",
         "linux-x86_64": "/usr/lib/gcc/x86_64-amazon-linux/11",
     },
     llvm_version = "17.0.6",
     stdlib = {
         "linux-aarch64": "dynamic-stdc++",
         "linux-x86_64": "dynamic-stdc++",
     },
  
  

runtime: pcHeader: magic= 0xfffffff1 pad1= 0 pad2= 0 minLC= 4 ptrSize= 8 pcHeader.textStart= 0x96c130 text= 0xaaaad7bec130 pluginpath= 
fatal error: invalid function symbol table
runtime: panic before malloc heap initialized

runtime stack:
runtime.throw({0xaaaad74db3c5?, 0x0?})
	GOROOT/src/runtime/panic.go:1101 +0x38 fp=0xffffee685370 sp=0xffffee685340 pc=0xaaaad7c5eea8
runtime.moduledataverify1(0xaaaad8537b20?)
	GOROOT/src/runtime/symtab.go:623 +0x644 fp=0xffffee685490 sp=0xffffee685370 pc=0xaaaad7c62284
runtime.moduledataverify(...)
	GOROOT/src/runtime/symtab.go:599
runtime.schedinit()
	GOROOT/src/runtime/proc.go:834 +0x88 fp=0xffffee685520 sp=0xffffee685490 pc=0xaaaad7c2cad8
runtime.rt0_go()
	src/runtime/asm_arm64.s:86 +0xa4 fp=0xffffee685550 sp=0xffffee685520 pc=0xaaaad7c64344

and adding pure = "on" as a temporary fix.