rules_go icon indicating copy to clipboard operation
rules_go copied to clipboard
verified publisher icon bazelbuild

Fix cdeps propagation through embed chains

Open jsharpe opened this issue 3 weeks ago • 9 comments

When a go_library embeds another go_library that has cdeps but not cgo=True (because it itself embeds a library with cgo=True), the cdeps were not being propagated. This is a regression from v0.51.0.

In v0.50.1, cdeps and related cgo options were merged unconditionally: source["cdeps"] = source["cdeps"] or s.cdeps

In v0.51.0 (PR #4030), the logic was changed to only merge these when s.cgo is True: if s.cgo: source["cdeps"] = s.cdeps

This breaks the pattern where:

  • Library A has cgo=True and C code
  • Library B embeds A, adds cdeps, but doesn't set cgo=True
  • Test embeds B

The cdeps from B don't propagate to the test because B doesn't have cgo=True, causing undefined symbol errors during linking.

This fix restores the v0.50.1 behavior: merge cgo-related attributes if the embedded library has them, regardless of the cgo flag.

jsharpe avatar Nov 28 '25 09:11 jsharpe

Yes, I have a simple reproducer for the issue so I can turn that into a test.

jsharpe avatar Nov 28 '25 11:11 jsharpe

Thanks! Hadn't realized this was a valid scenario, adding a test would be great

dzbarsky avatar Nov 28 '25 12:11 dzbarsky

@fmeum I've added the test

jsharpe avatar Nov 28 '25 13:11 jsharpe

Sorry will fix it in CI - I can't build locally due to macOS SDK versions needed to be installed so hadn't actually tested the compile!

jsharpe avatar Nov 28 '25 13:11 jsharpe

@fmeum any ideas how to fix the windows build? Seems to be failing to find the msys gcc.

jsharpe avatar Nov 28 '25 15:11 jsharpe

I don't understand why this fails - @jayconrod do you? We exclude some cgo tests on Windows, but similar ones are run and pass.

fmeum avatar Nov 28 '25 15:11 fmeum

Sorry, I didn't have much time today to dig into this. But I was able to reproduce the error on my Windows machine. It, er, doesn't look like we print errors from failing C compiler commands. Someone should fix that at some point 😬 But with a quick hack, here's the error output. Does this help?

ERROR: C:/users/jay/code/rules_go/tests/core/cgo/BUILD.bazel:565:11: GoCompilePkg tests/core/cgo/cdeps_embed_base.a failed: (Exit 1): builder.exe failed: error executing GoCompilePkg command (from target //tests/core/cgo:cdeps_embed_base) bazel-out\x64_windows-opt-exec-ST-7daeda47919b\bin\external\go_sdk\builder_reset\builder.exe compilepkg -sdk external/go_sdk -goroot bazel-out/x64_windows-fastbuild/bin/stdlib_ -installsuffix ... (remaining 39 arguments skipped)
compilepkg: WORK=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
external\go_sdk\pkg\tool\windows_amd64\cgo.exe -srcdir C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -objdir C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -trimpath C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go;C:\users\jay\_bazel_jay\ajtxh6lj\execroot=>.. -ldflags @C:\Users\jay\AppData\Local\Temp\cgo-ldflags-174498093.txt -importpath github.com/bazelbuild/rules_go/tests/core/cgo/cdeps_embed -- -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC cdeps_embed_base.go
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC -fdebug-prefix-map=C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go=. -fdebug-prefix-map=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed=. -mthreads -c C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_export.c -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x0.o
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC -fdebug-prefix-map=C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go=. -fdebug-prefix-map=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed=. -mthreads -c C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\cdeps_embed_base.cgo2.c -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC -fdebug-prefix-map=C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go=. -fdebug-prefix-map=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed=. -mthreads -c C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.c -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.o
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x0.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o -mthreads
compilepkg: error running subcommand c:/msys64/mingw64/bin/gcc c:/msys64/mingw64/bin/gcc -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x0.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o -mthreads: exit status 1
c:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o:cdeps_embed_base.cgo2.c:(.text+0x65): undefined reference to `native_greeting'
collect2.exe: error: ld returned 1 exit status
Target //tests/core/cgo:cdeps_embed_with_cdeps failed to build
Use --verbose_failures to see the command lines of failed build steps.

jayconrod avatar Nov 29 '25 00:11 jayconrod

Sorry, I didn't have much time today to dig into this. But I was able to reproduce the error on my Windows machine. It, er, doesn't look like we print errors from failing C compiler commands. Someone should fix that at some point 😬 But with a quick hack, here's the error output. Does this help?

ERROR: C:/users/jay/code/rules_go/tests/core/cgo/BUILD.bazel:565:11: GoCompilePkg tests/core/cgo/cdeps_embed_base.a failed: (Exit 1): builder.exe failed: error executing GoCompilePkg command (from target //tests/core/cgo:cdeps_embed_base) bazel-out\x64_windows-opt-exec-ST-7daeda47919b\bin\external\go_sdk\builder_reset\builder.exe compilepkg -sdk external/go_sdk -goroot bazel-out/x64_windows-fastbuild/bin/stdlib_ -installsuffix ... (remaining 39 arguments skipped)
compilepkg: WORK=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
external\go_sdk\pkg\tool\windows_amd64\cgo.exe -srcdir C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -objdir C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -trimpath C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go;C:\users\jay\_bazel_jay\ajtxh6lj\execroot=>.. -ldflags @C:\Users\jay\AppData\Local\Temp\cgo-ldflags-174498093.txt -importpath github.com/bazelbuild/rules_go/tests/core/cgo/cdeps_embed -- -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC cdeps_embed_base.go
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC -fdebug-prefix-map=C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go=. -fdebug-prefix-map=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed=. -mthreads -c C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_export.c -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x0.o
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC -fdebug-prefix-map=C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go=. -fdebug-prefix-map=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed=. -mthreads -c C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\cdeps_embed_base.cgo2.c -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -I tests/core/cgo -iquote . -iquote C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go\tests\core\cgo -iquote C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed -fPIC -fdebug-prefix-map=C:\users\jay\_bazel_jay\ajtxh6lj\execroot\io_bazel_rules_go=. -fdebug-prefix-map=C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed=. -mthreads -c C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.c -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.o
CC=c:/msys64/mingw64/bin/gcc \
CGO_ENABLED=1 \
GOARCH=amd64 \
GODEBUG=winsymlink=0 \
GOEXPERIMENT= \
GOOS=windows \
GOPATH= \
GOROOT=bazel-out/x64_windows-fastbuild/bin/stdlib_ \
GOROOT_FINAL=GOROOT \
GOTOOLCHAIN=local \
PATH=c:/msys64/mingw64/bin \
SYSTEMDRIVE=C: \
SYSTEMROOT=C:\WINDOWS \
TEMP=C:\Users\jay\AppData\Local\Temp \
TMP=C:\Users\jay\AppData\Local\Temp \
c:/msys64/mingw64/bin/gcc -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x0.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o -mthreads
compilepkg: error running subcommand c:/msys64/mingw64/bin/gcc c:/msys64/mingw64/bin/gcc -o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_cgo_main.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x0.o C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o -mthreads: exit status 1
c:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\jay\AppData\Local\Temp\rules_go_work-1620827554\cgo\github.com\bazelbuild\rules_go\tests\core\cgo\cdeps_embed\_x1.o:cdeps_embed_base.cgo2.c:(.text+0x65): undefined reference to `native_greeting'
collect2.exe: error: ld returned 1 exit status
Target //tests/core/cgo:cdeps_embed_with_cdeps failed to build
Use --verbose_failures to see the command lines of failed build steps.

So this is the error that this change fixes on linux / macOS but clearly its not enough for windows. I don't have a windows box to debug this on; what's the best approach forward for this?

jsharpe avatar Dec 10 '25 18:12 jsharpe

@jsharpe I would be totally fine with you adding target_compatible_with to skip the new targets on Windows.

fmeum avatar Dec 10 '25 19:12 fmeum