Can not use genrule with go:embed

[wichert]

What version of rules_go are you using?

v0.29.0

What version of gazelle are you using?

v0.24.0

What version of Bazel are you using?

Build label: 4.2.2-homebrew
Build target: bazel-out/darwin_arm64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800

Does this issue reproduce with the latest releases of all the above?

Yes, I am using the latest releases.

What operating system and processor architecture are you using?

macOS 12.1 Monterey / M1 Max

Any other potentially useful information about your toolchain?

Go version 1.17

What did you do?

I am trying to embed files that are generated. A minimal case to reproduce it is this BUILD.bazel file:

genrule(
    name = "hellodata",
    srcs = ["hello.go"],
    outs = ["hello.txt"],
    cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $@",
)

go_library(
    name = "hello",
    srcs = ["hello.go"],
    importpath = "wiggy.net/hello",
    visibility = ["//visibility:public"],
    embedsrcs = [":hellodata"],
)

with hello.go like this:

package hello

import (
    _ "embed"
    "io"
)

//go:embed hello.txt
var greeting []byte

func Hello(out io.Writer) error {
    _, err := out.Write(greeting)
    return err
}

What did you expect to see?

I expect to be able to generate hello.txt, and embed that in my Go source.

What did you see instead?

A build error:

❯ bazel build //...                              
INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured).
INFO: Found 5 targets...
ERROR: /Users/wichert/Hack/bzl/BUILD.bazel:14:11: GoCompilePkg hello.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_arm64 -src hello.go -embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt -importpath wiggy.net/hello ... (remaining 12 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: /private/var/tmp/_bazel_wichert/e7573342ee9452df4c3dfa671d399a16/sandbox/darwin-sandbox/76/execroot/__main__/hello.go:8:12: could not embed hello.txt: no matching files found
INFO: Elapsed time: 0,112s, Critical Path: 0,04s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

This happens because the output file is created in bazel-out/darwin_arm64-fastbuild/bin, which I can not reference in the go:embed directive (without making that very system-specific and unportable).

[jacobbogdanov]

I think you want embedsrcs = ["hello.txt"]

[Kruemelmann]

hey @wichert did you find a solution to your problem? I am facing a, how i think, simular issue 3119. Mine is kind of building on top of yours. I am trying to embed files that are generated by a nodejs bazel rule.

[wichert]

I think I did, but I'm afraid I don't remember how exactly. We ended up not using bazel for that project because the learning curve was too steep for us.

[wichert]

I suspect the trick was to use go_embed_data, and include the result as a dep.

[Kruemelmann]

Thank you so much i will try this :)

[achew22]

We do have a passing test that demonstrates exactly this.

It embeds a genrule into a go_test (which under the covers behaves exactly like a go_binary or go_library for these purposes).

https://github.com/bazelbuild/rules_go/blob/3e49c5ec6c467b30e10ea7172ea3bfa9ef0e6efa/tests/core/go_library/BUILD.bazel#L114-L119

and then embeds it in the test

https://github.com/bazelbuild/rules_go/blob/3e49c5ec6c467b30e10ea7172ea3bfa9ef0e6efa/tests/core/go_library/embedsrcs_test.go#L25-L38

and makes assertions on it throughout the rest of the file. I don't believe this is a standing issue in the codebase. Can you show me what I'm missing here?

[wichert]

As I see it today the functionality is there, but some more tutorial-style documentation would be incredibly helpful.