Is there a way to specify env vars for the go toolchain?

[alexbozhenko]

What version of rules_go are you using?

v0.44.0

What version of gazelle are you using?

v0.35.0

What version of Bazel are you using?

6.4.0

Does this issue reproduce with the latest releases of all the above?

Yes.

What operating system and processor architecture are you using?

Ubuntu18, x86_64

What did you do?

 bazel run --action_env=GONOSUMDB=example.com  @io_bazel_rules_go//go -- env |& grep GONOSUMDB
GONOSUMDB=''

What did you expect to see?

I expected the env vars to be used by the go toolchain.

What did you see instead?

Env vars are ignored.

https://github.com/bazelbuild/rules_go/pull/3429 made the go toolchain available. When private go proxy is used, it requires setting those env variables: https://go.dev/ref/mod#private-module-proxy-private

I would like to be able to run go commands, e.g.

bazel run @io_bazel_rules_go//go -- mod tidy

(or go get) , without explicitly configuring the env vars on the CLI. So I was hoping to check-in those vars to .bazelrc(or some other place, really), so it is transparent for all the developers.

To be clear, this works:

$ GONOSUMDB=example.com bazel run   @io_bazel_rules_go//go -- env |& grep GONOSUMDB
GONOSUMDB='example.com'

and I can put it in the tools/bazel wrapper: https://github.com/bazelbuild/bazelisk?tab=readme-ov-file#toolsbazel but I was hoping to make it work in more bazel-native way...

Also, note that I do have to specify GOPROXY and GONOSUMDB in WORKSPACE, like this, as documented at https://github.com/bazelbuild/bazel-gazelle#user-content-running-gazelle-with-bazel:

gazelle_dependencies(
	go_env = {
		"GOPROXY": "https://myproxy.example.com|https://proxy.golang.org,direct",
		"GONOSUMDB": "example.com",
	},
)

, to make sure bazel run //my_go_program still will be able to fetch the go repo. If those are not specified, repo fetch will fail here

So maybe the more general problem is to allow to configure go-specific env vars(go help environment) for the go toolchain configured by Bazel?

Also, see discussion where community asked for a way to avoid making every dev configure the env vars: https://github.com/golang/go/issues/33985 And Russ Cox's position: https://github.com/golang/go/issues/42343#issuecomment-737406453 https://github.com/golang/go/issues/42343#issuecomment-746779560 When he said:

that root problem is not something the go command aims to solve. Shell scripts and more complex build systems are possible solutions.

I am sure he meant Bazel :)

Related issue in gazelle: https://github.com/bazelbuild/bazel-gazelle/issues/579

[alexbozhenko]

Another thought after looking at https://github.com/golang/go/commit/7aa85e01376d840acc8bb931156d607a00b64a60

Should go_register_toolchains just allow to specify go.env file? https://github.com/bazelbuild/rules_go/blob/master/go/toolchains.rst#go-register-toolchains Maybe https://github.com/bazelbuild/rules_go/pull/3401 should be further generalized to all go env vars?

[fmeum]

Should go_register_toolchains just allow to specify go.env file?

This was actually my first thought as well, but I now see undesirable side effects: The env variables one typically wants to set affect the fetches of dependencies, but not the compilation of .go code itself. If we add these variables to the go.env of the Go SDK, they will be an input to compilation actions and thus affect cache hits.

Instead, we can forward the variables to only the two places related to dependency fetching: @rules_go//go and go_repository.

[alexbozhenko]

@fmeum: https://pkg.go.dev/cmd/go#hdr-Environment_variables lists all possible variables, and it seems like is not just about fetches of dependencies. With standard go toolchain, and go.env file, it is easy to set all the env vars in one place.

But with Bazel some of them can be set by these settings: https://github.com/bazelbuild/rules_go/blob/master/go/modes.rst#build-settings

Some are chosen for you by rules_go for you: https://github.com/bazelbuild/rules_go/blame/55ea579a70958d66f24a907433978b21edd36b29/go/private/sdk.bzl#L491

So maybe allowing to specify all env vars in a single place will ease the mental load of users?

[linzhp]

but I was hoping to make it work in more bazel-native way... With standard go toolchain, and go.env file, it is easy to set all the env vars in one place.

I found these two requirements contradicting. Bazel doesn't use the go command from Go toolchain to compile and link. So even with what @fmeum did in #3879, the envs only affects the go command, not the compilation and linking.

In fact, I am worried that go_sdk.config(go_env = {...}) gives users a false impression that they can control compilation and linking by setting things likeCGO_CFLAGS or GOOS there.

So I think the "bazel-native way" is to set the envs in different ways than Go:

• GOPROXY can potentially be set with gazelle_dependencies(go_env = ...) like @fmeum in #3879, but we found it working well to set it in Bazel wrapper (e.g. tools/bazel)
• GOOS is set with --platforms
• CXX, CC should be the C toolchain resolved by Bazel
• GOFLAGS should be set from --@io_bazel_rules_go//go/config:gc_goopts
• This goes on and on...

What I am trying to say is Bazel and Go are two fundamentally different build systems. The behavior of Go build is controlled by env vars, but Bazel prefers configuration files and command line flags.

[fmeum]

@linzhp makes a very good point. I will change the name of the go_env attribute to something like go_fetch_env to make it clear that build-related env vars won't be read.

[fmeum]

I submitted https://github.com/bazelbuild/bazel-gazelle/pull/1748 instead, which moves the new go_env field to go_deps, hopefully making it more clear that this is only targetted at fetching of Go deps. What do you think?