bazelbuild
--action_env not respected?
I'm trying to build the latest envoy, and it seems that some protoc calls do not have the correct environment.
What version of rules_go are you using?
0.17.4
What version of Bazel are you using?
0.25.1
Does this issue reproduce with the latest releases of all the above?
Other things break if I bump to 0.18.4.
What operating system and processor architecture are you using?
Linux x86_64
Any other potentially useful information about your toolchain?
CC/CXX/LD_LIBRARY_PATH are set to a non-system gcc 9.1.0. The system gcc is 4.8.x.
What did you do?
Effectively this:
$ export CC=/path/to/gcc
$ export CXX=/path/to/g++
$ export LD_LIBRARY_PATH=/path/to/gcc/libs
$ git clone https://github.com/envoyproxy/envoy.git
$ cd envoy
$ git checkout v1.10.0 # same results on master too
$ bazel build --action_env=LD_LIBRARY_PATH -s //source/exe:envoy-static
What did you expect to see?
Successful build
What did you see instead?
INFO: Analysed target //source/exe:envoy-static (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
SUBCOMMAND: # @io_bazel_rules_go//proto/wkt:any_go_proto [action 'Generating into bazel-out/host/bin/external/io_bazel_rules_go/proto/wkt/linux_amd64_stripped/any_go_proto%/github.com/golang/protobuf/ptypes/any [for host]']
(cd /data/.cache/bazel/_bazel_builder/e584d00930a6e29692a9a5320943ac6a/execroot/envoy && \
exec env - \
bazel-out/host/bin/external/io_bazel_rules_go/go/tools/builders/linux_amd64_stripped/go-protoc -protoc bazel-out/host/bin/external/com_google_protobuf/protoc -importpath github.com/golang/protobuf/ptypes/any -out_path bazel-out/host/bin/external/io_bazel_rules_go/proto/wkt/linux_amd64_stripped/any_go_proto%/ -plugin bazel-out/host/bin/external/com_github_golang_protobuf/protoc-gen-go/linux_amd64_stripped/protoc-gen-go -compiler_path /data/.cache/bazel/_bazel_builder/e584d0
0930a6e29692a9a5320943ac6a/external/local_config_cc/extra_tools -descriptor_set bazel-out/host/genfiles/external/com_google_protobuf/any_proto-descriptor-set.proto.bin -expected bazel-out/host/bin/external/io_bazel_rules_go/proto/wkt/linux_amd64_stripped/any_go_proto%/github.com/golang/protobuf/ptypes/any/any.pb.go -import 'google/protobuf/any.proto=github.com/golang/protobuf/ptypes/any' google/protobuf/any.proto)
ERROR: /data/.cache/bazel/_bazel_builder/e584d00930a6e29692a9a5320943ac6a/external/io_bazel_rules_go/proto/wkt/BUILD.bazel:3:1: Generating into bazel-out/host/bin/external/io_bazel_rules_go/proto/wkt/linux_amd64_stripped/any_go_proto%/github.com/golang/protobuf/ptypes/any failed (Exit 1) go-protoc failed: error executing command bazel-out/host/bin/external/io_bazel_rules_go/go/tools/builders/linux_amd64_stripped/go-protoc -protoc bazel-out/host/bin/external/com_google_protob
uf/protoc -importpath ... (remaining 14 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
bazel-out/host/bin/external/com_google_protobuf/protoc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/host/bin/external/com_google_protobuf/protoc)
bazel-out/host/bin/external/com_google_protobuf/protoc: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/com_google_protobuf/protoc)
bazel-out/host/bin/external/com_google_protobuf/protoc: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/host/bin/external/com_google_protobuf/protoc)
2019/05/10 18:02:42 error running protoc: exit status 1
Target //source/exe:envoy-static failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.262s, Critical Path: 0.01s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
I'm somewhat of a bazel novice so not 100% sure why the LD_LIBRARY_PATH is not being passed through to this go-protoc step via --action_env. The other envoy build steps appear to have LD_LIBRARY_PATH set correctly. I've seen some mention of a use_default_shell_env flag, maybe we need to set that somewhere?
It appears that Bazel does not pass --action_env variables through to actions by default. Actions have to opt into that by setting use_default_shell_env = True when calling ctx.actions.run. When that flag is set, only the environment variables set with --action_env will be added; other stuff in the invocation environment is still ignored.
From the design doc, it seems like the intent was to make use_default_shell_env = True the default, but that hasn't happened.
I think it's probably safe to switch rules_go actions to use this, given the limited scope.
However, please do consider configuring your toolchain with CROSSTOOL instead of environment variables. That should be the main way to do this.
Cool, sounds good with adding use_default_shell_env = True.
The envoy folks have their own compiler wrappers and stuff that appear to be non-standard (i.e. not CROSSTOOL), so it wasn't obvious to me how to modify their setup to affect dependencies' environments or linkopts. That might be something to improve over there, but nonetheless it would be helpful to add the action-env passthrough here.
It looks like when use_default_shell_env = True is set, Bazel silently ignores env, and we need env for every action. We set GOROOT, GOARCH, GOOS, and a bunch of other things in there.
From bazelbuild/bazel#5980, it doesn't seem like this is intended behavior. I'll leave this issue open, but that needs to be resolved first.
It looks like when
use_default_shell_env = Trueis set, Bazel silently ignoresenv, and we needenvfor every action. We setGOROOT,GOARCH,GOOS, and a bunch of other things in there.From bazelbuild/bazel#5980, it doesn't seem like this is intended behavior. I'll leave this issue open, but that needs to be resolved first.
Just a note that https://github.com/bazelbuild/bazel/issues/5980 is now fixed, FWIW.
