rules_go icon indicating copy to clipboard operation
rules_go copied to clipboard
verified publisher icon bazelbuild

Correct way to use the core gRPC libraries?

Open popovicu opened this issue 1 year ago • 4 comments

What version of rules_go are you using?

0.46.0

What version of gazelle are you using?

N/A, not using explicitly. Gazelle is an indirect dependency.

What version of Bazel are you using?

7.2.0 (via Bazelisk v1.18.0)

Does this issue reproduce with the latest releases of all the above?

Yes.

What operating system and processor architecture are you using?

Mac OS on M2. Also Linux/x86_64.

Any other potentially useful information about your toolchain?

Invoking Bazel via Bazelisk.

What did you do?

Please check out my BUILD file for building the Go gRPC library here. The BUILD file for my simple server is here and the alias for the core gRPC library is here.

What did you expect to see?

Instead of debugging what are the repos that got injected by Gazelle during the build flow, I hoped it would be more straightforward to use something like @grpc//:go_default_library.

What did you see instead?

The only way I figured out to get the Go library for the core gRPC functionality is via the @@gazelle~~go_deps~org_golang_google_grpc//:go_default_library label.

popovicu avatar Jun 24 '24 12:06 popovicu

We have moved away from "injecting" third-party dependencies into your build as that only leads to trouble down the road. You can add a dependency on grpc like you would outside Bazel and then use the go_deps extension to make it available under a nice label. See https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md#specifying-external-dependencies for the details and feel free to reopen if that doesn't resolve your issue.

fmeum avatar Jun 24 '24 12:06 fmeum

Right. I've tried this before, but I get issues with cyclical dependencies.

Please take a look at this pull request to see the changes I've made: https://github.com/popovicu/go-grpc-bazel-example/pull/1/commits/f215918d992b0811295da566ca628ed7349bc170

If I do bazel build //proto:foo_go_proto, this is the output I get:

ERROR: /private/var/tmp/_bazel_uros/670d6fc3c8a0fd52c40728110c12cfd3/external/gazelle~~go_deps~org_golang_google_protobuf/compiler/protogen/BUILD.bazel:3:11: in go_library rule @@gazelle~~go_deps~org_golang_google_protobuf//compiler/protogen:protogen: cycle in dependency graph:
    //proto:foo_go_proto (8e4ab22e89843aa348ba782be25eadb5bae40f2ae62dec3edd2cdaf809e8d7b1)
    //proto:foo_go_proto (096dcc84165363e69a851ebe8131b032f5448c94ddc4951775429dc78e79f898)
    @@rules_go~//proto:go_grpc_v2 (096dcc84165363e69a851ebe8131b032f5448c94ddc4951775429dc78e79f898)
    @@rules_go~//proto:go_grpc_v2_reset_plugin_ (6f6e3fee8212c902eb428d9d1af78e5a07aed64216bca6679c1cfc5d78f89b4e)
    @@gazelle~~go_deps~org_golang_google_grpc_cmd_protoc_gen_go_grpc//:protoc-gen-go-grpc (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
    @@gazelle~~go_deps~org_golang_google_grpc_cmd_protoc_gen_go_grpc//:protoc-gen-go-grpc_lib (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
.-> @@gazelle~~go_deps~org_golang_google_protobuf//compiler/protogen:protogen (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@gazelle~~go_deps~org_golang_google_protobuf//reflect/protodesc:protodesc (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@gazelle~~go_deps~org_golang_google_protobuf//types/gofeaturespb:gofeaturespb (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@gazelle~~go_deps~org_golang_google_protobuf//types/gofeaturespb:gofeaturespb_go_proto (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@rules_go~//proto:go_proto (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@rules_go~//proto:go_proto_reset_plugin_ (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@gazelle~~go_deps~org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
|   @@gazelle~~go_deps~org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go_lib (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
`-- @@gazelle~~go_deps~org_golang_google_protobuf//compiler/protogen:protogen (0247d78883ba174476539e1ef0d8f1b79d20e12e1df16389f8f4a830b863103a)
ERROR: Analysis of target '//proto:foo_go_proto' failed; build aborted
INFO: Elapsed time: 9.346s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

Am I doing something wrong here?

popovicu avatar Jun 24 '24 13:06 popovicu

@linzhp I remember having seen this before, do you know how it could be solved?

fmeum avatar Jun 24 '24 13:06 fmeum

@fmeum, can we please reopen the issue? I don't have the permission to do it. 😞

popovicu avatar Jun 24 '24 13:06 popovicu

@fmeum any new thoughts in the meantime before @linzhp responds?

popovicu avatar Jul 02 '24 03:07 popovicu

hmm, I don't remember seeing this issue before

linzhp avatar Jul 03 '24 04:07 linzhp

@fmeum Is there anyone else this can be assigned to?

popovicu avatar Jul 03 '24 22:07 popovicu

I have a reproduce over in https://github.com/sluongng/nogo-analyzer/pull/44

(18:12:05) ERROR: /private/var/tmp/_bazel_sluongng/cec20b3f4315ccc0a0e690289c7b751e/external/org_golang_google_protobuf/cmd/protoc-gen-go/BUILD.bazel:15:10: in go_binary rule @@org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go: cycle in dependency graph:
    //golangci-lint/cmd/list-analyzer:list-analyzer (a0f56786d549e6e0d142c4461f3d16feecda1324aaef437d40195bb3d17996a3)
    //golangci-lint/cmd/list-analyzer:list-analyzer (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    //golangci-lint/cmd/list-analyzer:list-analyzer_lib (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    //golangci-lint/util:util (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    //golangci-lint/constructor:constructor_lib (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_golangci_golangci_lint//pkg/golinters:golinters (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_yeya24_promlinter//:promlinter (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_prometheus_client_golang//prometheus/testutil/promlint:promlint (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_prometheus_common//expfmt:expfmt (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_matttproud_golang_protobuf_extensions//pbutil:pbutil (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_golang_protobuf//proto:proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@org_golang_google_protobuf//reflect/protodesc:protodesc (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@org_golang_google_protobuf//src/google/protobuf:gofeaturespb (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@org_golang_google_protobuf//src/google/protobuf:gofeaturespb_go_proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@io_bazel_rules_go//proto:go_proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@io_bazel_rules_go//proto:go_proto_reset_plugin_ (22c80bf8eea6aa46994d32e961ec2e4f902607d7984e90667cf4cf1197d31989)
.-> @@org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go_lib (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@org_golang_google_protobuf//compiler/protogen:protogen (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@org_golang_google_protobuf//reflect/protodesc:protodesc (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@org_golang_google_protobuf//src/google/protobuf:gofeaturespb (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@org_golang_google_protobuf//src/google/protobuf:gofeaturespb_go_proto (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@io_bazel_rules_go//proto:go_proto (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
|   @@io_bazel_rules_go//proto:go_proto_reset_plugin_ (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)
`-- @@org_golang_google_protobuf//cmd/protoc-gen-go:protoc-gen-go (d94cbaab5de39cd848f532c422db4447846533fa1322243b242e2f47ce589d53)

and

(18:12:07) ERROR: /private/var/tmp/_bazel_sluongng/cec20b3f4315ccc0a0e690289c7b751e/external/com_github_golang_protobuf/proto/BUILD.bazel:3:11: in go_library rule @@com_github_golang_protobuf//proto:proto: cycle in dependency graph:
    //golangci-lint/cmd/list-analyzer:list-analyzer_lib (a0f56786d549e6e0d142c4461f3d16feecda1324aaef437d40195bb3d17996a3)
    //golangci-lint/cmd/list-analyzer:list-analyzer_lib (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    //golangci-lint/util:util (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    //golangci-lint/constructor:constructor_lib (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_golangci_golangci_lint//pkg/golinters:golinters (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_yeya24_promlinter//:promlinter (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
    @@com_github_prometheus_client_golang//prometheus:prometheus (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
.-> @@com_github_golang_protobuf//proto:proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
|   @@org_golang_google_protobuf//reflect/protodesc:protodesc (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
|   @@org_golang_google_protobuf//src/google/protobuf:gofeaturespb (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
|   @@org_golang_google_protobuf//src/google/protobuf:gofeaturespb_go_proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
|   @@io_bazel_rules_go//proto:go_proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
|   @@com_github_golang_protobuf//proto:go_default_library (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)
`-- @@com_github_golang_protobuf//proto:proto (b17e51af0f2071eac622e8b85f2aad9698f9695637507df74957d07ae54e74c3)

Both involves @rules_go//proto:go_proto

sluongng avatar Jul 08 '24 16:07 sluongng

Friendly ping. Should this be considered a bug and if yes, what would be the priority?

I personally think that this would draw a lot of people away from using Bazel + Go + gRPC. It requires somewhat advanced Bazel knowledge to work around, in my opinion.

popovicu avatar Jul 11 '24 17:07 popovicu

I suspect that this can be fixed by manually specifying the gazelle directives that are automatically added with Bzlmod: https://github.com/bazelbuild/bazel-gazelle/blob/master/internal/bzlmod/default_gazelle_overrides.bzl

Could you try that?

fmeum avatar Jul 11 '24 18:07 fmeum

Yeah adding "gazelle:proto disable" to relevant go_repository made it work.

sluongng avatar Jul 12 '24 18:07 sluongng

@popovicu Please let me know if this doesn't work for you and we can reopen.

fmeum avatar Jul 12 '24 22:07 fmeum

Just wanted to confirm that @fmeum provided the correct solution and @sluongng provided a good hint for how to make it work with WORKSPACE projects. For posterity, I'm using this in a module-based repo, so I added this to my MODULE.bazel and things worked:

go_deps.gazelle_default_attributes(
    # Pointers here: https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md#gazelle-directives
    directives = [
        "gazelle:proto disable",
    ],
)

I don't know what are all the consequences of doing this and my intuition is that if your dependencies are somehow building gRPC libraries, you may get some problems, but I guess that's a bit of an edge case at the moment, so I think we can consider this is settled. 👍

popovicu avatar Oct 15 '24 01:10 popovicu