Demonstrate usage with protocol buffers and grpc

[alexeagle]

Show https://rules-proto-grpc.com/en/latest/lang/js.html and/or https://github.com/stackb/rules_proto/tree/master/rules/nodejs examples in this repo.

[codersasha]

This would be great. I'm having some trouble getting this to work, where it did in the old rules_js repo. I defined a custom plugin:

tools/bazel/ts_proto_compile/BUILD.bazel:

load("@rules_proto_grpc//:defs.bzl", "proto_plugin")

proto_plugin(
    name = "ts_proto_compile",
    outputs = ["{protopath}.ts"],
    protoc_plugin_name = "ts_proto",
    tool = "@npm//ts-proto/bin:protoc-gen-ts_proto",
    visibility = ["//visibility:public"],
)

tools/bazel/ts_proto_compile/def.bzl:

load(
    "@rules_proto_grpc//:defs.bzl",
    "ProtoPluginInfo",
    "proto_compile_attrs",
    "proto_compile_impl",
)

ts_proto_compile = rule(
    implementation = proto_compile_impl,
    attrs = dict(
        proto_compile_attrs,
        _plugins = attr.label_list(
            providers = [ProtoPluginInfo],
            default = [
                Label("//tools/bazel/ts_proto_compile"),
            ],
            doc = "List of protoc plugins to apply",
        ),
    ),
    toolchains = [
        str(Label("@rules_proto_grpc//protobuf:toolchain_type")),
    ],
)

Then using it like:

load("//tools/bazel/ts_proto_compile:defs.bzl", "ts_proto_compile")

ts_proto_compile(
    name = "google_ts",
    protos = ["@com_google_protobuf//:any_proto"],
)

However, changing to the new aspect-build rules_js repo, and adding the new tool path:

   tool = "//:node_modules/ts-proto/protoc-gen-ts_proto",

gives the error /usr/bin/env: 'node': No such file or directory:

INFO: Invocation ID: 7955c9de-0498-43e7-868f-7b323d996ac2
INFO: Analyzed target //common/proto/google:google_lib (25 packages loaded, 186 targets configured).
INFO: Found 1 target...
ERROR: /home/sasha/paypa-stack/common/proto/google/BUILD.bazel:8:17: Compiling protoc outputs for ts_proto_compile plugin on target //common/proto/google:google_lib failed: (Exit 1): bash failed: error executing command /bin/bash -c 'mkdir -p '\''bazel-out/k8-fastbuild/bin/common/proto/google/_rpg_premerge_google_lib'\'' && external/com_google_protobuf_protoc_linux_x86_64/bin/protoc $@' '' ... (remaining 11 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
/usr/bin/env: 'node': No such file or directory
--ts_proto_out: protoc-gen-ts_proto: Plugin failed with status code 127.
Target //common/proto/google:google_lib failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.447s, Critical Path: 0.98s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

So, it seems node is no longer available to the toolchain, despite it being the same binary?

[codersasha]

I think I can see part of the issue - the file is actually a shell script to run node. Setting the tool to the new form of js_binary should fix it:

load("@npm//:ts-proto/package_json.bzl", "bin")

bin.protoc_gen_ts_proto_binary(
    name = "protoc-gen-ts_proto",
)

proto_plugin(
    ...
    tool = ":protoc-gen-ts_proto",
)

except, this gives an error about $(BINDIR):

FATAL: aspect_rules_js[js_binary]: BAZEL_BINDIR must be set in environment to the makevar $(BINDIR) in js_binary build actions (which run in the execroot) so that build actions can change directories to always run out of the root of the Bazel output tree. See https://docs.bazel.build/versions/main/be/make-variables.html#predefined_variables. This is automatically set by 'js_run_binary' (https://github.com/aspect-build/rules_js/blob/main/docs/js_run_binary.md) which is the recommended rule to use for using a js_binary as the tool of a build action. If this is not a build action you can set the BAZEL_BINDIR to '.' instead to supress this error. For more context on this design decision, please read the aspect_rules_js README https://github.com/aspect-build/rules_js/tree/dbb5af0d2a9a2bb50e4cf4a96dbc582b27567155#running-nodejs-programs.
--ts_proto_out: protoc-gen-ts_proto: Plugin failed with status code 1.

Adding BAZEL_BINDIR=$(BINDIR) to the command sets it to <derived root>, since @rules_proto_grpc does proto generation in the output directory.

[gonzojive]

Here's what I'm currently trying:

WORKSPACE.bazel:

git_repository(
    name = "rules_proto_grpc",
    commit = "b9e6b2922d8b6177d0747f30b738ea467161fc33",
    remote = "https://github.com/gonzojive/rules_proto_grpc.git",
)

bazel/web/ts_proto_library.bzl:

load(
    "@rules_proto_grpc//:defs.bzl",
    "ProtoPluginInfo",
    "proto_compile_attrs",
    "proto_compile_impl",
)

def _ts_proto_compile_impl(ctx):
    """
    Implementation function for ts_proto_compile.

    Args:
        ctx: The Bazel rule execution context object.

    Returns:
        Providers:
            - ProtoCompileInfo
            - DefaultInfo

    """
    base_env = {
        # Make up for https://github.com/bazelbuild/bazel/issues/15470.
        "BAZEL_BINDIR": ctx.bin_dir.path,
    }
    return proto_compile_impl(ctx, base_env = base_env)

# based on https://github.com/aspect-build/rules_js/issues/397
ts_proto_compile = rule(
    implementation = _ts_proto_compile_impl,
    attrs = dict(
        proto_compile_attrs,
        _plugins = attr.label_list(
            providers = [ProtoPluginInfo],
            default = [
                Label("//bazel/web/targets:ts_proto_compile"),
            ],
            doc = "List of protoc plugins to apply",
        ),
    ),
    toolchains = [
        str(Label("@rules_proto_grpc//protobuf:toolchain_type")),
    ],
)

bazel/web/targets/BUILD.bazel:

load("@npm//:ts-proto/package_json.bzl", _ts_proto_bin_factories = "bin")
load("@rules_proto_grpc//:defs.bzl", "proto_plugin")

_ts_proto_bin_factories.protoc_gen_ts_proto_binary(
    name = "protoc-gen-ts-proto",
)

proto_plugin(
    name = "ts_proto_compile",
    outputs = ["{protopath}.ts"],
    protoc_plugin_name = "ts_proto",
    tool = "//bazel/web/targets:protoc-gen-ts-proto",
    use_built_in_shell_environment = False,
    visibility = ["//visibility:public"],
)

Usage:

load("@rules_proto//proto:defs.bzl", "proto_library")
load("//bazel/web:ts_proto_library.bzl", "ts_proto_compile")

proto_library(
    name = "geompb_proto",
    srcs = ["geom.proto"],
    import_prefix = "blah.xyz",
    visibility = ["//visibility:public"],
    deps = [
        "@go_googleapis//google/type:latlng_proto",
    ],
)

ts_proto_compile(
    name = "geom_ts",
    protos = [
        ":geompb_proto",
    ],
    verbose = 4,
    visibility = ["//visibility:public"],
)

[sacsar]

I've been trying to get this to work with stackb/rules_proto, but I can't figure out how to refer to the plugin binary.

I have the following in the WORKSPACE file:

npm_translate_lock(
    name = "npm_ts_proto",
    # we have to manually wire-up the bin, unfortunately
    bins = {
        "ts-proto": {
            "protoc-gen-ts_proto": "./protoc-gen-ts_proto",
        },
    },
    pnpm_lock = "//plugin/stephenh/ts-proto:pnpm-lock.yaml",
    verify_node_modules_ignored = "//:.bazelignore",
)

load("@npm_ts_proto//:repositories.bzl", npm_ts_proto_repositories= "npm_repositories")

npm_ts_proto_repositories()

Using the old rules_js v5 (note that the main repo is still on v4), the plugin is defined as so:

proto_plugin(
    name = "protoc-gen-ts-proto",
    data = [
        "@nodejs_host//:npm",
        "@npm_ts_proto//:node_modules",
    ],
    tool = "@npm_ts_proto//ts-proto/bin:protoc-gen-ts_proto",
    visibility = ["//visibility:public"],
)

I haven't been able to figure out how to refer to the binary when using the new rules_js. Trying the same trick as worked for the other proto ruleset didn't work: cannot load '@npm_ts_proto//:ts-proto/package_json.bzl': no such file (Note: This is fundamentally the same question as being asked in the discussion of #630.)

[sacsar]

I realised I was forgetting how labels work and it should be ":node_modules/.bin/protoc-gen-ts_proto" once you link the packages. However, while that's a good label, I'm still failing to compile the protos: Compiling protoc outputs for ["routeguide.proto"] failed: missing input file '//plugin/stephenh/ts-proto:node_modules/.bin/protoc-gen-ts_proto'

[gonzojive]

Maybe you need something like the following?

load("@npm//:ts-proto/package_json.bzl", _ts_proto_bin_factories = "bin")

_ts_proto_bin_factories.protoc_gen_ts_proto_binary(
    name = "protoc-gen-ts-proto",
)

# label :protoc-gen-ts-proto is now the "tool" binary you can execute

[sacsar]

That did end up being the trick--I'd messed up setting up the node modules, clearing out everything and starting over got it working. Now I'm stuck on the proto_ts_library macro. (I'm taking that question to the Slack, though, as it's clearly beyond this issue. I'll return with a link to a PR or something if I get this working.)

[alexeagle]

FYI it seems likely that we have some funding lined up to create a proper ts_proto_library rule this quarter.

[sacsar]

Ooh, that is good news. The attraction of stackb/rules_proto for me are the gazelle rules, but it's proving to be an uphill battle. But then I'm left figuring out how to publish less granular packages, so I may end up globbing for protos anyway.

[zfy0701]

proto_plugin(
    name = "protoc-gen-ts-proto",
    data = [
        "@nodejs_host//:npm",
        "@npm_ts_proto//:node_modules",
    ],
    tool = "@npm_ts_proto//ts-proto/bin:protoc-gen-ts_proto",
    visibility = ["//visibility:public"],
)

I haven't been able to figure out how to refer to the binary when using the new rules_js. Trying the same trick as worked for the other proto ruleset didn't work: cannot load '@npm_ts_proto//:ts-proto/package_json.bzl': no such file (Note: This is fundamentally the same question as being asked in the discussion of #630.)

Maybe you need something like the following?

load("@npm//:ts-proto/package_json.bzl", _ts_proto_bin_factories = "bin")

_ts_proto_bin_factories.protoc_gen_ts_proto_binary(
    name = "protoc-gen-ts-proto",
)

# label :protoc-gen-ts-proto is now the "tool" binary you can execute

seems we also need in proto_plugin rule for this

    env = {
        "BAZEL_BINDIR": "{bindir}",
    },

[alexeagle]

I think we want to show an example with protobuf-es: https://github.com/bufbuild/protobuf-es

[alexeagle]

Okay finally getting some progress... https://github.com/aspect-build/rules_ts/issues/415

[alexeagle]

ts_proto_library is introduced in rules_ts v2.0.0-beta1

[alexeagle]

Marking this closed since rules_ts shows what to do.