How can I pass in `protoc-gen-openapiv2()` args like 'single_output=' to `gateway_openapiv2_compile()`?

[mmellin]

Description

I'm currently using rules-proto-grpc in a Bazel environment. My understanding is that rules_proto_grpc has rules which utilize com_github_grpc_ecosystem_grpc_gateway and wrap rules like protoc_gen_swagger() (v1) and protoc_gen_openapiv2() (v2) functionality within gateway_openapiv2_compile().

Question:

I would like to know if rules like gateway_openapiv2_compile offer a way to pass args such as json_names_for_fields and single_output through?

This would be to emulate the following functionality of merging all *.swagger.json files into one output file:

load("@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-openapiv2:defs.bzl", "protoc_gen_openapiv2")

protoc_gen_openapiv2(
    name = "device_proto_gen_swagger_merged",
    proto = ":device_proto",
    json_names_for_fields = True,
    single_output = True,  # Outputs a single swagger.json file.
)

I have tried the following with no success:

load("@rules_proto_grpc//grpc-gateway:defs.bzl", "gateway_openapiv2_compile")

gateway_openapiv2_compile(
    name = "device_proto_gen_swagger_merged",
    protos = [":device_proto"],
    options = {
        "@rules_proto_grpc//grpc-gateway:openapiv2_plugin": [
            "json_names_for_fields = True",
            "single_output = True",
        ]
    },
)

Environment:

Bazel v4.2.1 rules-proto-grpc v4.0.1 com_github_grpc_ecosystem_grpc_gateway v2.x.x

[aaliddell]

We don't wrap the rules in the com_github_grpc_ecosystem_grpc_gateway repo, but rather run the protoc plugin ourselves directly. I would expect you will have issues with single_output, since it changes the plugin output structure. Howevr, I would have thought your example should work for json_names_for_fields. I'll have to test.

[ashwin153]

Would it be possible to make this single_output? Much of the OpenAPI ecosystem use single files (e.g., https://github.com/OpenAPITools/openapi-generator-bazel).

[tivvit]

Hi, I am not sure, but it may be caused by this https://github.com/rules-proto-grpc/rules_proto_grpc/blob/21efd5d352ce8b1f6ccae101986fce7492f13c3f/internal/protoc.bzl#L99

The proto plugins definition is missing separate_options_flag = True, in https://github.com/rules-proto-grpc/rules_proto_grpc/blob/21efd5d352ce8b1f6ccae101986fce7492f13c3f/grpc-gateway/BUILD.bazel

which causes output like this --openapiv2_plugin_out=grpc_api_configuration=proto/service.yml:bazel-out/k8-fastbuild/bin/api/_rpg_premerge_openapi-spec

I am right I can send a fix.

Fortunately, extra_protoc_args = ["--openapiv2_plugin_opt=grpc_api_configuration=proto/service.yml"] may be used to get the same result.

[aaliddell]

Interesting, both methods of passing opts to plugins are supposed to be equivalent, so the fact it breaks without separate_options_flag suggests the plugin is doing something odd. I'll take the PR then if that makes it work.

[waqas-anonymco]

@tivvit @mmellin

What's the correct option to use to generate single output file after this fix?

I have the latest 5.4.0 release but all of the following examples don't work:

load("@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-openapiv2:defs.bzl", "protoc_gen_openapiv2")

load("@rules_proto_grpc//grpc-gateway:defs.bzl", "gateway_openapiv2_compile")

gateway_openapiv2_compile(
    name = "gen_swagger_merged",
    protos = [":a_proto", ":b_proto"],
    options = {
        "@rules_proto_grpc//grpc-gateway:openapiv2_plugin": [
            "allow_merge=true",
            "merge_file_name=merged_swagger.json",
        ]
    },
    single_output = True,
)

Fails with error: no such attribute 'single_output' in 'gateway_openapiv2_compile' rule

load("@rules_proto_grpc//grpc-gateway:defs.bzl", "gateway_openapiv2_compile")

gateway_openapiv2_compile(
    name = "gen_swagger_merged",
    protos = [":a_proto", ":b_proto"],
    options = {
        "@rules_proto_grpc//grpc-gateway:openapiv2_plugin": [
            "allow_merge=true",
            "merge_file_name=merged_swagger.json",
            "single_output = True",
        ]
    },
)

Fails with error: F1107 12:14:00.990696 17969 main.go:83] Error parsing flags: cannot set flag single_output = True: no such flag -single_output

load("@rules_proto_grpc//grpc-gateway:defs.bzl", "gateway_openapiv2_compile")

gateway_openapiv2_compile(
    name = "gen_swagger_merged",
    protos = [":a_proto", ":b_proto"],
    options = {
        "@rules_proto_grpc//grpc-gateway:openapiv2_plugin": [
            "allow_merge=true",
            "merge_file_name=merged_swagger.json",
        ]
    },
)

Fails with error: Compiling protoc outputs for openapiv2_plugin plugin on target @...:gen_swagger_merged failed: not all outputs were created or valid

[mmellin]

From what I have working currently:

In my apps BUILD.bazel file (I do not Gazelle ignore it):

load("@rules_proto//proto:defs.bzl", "proto_library")
load("@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-openapiv2:defs.bzl", "protoc_gen_openapiv2")

proto_library(
    name = "iam_proto",
    srcs = ["authentication.proto"],
    deps = [
        "//base/go/auth/proto:iam_proto",
        "@go_googleapis//google/api:annotations_proto",
    ],
) // added by Gazelle

...(truncated)...

protoc_gen_openapiv2(
    name = "authentication_proto_gen_swagger_merged",
    json_names_for_fields = True,
    proto = ":iam_proto",
    single_output = True,  # Outputs a single swagger.json file.
)

In my top level BUILD.bazel:

# gazelle:go_grpc_compilers @io_bazel_rules_go//proto:go_grpc,@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-grpc-gateway:go_gen_grpc_gateway

My version of ecosystem v2:

go_repository(
        name = "com_github_grpc_ecosystem_grpc_gateway_v2",
        build_file_proto_mode = "disable",
        importpath = "github.com/grpc-ecosystem/grpc-gateway/v2",
        sum = "h1:ESEyqQqXXFIcImj/BE8oKEX37Zsuceb2cZI+EL/zNCY=",
        version = "v2.10.0",
    )

My rules_proto_grpc in WORKSPACE is at 2.0.0. Rules_go -> v0.39.1 Gazelle -> v0.30.0

[waqas-anonymco]

Thanks @mmellin.

You are using protoc_gen_openapiv2. protoc_gen_openapiv2 only takes one proto value, but gateway_openapiv2_compile takes an array of protos. I cannot use protoc_gen_openapiv2 to generate single swagger file for multiple proto files.

It looks like we still cannot use single_output in gateway_openapiv2_compile rule. I am not sure how the change form @tivvit fixed it. Probably I am missing something.

[creste]

I found a workaround for generating a single Swagger JSON file when calling gateway_openapiv2_compile.

Steps:

1. Patch rules_proto_grpc to add a new openapiv2_plugin_single_output proto plugin that removes outputs = ["{protopath}.swagger.json" and sets output_directory = True:

diff --git a/grpc-gateway/BUILD.bazel b/grpc-gateway/BUILD.bazel
index 3e951201..ff83583c 100644
--- a/grpc-gateway/BUILD.bazel
+++ b/grpc-gateway/BUILD.bazel
@@ -21,6 +21,22 @@ proto_plugin(
         "google/protobuf",
     ],
     outputs = ["{protopath}.swagger.json"],
+    quirks = [
+        "QUIRK_DIRECT_MODE",
+        "QUIRK_OUT_SINGLE_FILE",
+    ],
+    separate_options_flag = True,
+    tool = "@com_github_grpc_ecosystem_grpc_gateway_v2//protoc-gen-openapiv2",
+    visibility = ["//visibility:public"],
+)
+
+proto_plugin(
+    name = "openapiv2_plugin_single_output",
+    exclusions = [
+        "google/api",
+        "google/protobuf",
+    ],
+    output_directory = True,
     quirks = [
         "QUIRK_DIRECT_MODE",
     ],
diff --git a/grpc-gateway/gateway_openapiv2_compile.bzl b/grpc-gateway/gateway_openapiv2_compile.bzl
index abf96216..970c0b01 100644
--- a/grpc-gateway/gateway_openapiv2_compile.bzl
+++ b/grpc-gateway/gateway_openapiv2_compile.bzl
@@ -16,6 +16,7 @@ gateway_openapiv2_compile = rule(
             providers = [ProtoPluginInfo],
             default = [
                 Label("//grpc-gateway:openapiv2_plugin"),
+                Label("//grpc-gateway:openapiv2_plugin_single_output"),
             ],
             doc = "List of protoc plugins to apply",
         ),

2. Use the new openapiv2_plugin_single_output proto plugin when using gateway_openapiv2_compile:

gateway_openapiv2_compile(
    name = "myapi",
    options = {"@rules_proto_grpc//grpc-gateway:openapiv2_plugin_single_output": [
        "allow_merge=true",
        "merge_file_name=Swagger.json",
    ]},
    protos = [
        ":service_a_proto",
        ":service_b_proto",
    ],
)

The patch was created against rules_proto_grpc version 4.5.0.

There's probably a better way to do this, but I'm not familiar enough with Bazel to know how to fix it a better way.