grpc-gateway icon indicating copy to clipboard operation
grpc-gateway copied to clipboard

protoc-gen-swagger: support maps in query parameters

Open george-oakling opened this issue 6 years ago • 10 comments

Hi guys, having following definition:

message Test {
    map<string, string> filters = 1;
}

service TestService {
    rpc TestRpc(Test) returns (Test) {
        option (google.api.http) = {
            get: "/test"
        };
    }
}

Which goes through swagger generator and ends up without the map being mentioend as parameters in request:

"/test": {
      "get": {
        "operationId": "TestRpc",
        "responses": {
          "200": {
            "description": "A successful response.",
            "schema": {
              "$ref": "#/definitions/offersTest"
            }
          }
        },
        "tags": [
          "TestService"
        ]
      }
    }

I thought that I can add map params to the query... but I dont think so now and I cant tell if it's a bug... KR Jirka

george-oakling avatar Aug 06 '19 11:08 george-oakling

Thanks for the bug report! A few questions:

  1. What does the generated gateway look like? Is it trying to parse the map from the query? If so, that would be a large indicator that we should be supporting this and that it's a bug in the swagger generator.
  2. Does it work even if you disregard the generated swagger docs?

johanbrandhorst avatar Aug 06 '19 12:08 johanbrandhorst

test.proto:

message TestRequest {
    map<string, string> filters = 1;
}

message TestResponse {
}

service TestSvc {
    rpc TestRpc(TestRequest) returns(TestResponse) {
        option (google.api.http) = {
            get: "/offers"
        };
    }
}

test.pb.gw.go

var (
	filter_TestSvc_TestRpc_0 = &utilities.DoubleArray{Encoding: map[string]int{}, Base: []int(nil), Check: []int(nil)}
)

func request_TestSvc_TestRpc_0(ctx context.Context, marshaler runtime.Marshaler, client TestSvcClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
	var protoReq TestRequest
	var metadata runtime.ServerMetadata

	if err := req.ParseForm(); err != nil {
		return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
	}
	if err := runtime.PopulateQueryParameters(&protoReq, req.Form, filter_TestSvc_TestRpc_0); err != nil {
		return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
	}

	msg, err := client.TestRpc(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
	return msg, metadata, err

}
...
func RegisterTestSvcHandlerClient(ctx context.Context, mux *runtime.ServeMux, client TestSvcClient) error {

	mux.Handle("GET", pattern_TestSvc_TestRpc_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
		ctx, cancel := context.WithCancel(req.Context())
		defer cancel()
		inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req)
		rctx, err := runtime.AnnotateContext(ctx, mux, req)
		if err != nil {
			runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
			return
		}
		resp, md, err := request_TestSvc_TestRpc_0(rctx, inboundMarshaler, client, req, pathParams)
		ctx = runtime.NewServerMetadataContext(ctx, md)
		if err != nil {
			runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
			return
		}

		forward_TestSvc_TestRpc_0(ctx, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)

	})

	return nil
}

test.swagger

      "get": {
        "operationId": "TestRpc",
        "responses": {
          "200": {
            "description": "A successful response.",
            "schema": {
              "$ref": "#/definitions/offersTestResponse"
            }
          }
        },
        "tags": [
          "TestSvc"
        ]
      }

george-oakling avatar Aug 06 '19 13:08 george-oakling

I can't tell if that is or isn't parsing the query for the map, could you test running it and confirm?

johanbrandhorst avatar Aug 06 '19 13:08 johanbrandhorst

Yep, working on it :)

george-oakling avatar Aug 06 '19 13:08 george-oakling

Request to: /test?filters[jirka]=test&filters[wow]=test2

And there I have log.Println, which prints: filters:<key:"jirka" value:"test" > filters:<key:"wow" value:"test2" >

So yes, its getting data, but the protoc-swagger-gen ommit the definitions from request...

george-oakling avatar Aug 06 '19 13:08 george-oakling

Also the code in the GRPC server looks like this to make everything clear:

func(s *Server) TestRpc(ctx context.Context, req *offers.TestRequest) (*offers.TestResponse, error){
	log.Println(req)
	return &offers.TestResponse{}, nil
}

george-oakling avatar Aug 06 '19 13:08 george-oakling

Thanks, that narrows down the scope of this bug. Would you be interested in trying to submit a fix yourself? I would recommend starting to look at https://github.com/grpc-ecosystem/grpc-gateway/blob/master/protoc-gen-swagger/genswagger/template.go#L95 and maybe add some log messages and rebuild the generator. It would be very welcome!

johanbrandhorst avatar Aug 06 '19 13:08 johanbrandhorst

I'll try, but not before next week... I "hacked" this by using POST request with full body request which works perfectly, but definitely will look at the source asap.

george-oakling avatar Aug 06 '19 13:08 george-oakling

Hi, I can take a look at this bug since we have a requirement to use this kind of pattern. Thanks

sum2000 avatar Sep 10 '19 16:09 sum2000

Note that it is not possible to have objects or maps in query parameter according to OpenAPI-v2 spec (which is what is available at the time of writing, protoc-gen-openapiv2):

Since the parameter is not located at the request body, it is limited to simple types (that is, not an object). The value MUST be one of "string", "number", "integer", "boolean", "array" or "file".

This issue can probably be merged into https://github.com/grpc-ecosystem/grpc-gateway/issues/441 which allows object type query parameters.

s4nji avatar Jan 05 '21 16:01 s4nji