gnostic icon indicating copy to clipboard operation
gnostic copied to clipboard

How should I specify ` application/x-www-form-urlencoded` as `Content-Type` in the post request?

Open yzhaoyu opened this issue 3 years ago • 9 comments

In the library.proto example:

service LibraryService {
  // Creates a shelf, and returns the new Shelf.
  rpc CreateShelf(CreateShelfRequest) returns (Shelf) {
    option (google.api.http) = {
      post: "/v1/shelves"
      body: "shelf"
    };
    option (google.api.method_signature) = "shelf";
  }
}

the default Content-Type is application/json. The generated openapi.yaml is:

post:
    tags:
        - LibraryService
    description: Creates a shelf, and returns the new Shelf.
    operationId: LibraryService_CreateShelf
    requestBody:
        content:
            application/json:
                schema:
                    $ref: '#/components/schemas/Shelf'
        required: true
    responses:
        "200":
            description: OK
            content:
                application/json:
                    schema:
                        $ref: '#/components/schemas/Shelf'
        default:
            description: Default error response
            content:
                application/json:
                    schema:
                        $ref: '#/components/schemas/Status'

If I want to use the application/x-www-form-urlencoded in the request and the text/plain in the response like this:

post:
    tags:
        - LibraryService
    description: Creates a shelf, and returns the new Shelf.
    operationId: LibraryService_CreateShelf
    requestBody:
        content:
            application/x-www-form-urlencoded:
                schema:
                    $ref: '#/components/schemas/Shelf'
        required: true
    responses:
        "200":
            description: OK
            content:
                text/plain:
                    schema:
                        $ref: '#/components/schemas/Shelf'
        default:
            description: Default error response
            content:
                application/json:
                    schema:
                        $ref: '#/components/schemas/Status'

What should I do in the proto file?

yzhaoyu avatar Apr 27 '22 04:04 yzhaoyu

Hi @yzhaoyu, That's a really good question. As far as I can remember, this is not currently possible. Maybe it's possible to build on @jeffsawatzky's recent work on extensions (#324 and #334) to get this functionality.

A simple solution, if you need this to work now, is to add an extra step in your build process that:

  • read the generated openapi.yaml
  • change the header
  • generate the openapi.yaml again

Hope this helps.

morphar avatar Apr 27 '22 07:04 morphar

Hi @yzhaoyu, That's a really good question. As far as I can remember, this is not currently possible. Maybe it's possible to build on @jeffsawatzky's recent work on extensions (#324 and #334) to get this functionality.

A simple solution, if you need this to work now, is to add an extra step in your build process that:

  • read the generated openapi.yaml
  • change the header
  • generate the openapi.yaml again

Hope this helps.

@morphar Thank you for your advice! I just tried in this way:

service LibraryService {
  // Creates a shelf, and returns the new Shelf.
  rpc CreateShelf(CreateShelfRequest) returns (Shelf) {
    option (google.api.http) = {
      post: "/v1/shelves"
      body: "shelf"
    };
    option (google.api.method_signature) = "shelf";
    option (openapi.v3.operation) = {
      description: "Request Content-Type is application/x-www-form-urlencoded"
      request_body: {
        request_body: {
          content: {
            additional_properties: [
              {
                name: "application/x-www-form-urlencoded"
                value: {
                  schema: {
                    schema: {
                      type: "string"
                    }
                  }
                }
              }
            ]
          }
        }
      },
      responses: {
        response_or_reference: [
          {
            name: "200"
            value: {
              response: {
                content: {
                  additional_properties: [
                    {
                      name: "text/plain"
                      value: {
                        schema: {
                          schema: {
                            type: "string"
                          }
                        }
                      }
                    }
                  ]
                }
              }
            }
          }
        ]
      }
    };
  }
}

However, the generated openapi.yaml file contains repeated content like this:

/v1/shelves:
    post:
        tags:
            - LibraryService
        description: Request Content-Type is application/x-www-form-urlencoded
        operationId: LibraryService_CreateShelf
        requestBody:
            content:
                application/json:
                    schema:
                        $ref: '#/components/schemas/CreateShelfRequest'
                application/x-www-form-urlencoded:
                    schema:
                        type: string
            required: true
        responses:
            "200":
                description: OK
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/Shelf'
            default:
                description: Default error response
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/Status'
            "200":
                description: ""
                content:
                    text/plain:
                        schema:
                            type: string

How can resolve the repeated field in the yaml? @morphar @jeffsawatzky @timburks

yzhaoyu avatar Apr 27 '22 09:04 yzhaoyu

Sorry, I meant that maybe it's possible to built this functionality into gnostic by building on the work @jeffsawatzky already did. I don't think it's currently possible to change the content type, without doing an extra build step, like the simple solution I suggested.

morphar avatar Apr 27 '22 09:04 morphar

Sorry, I meant that maybe it's possible to built this functionality into gnostic by building on the work @jeffsawatzky already did. I don't think it's currently possible to change the content type, without doing an extra build step, like the simple solution I suggested.

Yeah, I will try your suggestion. But I have a question about your step 3: generate the openapi.yaml again. If I have changed the header, why not directly use the changed yaml file. I don't understand how to generate the openapi.yaml.

yzhaoyu avatar Apr 27 '22 09:04 yzhaoyu

It was just an exmple of what I have done previously. I created a small Go program, that read the file, changed some of the fields and wrote the result to the same file.

In your case, the entire process, could probably be handled with something like sed.

morphar avatar Apr 27 '22 09:04 morphar

@morphar Thank you, I will try this.

yzhaoyu avatar Apr 27 '22 09:04 yzhaoyu

@yzhaoyu protoc-gen-openapi is meant to be used mainly for transcoders that transcode from JSON to gRPC, such as the grpc-gateway, the envoy gRPC-JSON transcode filter, or Google's own cloud endpoints projects.

If you are writing your own transcoder that doesn't use JSON, my first question would be "why?". Why not just use an off the shelf solution like one of the above.

If you happen to have a good reason as to why you are writing your own, then currently what you want isn't possible out of the box.

Normally if you wanted to send something other than JSON you'd use the google.api.HttpBody message. An example for use in envoy is here. And that's for sending content. I'm not even sure what you'd use for receiving arbitrary content.

Are you trying to create an HTML form that is posting to your endpoint? If you want you can just use javascript to submit the form for you with JSON using fetch or something like jQuery.

jeffsawatzky avatar Apr 27 '22 15:04 jeffsawatzky

+1 to @jeffsawatzky's comment - I think we want to stay as close as possible to the Envoy and grpc-gateway transcoding implementations. If we had to give precedence to one of these, I would probably go with Envoy for its wide use and support from Google's internal proxying team.

Looking in httpbody.proto, "This message [HttpBody] can be used both in streaming and non-streaming API methods in the request as well as the response." I don't recall trying this recently, but I believe that an rpc with HttpBody as the request type will accept arbitary content into the data field and set the content_type based on the request header.

timburks avatar Apr 27 '22 20:04 timburks

Beacuse of some historical reasons, some services use application/x-www-form-urlencoded to serve. @timburks @jeffsawatzky Thanks for your suggestions, I will have a try with httpbody.proto!

yzhaoyu avatar Apr 28 '22 10:04 yzhaoyu