incomplete response when using mode=grpcwebtext

[anthonyrouseau]

When using mode=grpcwebtext for client configuration, responses are giving an error:

{code: StatusCode.UNKNOWN, message: 'Incomplete response'}

The response is received by the client, but is truncated (e.g. string should be 'hello world' but is 'hello'). This error does not occur when using mode=grpcweb. I'm unsure if additional configuration is needed to use grpcwebtext, but following the grpc-web examples it does not seem like it.

It looks like this may be related to https://github.com/grpc/grpc-web/pull/903 and issue https://github.com/grpc/grpc-web/issues/881

[stanley-cheung]

Can you give us more information please? There is not enough information to go on.

[adityavikasd]

I'm facing the same issue. I get an error in the response

{code: 2, message: 'Incomplete response'}

I followed the hello world example just as mentioned and used the following nginx.conf file (along with some CORS config not provided below) in a docker image

master_process off;
daemon off;
worker_processes 1;
pid nginx.pid;
error_log stderr info;
env GRPC_VERBOSITY=DEBUG;
error_log /dev/stdout info;

events {
  worker_connections 1024;
}

http {
  access_log /dev/stdout;
  client_max_body_size 0;
  client_body_temp_path client_body_temp;
  proxy_temp_path proxy_temp;
  proxy_request_buffering off;
  server {
    listen 8080;
    listen 8443 http2;
    server_name localhost;
    
    location / {
        grpc_pass host.docker.internal:9090;
    }
  }
}

I have added a line to print the incoming request to the method handler in the server.js file to see what is being received on the server when we make a gRPC-web request

function doSayHello(call, callback) {
    callback(null, {
        message: 'Hello ! ' + call.request.name
    });
    console.log("Request: ", call.request)
}

I see that there is no name in the request when the request is being logged. And, there are no errors on the server. It makes sense because if there was no name sent, then the server should just say hello without a name.

❯ node server.js                                                                                          
Request:  { name: '' }

The Chrome Dev tools shows that a payload is being sent for the request and I'm not sure if the payload contains a name, and a 200 status code is received, with a hello ! response too

I used the gRPC Web Dev tools chrome extension, found here https://github.com/SafetyCulture/grpc-web-devtools and I see that the name is being sent.

Note

After coming across this issue, I tried using grpcweb as the wire format and this works just fine. But, if I use the grpcwebtext format I can still reproduce this issue.

[adam-rocska]

With our case it happens even if I have our protos compiled as mode=grpcweb & we get the same rubbish error in the grpc-web-devtools mumbo.

We can confirm however, that the message should be there, as other clients in more appropriate languages & stacks can understand the receive error; only js has problems ¯_(ツ)_/¯

[adam-rocska]

@stanley-cheung I can grant you access to our stuff and guide you around if you'd need a repro-environment. It's quite annoying as it really works with all clients... even with the most disgusting php scripts 😆

This is how we do the rejection in the swift server for example (dumb down version of-course):

    func deAuthenticate(request: _21gram_Authentication_Token, context: StatusOnlyCallContext) -> EventLoopFuture<Google_Protobuf_Empty> {
        let status = GRPCStatus(code: .permissionDenied, message: "Pay up buttercup.")
        context.responseStatus = status
        return context.eventLoop.makeFailedFuture(status)
    }

[stanley-cheung]

Would help if there's a simple reproducible test case that I can clone, launch a docker image or something and reproduce the error.

Or is there a special situation that triggers this?

[adam-rocska]

@stanley-cheung I don't know if it's special or not; I don't think so. It shouldn't be.

You know what? I'll look into creating such a package later today (GMT+1) and ping you once it's done. I'm only worried about the back-end service side of things, as we roll with Swift & I know it's not mainstream yet ¯_(ツ)_/¯ Oh, and we don't do docker or any local env. virtualization, so that'll also add some fun.

[remvst]

I'm also facing the same issue. I am able to query my APIs using BloomRPC with the web toggle on, and targetting the envoy port, but when querying them from the web client, I get Object { code: 2, message: "Incomplete response" }. It doesn't seem like the request makes it even to the server, and I don't see anything in the network tab of the dev tools (in the client).

Both client and server are using typescript exports, in case that matters.

Server dependencies:

        "google-protobuf": "^3.14.0",
        "grpc": "^1.24.2",
        "grpc-tools": "^1.8.1",

Proto generated using:

protoc \
        --proto_path=../../proto \
        --js_out=import_style=commonjs:${PROTO_DEST} \
        --grpc-web_out=import_style=commonjs+dts,mode=grpcwebtext:${PROTO_DEST} \
        -I ../../proto \
        $file

Client dependencies:

        "google-protobuf": "^3.14.0",
        "grpc-web": "^1.2.1",

[remvst]

Okay seems like I was able to fix my issue. In case anyone is experiencing something similar, make sure you do specify the HTTP protocol in your client:

// Works
const client = new VideoClient('http://localhost:9000');

// Doesn't work
const client = new VideoClient('localhost:9000');

Would it make sense to add some assertion to the constructor to avoid these kinds of issues?

[maxekman]

I'm facing the same issue. Running grpc-web 1.2.1 against an Istio 1.7.6 ingress gateway. In my case I create the clients without hostname:

const client = new BookingServicePromiseClient('')

The problem occurs in Safari 14.0.1.

[chriss2401]

+1 here, facing the same issues with localhost:${port} or http://localost:${port}

[bram-abe]

I can solve the issue Object { code: 2, message: "Incomplete response" } same as @remvst stated above. Specify http:// in your client target URL.

[LukeLaScala]

Specifying http:// didn't resolve the issue for me. Are there any other workarounds at the moment? This is a super frustrating bug.

[mateodelnorte]

I'm in the same boat, @LukeLaScala. Did you find any resolution?

[mateodelnorte]

I'm in the same boat, @LukeLaScala. Did you find any resolution?

I was able to solve this with mode=grpcweb instead of mode=grpcwebtext. However, I'm concerned I'm limiting capabilities in doing so:

It appears long-running server side streaming is only available when using grpcwebtext to generate client code.

Note, the exact behavior is that any generated client calls to .unaryCall return a promise, but that promise never resolves. A 200 response is received with what looks like correct data, but it may be incomplete.

proto.company.proto.energy.thing.otherthing.v1.TheDomainThingAPIPromiseClient.prototype.getSomething =
    function(request, metadata) {
  return this.client_.unaryCall(this.hostname_ +
      '/company.proto.energy.thing.otherthing.v1.TheDomainThingAPI/GetSomething',
      request,
      metadata || {},
      methodDescriptor_TheDomainThingAPI_GetSomething);
};

The call, above, for instance, would return a Promise but it would never resolve or eventually resolve an error that looks like:

We are generating client code with protoc-gen-grpc-web-1.3.0-linux-x86_64 and leveraging latest grpc-web and google-protobufs versions in all code:

    "google-protobuf": "~3.14.0",
    "grpc-web": "^1.3.0"

[GoncaloHit]

Im having the same issue as above, i have tried, both grpcweb and grpcwebtext on the client url i use https everything that is successfully works, but on error it doesnt, the gateway is Kong with grpcweb plugin

[patryksliwinski]

I faced the same issue, grpcweb resolved this for me but I cannot use streams now

[dalazx]

In case you see { code: 2, message: "Incomplete response" } while you expect a properly set error, it might mean that the CORS configuration of your gateway/proxy does not allow browsers read grpc-status and grpc-message response header values. These two headers should be explicitly exposed as follows (along with any other headers that you might use):

access-control-expose-headers: grpc-status,grpc-message

This worked well for me using grpcwebtext.

See the example configuration for Envoy: https://github.com/grpc/grpc-web/blob/6d1ee0d3dfd60dae655d8ec983686872132af214/net/grpc/gateway/examples/echo/envoy.yaml#L30-L36

cc @GoncaloHit

[EByrdS]

Adding to my Envoy proxy

expose_headers: grpc-status,grpc-message

actually solved the issue for me, thanks a lot