connectrpc
Missing trailers override?
I am trying to use connectrpc-es with an nginx grpc-web proxy https://nginx.org/en/docs/http/ngx_http_grpc_module.html
I have the proxy working, with the exception that connectrpc-es throws an error over "missing trailers" despite having received the message intact. I am aware that nginx should send trailers, but it would seem that they're not planning to do that https://github.com/grpc/grpc-go/issues/892.
Is there any way I can work around this so that connectrpc-es will forego throwing an error when the trailers are missing? I was hoping for something like shouldThrowOnMissingTrailers: false but I checked the docs and see no such configuration options.
Hey Gryphon, the error message is not about actual HTTP trailers. Since browsers do not expose headers to JavaScript, gRPC-web encodes the trailers that gRPC uses into the response body. Payload messages and trailers are delimited with a binary framing.
For some reason, it appears that the response body does not contain the gRPC-web trailer. We cannot ignore this case, since gRPC and gRPC-web encode errors in the trailer. Without trailers, you could call RPCs all day, and will never know that they failed :sweat_smile:
I suspect that something else is happening. Can you:
- Make sure that you have set up CORS correctly, if you are calling RPCs across origins. See here for thorough documentation: https://connectrpc.com/docs/cors
- Check request and response:
- Request field Content-Type should be
application/grpc-web(+protoor+jsonis fine). - Same for response field Content-Type.
- For successful calls, response field Grpc-Status should be
0or omitted, and response body should be non-empty. - For failed calls, response field
Grpc-Statusmay be > 0 - in this case, the body should be empty.
- Request field Content-Type should be
If this does not help you resolve the issue, please feel free to share request and response details from the browsers network explorer here, and we might be able to help.
Hey Gryphon, the error message is not about actual HTTP trailers. Since browsers do not expose headers to JavaScript, gRPC-web encodes the trailers that gRPC uses into the response body. Payload messages and trailers are delimited with a binary framing.
For some reason, it appears that the response body does not contain the gRPC-web trailer. We cannot ignore this case, since gRPC and gRPC-web encode errors in the trailer. Without trailers, you could call RPCs all day, and will never know that they failed :sweat_smile:
I suspect that something else is happening. Can you:
- Make sure that you have set up CORS correctly, if you are calling RPCs across origins. See here for thorough documentation: https://connectrpc.com/docs/cors
- Check request and response:
- Request field Content-Type should be
application/grpc-web(+protoor+jsonis fine).- Same for response field Content-Type.
- For successful calls, response field Grpc-Status should be
0or omitted, and response body should be non-empty.- For failed calls, response field
Grpc-Statusmay be > 0 - in this case, the body should be empty.If this does not help you resolve the issue, please feel free to share request and response details from the browsers network explorer here, and we might be able to help.
Thanks for the quick response. CORS is setup, content-type is set correctly, the response has no gRPC errors, the only evidence of anything going wrong is the "missing trailers" error being thrown by connectrpc-es. In fact, if I set a breakpoint on the thrown error, I can even see the data payload was received, but then it throws the error anyway over the lack of trailers in the response.
In searching this error, I've found evidence online indicating that this is a quirk of the nginx implementation of grpc-web https://github.com/grpc/grpc-web/discussions/1322#discussioncomment-9367998 (also see link in my previous comment indicating that nginx does not intend to address this).
I understand that nginx really should stick to the spec, but from what I can see, the lack of trailers is not actually interfering with the receipt of a well-formed payload. This is why I ask if we might have a way of putting connectrpc-es into a more forgiving mode where it will not throw when the trailers are missing.
Also, "just use Envoy" is not an option. I can confirm that it does make this missing trailers issue go away, but for other reasons, it is not viable for our use case.
evidence online indicating that this is a quirk of the nginx implementation of grpc-web
Okay, I'm starting to see what's going on, thanks for the link to grpc-web discussion. To clarify: nginx does not implement grpc-web with the ngx_http_grpc_module. The purpose of the module is to pass gRPC requests to gRPC servers. This is not a quirk of it's grpc-web support - it doesn't support grpc-web.
While it's possible to add CORS headers and tweak the Content-Type header with nginx, the crucial part for gRPC-web - encoding trailers in the body - is missing.
If all you need is support for the happy path, the best way forward is to copy grpc-web-transport.ts into your project, and to modify it not to throw on line 227.
If we were to remove this check, users could easily get into a situation where everything seems to work, but the server is broken in a way that prevents the client from reporting an error. I hope you understand that this is not a good option.
evidence online indicating that this is a quirk of the nginx implementation of grpc-web
Okay, I'm starting to see what's going on, thanks for the link to grpc-web discussion. To clarify: nginx does not implement grpc-web with the
ngx_http_grpc_module. The purpose of the module is to pass gRPC requests to gRPC servers. This is not a quirk of it's grpc-web support - it doesn't support grpc-web.While it's possible to add CORS headers and tweak the Content-Type header with nginx, the crucial part for gRPC-web - encoding trailers in the body - is missing.
If all you need is support for the happy path, the best way forward is to copy grpc-web-transport.ts into your project, and to modify it not to throw on line 227.
If we were to remove this check, users could easily get into a situation where everything seems to work, but the server is broken in a way that prevents the client from reporting an error. I hope you understand that this is not a good option.
Of course not. The check should absolutely stay. My suggestion was opt-in configuration to bypass it. I would understand hesitance to do this because it seems a specific implementation (nginx) is the problem, even if it is one of only a few options serving this need.
I had the same issue but i did not using Nginx. But i was getting missing trailer with no reason! That was interesting to me why is this happening. And after hours of researching i found that createGrpcWebTransport raises the error. And i migrated to createConnectTransport and problem fixed ^-^.
We will not add an option to bypass the check. It will look like a solution, but it will completely discard all errors reported by the server, and we cannot guarantee that calls might hang or return invalid data (e.g. a unary call with an empty body without trailer data).
I have the same issue of ConnectError: [unknown] missing trailer. My CORS settings look good in my opinion. Also, switching from createGrpcWebTransport to createConnectTransport did not help at all. With the first one the server normally responds and the nginx proxy as well as the browser show a 200. I also read the following statements of yours:
The message prefixes in the body have one byte reserved for flags. gRPC-Web uses one bit to designate a message as trailers. Trailers are encoded like HTTP 1.1 headers, but keys must be lower-case. ngx_http_grpc_module clearly does not implement this transcoding, envoy's gRPC-Web filter does.
Interestingly enough the same nginx config worked perfectly fine for google's grpc-web lib/generated client.
Could you help me @timostamm? If this is the case I can post my config etc. Maybe even in a seperate discussion?
