js-sdk icon indicating copy to clipboard operation
js-sdk copied to clipboard

can't send msg >4mb

Open giang12 opened this issue 2 years ago • 10 comments

Expected Behavior

to be able to invoke/pub with req size > 4mb fixed...for both http and grpc clients

Actual Behavior

grpc client: error "code": 8, "details": "Received message larger than max (6632286 vs. 4194304)", http client: nothing happen.. return undefined...

Steps to Reproduce the Problem

try invoke remote services via sidecar and see... seem like missing 3rd options params for grpc client https://github.com/dapr/js-sdk/blob/7bf1bb8e965fee48144e6f9b5707f39e6cf4950b/src/implementation/Client/GRPCClient/GRPCClient.ts#L76

giang12 avatar Jul 24 '22 14:07 giang12

tried setting grpc.max_send_message_length grpc.max_receive_message_length from https://www.npmjs.com/package/@grpc/grpc-js ..as gathered from https://github.com/dapr/js-sdk/issues/70 still no success..

any recommendation?

giang12 avatar Jul 24 '22 14:07 giang12

Can you list your Dapr run command?

XavierGeerinck avatar Jul 24 '22 16:07 XavierGeerinck

dapr run --app-id ingress-app --app-port 8080 --dapr-grpc-port 3500 --dapr-http-max-request-size 20 -- node bin/cli.js (there is no equivalent --dapr-hrpc-max-request-size?) const client = new DaprClient(settings.daprHost, settings.daprPort, CommunicationProtocolEnum.GRPC, {'grpc.max_send_message_length': 1024 * 1024 * 1000,'grpc.max_receive_message_length': 1024 * 1024 * 1000});

tx for getting back so quickly

giang12 avatar Jul 24 '22 21:07 giang12

https://github.com/dapr/go-sdk/blob/main/examples/service/custom-grpc-client/main.go#L43 i saw in go-sdk they recommend manually setting up grpc conns https://github.com/dapr/go-sdk/pull/197

giang12 avatar Jul 24 '22 21:07 giang12

Hello @giang12, is HTTP mode working with --dapr-http-max-request-size?

shubham1172 avatar Jul 25 '22 16:07 shubham1172

hi @shubham1172 no unfortunately.. just tried again to make sure. I ran both sender and receiver in http mode like this dapr run --app-id ingress-app --app-port 8080 --dapr-http-port 3500 --dapr-http-max-request-size 44 -- node bin/cli.js dapr run --app-id echo-app --app-protocol http --app-port 50051 --dapr-http-port 3503 --dapr-http-max-request-size 44 -- node dist/index.js image never reached 2nd console.log..just return without any error somehow, just an empty {}

also seemed like the call never reach receiver app or dapr sidecar

giang12 avatar Jul 26 '22 02:07 giang12

Is it possible for you to share a Minimal, Reproducible Example? I can run this on my end and check.

shubham1172 avatar Jul 26 '22 02:07 shubham1172

zip attached sample.zip

tried sending to "go" app; same behavior

giang12 avatar Jul 26 '22 03:07 giang12

So, if I understand correctly, the scenario is from client -> invoke -> server

This would mean we need to implement https://github.com/dapr/js-sdk/blob/7bf1bb8e965fee48144e6f9b5707f39e6cf4950b/src/implementation/Server/HTTPServer/HTTPServer.ts#L48 correctly where the bodyParser utilizes limit (e.g. app.use(bodyParser.json({limit: '50mb'}));)

For gRPC some more research is required though, theoretically we need to pass options flags (such as the below)

channel = grpc.insecure_channel(
    'localhost:50051',
    options=[
        ('grpc.max_send_message_length', MAX_MESSAGE_LENGTH),
        ('grpc.max_receive_message_length', MAX_MESSAGE_LENGTH),
    ],
)

The workaround for this is to create your own custom HTTP server that utilizes the limit functionality.

XavierGeerinck avatar Jul 26 '22 06:07 XavierGeerinck

yeah accepting custom http servers/grpc conns seem to be simplest

giang12 avatar Jul 26 '22 16:07 giang12

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

dapr-bot avatar Jan 08 '23 15:01 dapr-bot

/keep-alive

XavierGeerinck avatar Jan 08 '23 16:01 XavierGeerinck