plumber icon indicating copy to clipboard operation
plumber copied to clipboard

Does plumber support gzipped files?

Open konradsemsch opened this issue 5 years ago • 3 comments

Hello!

there's been a recent discussion on the R Studio Community forum (link) about timing differences in local vs. remote API calls. One of the conclusions was that when the response is large it could be a good idea to compress is with gzip or so.

I wanted to follow-up and ask whether it's also possible for plumber to accept gzipped files in requests? I tried a couple of solutions but none of them worked and couldn't find any information in docs or online. Could you provide some minimal example that illustrates that functionality? A piece of advice on that would be very helpful!

konradsemsch avatar May 12 '19 12:05 konradsemsch

Check out https://github.com/jcpsantiago/protoplumb-playground/blob/9e677bcb07f46ba4063ea768c0cdbf6239e8c90a/prototest/plumber.R#L23 from #386 for an example on how protobuf is decompressed for all incoming requests.

I'm going to guess there's a header value set that can determine if the input value should be unzipped.

There is a lot of room for improvement on how plumber handles post body inputs.


Also, if you're doing R to R communication, check out serializer_rds (master branch). It's very small and keeps native objects.

schloerke avatar May 13 '19 15:05 schloerke

Thanks for your response! So in your view, at the current stage of plumber development, what would be the recommended path to compress the response and request in a way that the API could communicate with other languages? Both the inputs and outputs of our APIs are large.

Could you perhaps make a small example of how such CURLs would look like using protobuf and the rds serializer?

konradsemsch avatar May 13 '19 16:05 konradsemsch

@konradsemsch from the code @schloerke mentioned above at https://github.com/jcpsantiago/protoplumb-playground :

echo "a : 123" | \
protoc --encode=protoplumb.TestPayload prototest.proto | \
curl -v --header "Content-Type: application/x-protobuf; messagetype=protoplumb.TestPayload" -X POST --data-binary @- http://localhost:8000/echo | \
protoc --decode=protoplumb.TestPayload prototest.proto

## expected output:
## a: 123

this is how you would send a binary request to a plumber api running a protobuf filter (see https://github.com/ozean12/protopretzel for an implementation)

I take the messagetype header to get the correct proto file.

jcpsantiago avatar Mar 21 '20 16:03 jcpsantiago