Best way to serve large static files

[loriopatrick]

I'm trying to find the best way to send a large binary file. Looking at the ResponseSender code I see that I can send a byte array however I'd rather not buffer up all the contents in memory. I've begun a fork where I'm trying to abstract ResponseSender and make it changeable through the ServerConfig. Is this a good approach? I also thought about adding a new step in the netty pipeline to handle a new type of endpoint however that seemed like too much work.

Am I on the right path? Do you have any suggestions?

[blling]

I see some code segment from another project which used to send file, maybe it could give you some idea

@Override
public void sendFile(File file, @Nullable Multimap<String, String> headers) {
    Preconditions.checkArgument(responded.compareAndSet(false, true), "Response has been already sent");
    HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);

    setCustomHeaders(response, headers);
    response.headers().set(HttpHeaderNames.CONTENT_LENGTH, file.length());

    final boolean responseKeepAlive = setResponseKeepAlive(response);

    // Write the initial line and the header.
    channel.write(response);

    // Write the content.
    ChannelFuture writeFuture;
    try {
        FileChannel fc = new RandomAccessFile(file, "r").getChannel();

        final FileRegion region = new DefaultFileRegion(fc, 0, file.length());
        writeFuture = channel.write(region);
        writeFuture.addListener(new ChannelFutureListener() {
            @Override
            public void operationComplete(ChannelFuture future) throws Exception {
            	// region.releaseExternalResources(); //TODO: Azeez fix
                if (!responseKeepAlive) {
                    channel.close();
                }
            }
        });
    } catch (IOException e) {
        throw Throwables.propagate(e);
    }
}

[nicmunroe]

@loriopatrick the workaround @VicBing suggests would almost work, but it would mess up the built-in access logs, distributed tracing, and metrics. You might be able to get something working that way, but I'm guessing there would be several undesirable side effects.

Supporting user-friendly streaming responses from standard endpoints in Riposte is something that's been on the todo backburner for a while but we haven't had a concrete need yet so it remained on the backburner until now. If you have the motivation and desire to help us work on this it would be greatly appreciated, and should be easier than adjusting ResponseSender and/or adding Netty handlers.

The ProxyRouterEndpoints already do exactly the kind of streaming responses that you need, so I think we could piggyback on that. Take a look at ProxyRouterEndpointExecutionHandler.StreamingCallbackForCtx to see the nuts and bolts of what needs to happen, but essentially:

1. Create a ResponseInfo that represents the first chunk of the response. Headers, etc. Essentially everything except the payload. This ResponseInfo must be created using the ResponseInfo.newChunkedResponseBuilder() builder.
2. Get your hands on the HttpProcessingState and call state.setResponseInfo(responseInfo) passing in the ResponseInfo from step 1.
3. Call ctx.fireChannelRead(OutboundMessageSendHeadersChunkFromResponseInfo.INSTANCE) on the ChannelHandlerContext. That will cause the first chunk of the response to be streamed to the caller based on what you setup in step 2, but without finalizing the response.
4. Create a new OutboundMessageSendContentChunk each time you want to stream a new chunk of the response payload to the caller, and call ctx.fireChannelRead(contentChunkToSend) to send it.
5. Repeat step 4 until you're down to the final chunk, at which point you must do step 4 with a new LastOutboundMessageSendLastContentChunk instead of the normal OutboundMessageSendContentChunk. That will finalize the response.

Since this mechanism is used for the proxy/router endpoints it already fully supports all the access logging, distributed tracing, metrics, etc features of Riposte. Here are some things that will need to be fixed before it will work and be user-friendly though:

• NonblockingEndpointExecutionHandler.asyncCallback(...) expects full responses, and will throw an exception if it finds a chunked ResponseInfo returned by a standard endpoint (step 1 above). This was an artificial restriction we put in since we didn't yet support streaming/chunked responses from standard endpoints. Instead of throwing an exception, it should probably do steps 2 and 3 above.
• We shouldn't expect users to have to interact with the ChannelHandlerContext to fire events directly. We should probably have some separate interface where users can call sendChunk(...), and sendLastChunk(...) or finalizeResponse() or similar to wrap steps 4 and 5 above and make it user-friendly. This will need to be connected somehow to what is available from the endpoint... maybe exposed in ChunkedResponseInfo?
• Error handling/protection. Look in StreamingCallbackForCtx for some of the things that need to be considered.
• Additionally we'll need to account for race condition stuff - e.g. streaming the payload cannot begin until after the first chunk (headers/etc) has been sent to the caller, and from the endpoint developer's standpoint you don't know when that has occurred. There are a few potential solutions, including linking up some kind of callback so the endpoint knows when it's safe to start streaming, or buffering up the data internally until we know it's safe. A callback solution is probably preferred to avoid the risk of pulling too much payload into memory while waiting for the first chunk to be sent.
• Ideally the solution wouldn't copy/paste stuff from StreamingCallbackForCtx, but the two could reuse shared logic, or maybe the proxy/router endpoints could get rid of StreamingCallbackForCtx and use the new solution.

Hopefully that gives you a roadmap to start investigating. Let me know what you think.

[loriopatrick]

This is really good, thank you. I'll start toying around with it this weekend.