Use `io.ReadSeekCloser` for message body
Currently the pool.Message uses an io.ReadSeeker as its body. This is convenient because it allows for setting a "handle" to response or request data, then fetching data incrementally as needed. However, it is common for a remote client to require connections to be closed after they are no longer in use in order to be returned to a pool. Currently, if a handler sets the message body, then sets the response on the ResponseWriter, it will return before that response is actually processed, meaning the handler itself cannot handle closing. This is especially pertinent in the case of large blockwise transfers, where data may be fetched incrementally over a long period of time.
I have a branch where I have experimented with updating the body from io.ReadSeeker to io.ReadSeekCloser, then called req.Body().Close() in pool.ReleaseMessage, which is a natural point to close a connection as the underlying request is being returned to the pool and callers should not access it directly anymore. Using an io.ReadSeekCloser is a little more cumbersome in some areas, such as all of the places where body is set to a bytes.Buffer, but a small helper (ReadSeekNopCloser, similar to io.NopCloser) can smooth over these rough edges.
@jkralik if you're interested in this change, I would be happy to implement!
@jkralik following up here -- is this a change you are interested in?
@jkralik any thoughts?
@jkralik could you indicate whether this is something you would accept upstream or not? If so, we'd be happy to provide an implementation.
@jkralik circling back here -- would love to get some feedback on whether this is something you are open to accepting upstream.