nakadi
nakadi copied to clipboard
zalando
Create an endpoint to close a subscription event stream
Currently there are only 2 known ways to close a subscription event stream, neither of which are completely ideal. The first way is to manual close the HTTP connection, which often requires clients to manually keep track of http connections, and the later is to stop committing cursors which is hacky because its actually causing an "error" which happens to close the connection.
An endpoint should be provided which allows you to close an event stream (provided by the GET /subscriptions/{subscription_id}/events route), something like the following
POST /subscriptions/{subscription_id}/close
The post request also requires you to provide a X-Nakadi-StreamId as a header. It will close the stream, and also cause the stream to respond with a different status code (maybe 202) so it can be detected that this is a graceful shutdown of the stream
If someone will decide to implement this thing, than It is probably better to do
DELETE /subscriptions/{subscription_id}/sessions/{session_id}
@antban By session_id, do you mean X-Nakadi-StreamId? I don't think Nakadi exposes sessions to the client (unless I am missing something)?
sorry, of course
DELETE /subscriptions/{subscription_id}/streams/{stream_id}
Can you explain a bit more the situation where you do not have access to the http connection, but know the stream id?
Most client libraries would automatically reconnect on connection close, which would lead to a new stream id instead of a different http status.
It is possible to delete the subscription which I think would close active streams and then return a 4xx status code on reconnection.
I suggest also to have DELETE /subscriptions/{subscription_id}/streams to force-close all connections to a given subscription.
@jhorstmann You can say that under "normal" circumstances, such a route isn't needed however in more complex scenarios it is required.
Its just basically defining a way to gracefully close a stream, and you are right that its not impossible for a client to store a reference to a http request to manually send a close connection, however doing so adds extra complexity to the clients that need to do this (i.e. in Java/Scala you need to create some sought of ConcurrentHashMap that stores a reference from SubscriptId + StreamId to the request)
Also I just remembered, an obvious use case that has just come up is testing, where you do often need to manually close connections so that
@jhorstmann Also according to https://github.com/akka/akka/issues/18010 (i.e. the HTTP specification), a client closing a connection to terminate the stream is not defined behaviour, and per spec it means the server would still continue sending the data even after the client has terminated the underlying stream (at which point future incoming data is either lost or ignored).
The graceful way to handle this is to get the server to gracefully signify that the payload has is completed and to close the stream.
@mdedetrich Sounds more like a bug/feature in akka. Spring frameworks ClientHttpRequest abstraction has the same issue where it tries to consume remaining bytes from the network in order to to reuse the connection for later requests. This consumption will never finish for an event stream, unless the commit timeout of the high level api is reached. With spring it's possible to workaround this and properly close the connection on the tcp level.
@jhorstmann It isn't an issue with any framework, the behavior is considered undefined according to the HTTP spec, which is why clients treat this differently.
The point is, the desired behavior should be the server closing the connection on the client, not the other way around (in context of closing a stream). If the client closes connection on the server, at this point on the server can do any number of things, and so can the client
I suggest also to have
DELETE /subscriptions/{subscription_id}/streamsto force-close all connections to a given subscription.
+1
