Allow for HTTP protocol negotiation and configuration

[grantr]

I've been thinking about how we evolve the HTTP transport binding. Some of this may be relevant to non-HTTP transports, but I'm focusing on HTTP for now.

In Knative Eventing, we're using the HTTP transport binding with synchronous acks. This is great for demos, kicking the tires, and simple use cases. But supporting asynchronous acks in the protocol would allow more sophisticated clients and servers to increase performance and efficiency, plus enable new workloads with long processing times. The flip side is that async acks are more complex, adding a new request type and side channel for the client and server to deal with. It might be possible to add this capability to the protocol without a negotiation request, but I'd like to set us up for long-term evolution and I don't think trying to cram every possible protocol option into the delivery request is a scalable or efficient approach.

In HTTP, the OPTIONS method suits this purpose. For example, the client can use an OPTIONS request to ask the server which request methods are supported, and the response will list supported methods. Knative Eventing clients could similarly use an OPTIONS request to ask the server if it supports asynchronous acks and would like to enable them, and the server can respond to indicate yes or no.

I don't expect the SDK to implement different protocol APIs. Likely those will be application-specific. I'm asking if it's possible for the SDK to support:

• The client initiating an OPTIONS request
• The server handling an OPTIONS request (apparently this is supported already with https://github.com/cloudevents/sdk-go/blob/master/v2/protocol/http/options.go#L208)
• Swapping protocol implementations

Maybe the current SDK already supports this. If so, this can probably be closed right away. :smiley:

[grantr]

Here's an illustrative example of how this could work.

Request 1: OPTIONS negotiation

sender -->
OPTIONS /
CE-Acks: ActiveAsync, Sync, PassiveAsync
# Sender supports sync, passive async, and active async, prefer active async

<-- consumer
200 OK
CE-Acks: ActiveAsync
# Consumer supports active async only

Request 2: Event delivery

sender -->
POST /
CE-Delivery-URI: http://broker.example.com/acks/123456-ab3fa
CE-Delivery-Timeout: 600s
CE-Specversion: 1.0
CE-Type: com.example.event
CE-Source: /
CE-Id: 123456
# Sender includes a resource URI unique to this delivery attempt and sets an initial ack timeout of 10 minutes

<-- consumer
202 Accepted

Request 3: Extend ack timeout

<-- consumer
PUT /acks/123456-ab3fa
CE-Delivery-Timeout: 60s
# Consumer extends Timeout by 1 minute

sender -->
200 OK
CE-Delivery-Timeout: 60s
# Sender responds confirming the new timeout

Request 4: Ack delivery

<-- consumer
DELETE /acks/123456-ab3fa
# Consumer acks the delivery by deleting the resource

sender -->
200 OK

[n3wscott]

I think we can give hooks to let an integrator implement this. The negotiation logic might be out of scope for the SDK because we will need to upstream it to the spec. But that does not mean we can experiment with the ideas here and have a sample implementation.

[deissnerk]

I wonder, if this should really be done with HTTP. How would the state handling work? You have to make sure the ACK reaches the instance that sent the event or somehow sync state between the different instances. What about error handling? What happens, if request 4 returns 50x? You probably won't like this, but protocols like AMQP 1.0 have been designed to support this. 😉

[grantr]

I wonder, if this should really be done with HTTP.

I think you're talking about the async acks example, but this issue is really about the OPTIONS request handling, not async acks. An alternate example could use an OPTIONS request to negotiate a switch to AMQP 1.0 (or a different stateful protocol). In Knative HTTP is the basic assumption and that's unlikely to change, but I'd like to introduce the flexibility to choose an alternative.

You have to make sure the ACK reaches the instance that sent the event or somehow sync state between the different instances.

Shared state would certainly be the simplest approach to implementing this example. All brokers and channels already have shared state of some kind, so the difficulty of implementation depends on the capabilities of the messaging backends.

What happens, if request 4 returns 50x?

If request 4 returns 50x, the consumer can try the request again, or not and the message will be retried. I expect AMQP is similar in this situation.

[slinkydeveloper]

You probably won't like this, but protocols like AMQP 1.0 have been designed to support this.

100% agree with @deissnerk, it seems to me we want to replicate the capabilities of a full fledged messaging system on top of HTTP, but IMO that idea brings different problems not trivial to solve. In particular, what puzzles me is that some capabilities requires a stateful communication, that is persisting some state for every connection. That's something the HTTP semantics were not designed to do imo. For example, Kafka just has one tcp duplex connection per consumer/producer, so it can easily handle state on a single thread. That's not the case of HTTP/1.1, were client opens n connections to the server. HTTP/2 gives you multiplexing, but still the semantics are the same as HTTP/1.1