Offer Server Sent Events (SSE) as an alternative to polling for client SDKs

[Jamess-Lucass]

Problem

When using feature flags on a client, instead of having to implement request polling to determine if flags have updated without the user refreshing the browser, server sent events (SSE) provide an alternative solution.

Ideal Solution

Offer a server sent events endpoint for the evaluation APIs.

Some arbitrary url, eg. /evaluate/v1/boolean/sse

This endpoint would return a stream.

Upon making any change that mutates feature flags (creating, updating, deleting) we'd send an event to the client.

On the client you would create an EventSource and then implement the logic for receiving messages in the onmessage function. It's up to the client how they want to handle messages.

Introducing such a feature then means we're introducing some state, into what I assume is a stateless application so far, if you're running multiple versions of flipt behind a reverse proxy, you may connect to one server (server a) with SSE, and make an API call to update value of a feature flag to the other server (server b), server b then triggers the functionality to send a message in the stream, but any connections to server a will not receive this.

A potential solution is to back off the state to whatever undying database provider is being used? Or introduce a separate Redis instance and use something like Redis pub/sub to handle this in a distributed scenario.

[GeorgeMac]

Thanks for the suggestion @Jamess-Lucass 🙏

This is, I believe, really similar to a concept we have thrown around internally that we call client-side evaluation. (correct me if I am wrong here:)

I think streaming the evaluation of a single flag + context might not be very valuable. As you would then have to create a stream per context you want to watch. Contexts are usually tied to some identity of a caller in your application. So you're likely to create many streams in that situation.

Instead, you want to watch for change in all flags right (or a subset, but still the whole flag state itself)? Evaluation logic for the flag state (e.g. the flag + rules + rollouts + segments + constraints etc) is currently implemented server side in Go. If we just transmit the raw state for these configurations to a client then it has to perform the evaluation logic on its side.

This means we have to have evaluation logic in each language client.

Now, that said, we think this is a really valuable idea. If we can push the flat flag state to clients and they evaluate their side, we remove a network hop from our clients code paths. Since this state can propagate asynchronous from application side request paths. This is great. We love this idea.

The challenge comes in implementing client side evaluation. We could write it in each language, starting with a select few. That is on the table (though quite a bit of work to write and maintain).

Another idea we had, was to either extract the current evaluator, or re-implement it (in e.g. Rust) and then compile it to WASM. This might allow us to simplify client creation down to fetching state and invoking the WASM binary in each language.

Just some food for thought. I think we should keep this issue open and on the table to discuss these ideas.

[markphelps]

Yes great suggestion @Jamess-Lucass !!

Couple things to add:

• SSE / streaming could be very beneficial for users who just care if a flag is enabled or not and don't care about evaluation. As @GeorgeMac mentioned above, evaluation would likely still require either state to be maintained on the server or the clients would need to become 'smarter' and be able to perform evaluation client-side.
• We could implement streaming via the GRPC API with relatively little code since GRPC already supports streaming out of the box. It would still require a new endpoint to subscribe to changes, but I don't think it would be a huge lift. We could actually make this an 'audit' API as audit events already get written for every change in Flipt: https://www.flipt.io/docs/configuration/observability#audit-events

[Jamess-Lucass]

Thanks for the responses :D

Correct, I agree streaming for a single flag and context wouldn't be very valuable. Watching for all flags is what I'm after, the Audit API sounds like a good fit if that includes all changes, however, there may need to be a way to specify whether certain flags should be made available to the client, I can imagine use cases where a flag is created, which is only meant to be available server side, and thus, would not want to be included.

I would prefer if the evaluation happens server side, this could possibly work by, when making the initial http request to the server sent events endpoint, pass the context in the url.

As an example, I will just base64 encode the following json

{
  "context": {
    "name": "John Doe"
  },
  "entityId": "1"
}

ewogICJjb250ZXh0IjogewogICAgIm5hbWUiOiAiSm9obiBEb2UiCiAgfSwKICAiZW50aXR5SWQiOiAiMSIKfQ==

curl --request GET \
  --url https://try.flipt.io/evaluate/v1/sse?value=ewogICJjb250ZXh0IjogewogICAgIm5hbWUiOiAiSm9obiBEb2UiCiAgfSwKICAiZW50aXR5SWQiOiAiMSIKfQ== 

Then whenever a flag has an update made, perform the evaluation server side as we have context and stream back the following in the EventStream Data.

{
  "key": "example-feature-flag",
  "value": false
}

This then still means there is a need to store some state on the server.

The only objections I would have to performing it client side, is the duplication of logic for performing evaluation, and needing to ensure any code changes are propagated through every client SDK, it sounds like you have ideas about how to potentially mitigate this and make it maintainable with Rust & WASM, I don't have any experience around this so can't comment, however if there are solutions to the maintenance of having it run client side then that's fine by me.

[markphelps]

Hey @Jamess-Lucass ! Sorry for the silence the past few days. I was wondering if this is potentially two separate feature requests:

1. Audit events streamed over SSE. So whenever anything thing changes you can consume these events in your application without polling. As of the latest minor release (1.27) audit events are also filterable in case you dont want everything
2. SSE for evaluation, where an event is pushed whenever a flag changes that affects some previous context that you sent us.

I can see how the latter would be very powerful, but it also has it's own things for us to figure out, not limited to:

• How we store the hashed contexts in your example
• How many we store (ie for each flag)/for how long
• What happens if you miss the event/dont consume it on your end?

How to implement 1 is a lot more clear to me currently, but wanted to get your thoughts and learn more about how you would like to use either/both features

[alexcardell]

Is there an open issue or discussion for the wider client-side evaluation chat? I am really keen for that, we're looking into Flipt and one of the things putting me off is server-side evaluation

[markphelps]

@alexcardell there isnt an issue/discussion currently I don't think , but we are actively prototyping a way to do client side eval now so this is very good timing!

would you mind creating one? either discussion or issue and we can use that for feedback?