Pre-RFC Live queries

[acjay]

I've heard live queries alluded to on various podcast episodes and in the RFC for subscriptions, but it's unclear to me whether there's a repository of ideas on what this feature would look like.

Purpose

GraphQL supports an event-based real-time mechanism called subscriptions. These are useful for many uses cases, such as when you need a UI to explicitly reflect domain events to the user.

However, these aren't the only type of real-time semantics. Sometimes, a the UX is designed around depicting the real-time state of a domain, rather than discrete domain events. It is possible to serve this need with GraphQL subscriptions. Each event could effectively expose the entire root query schema, or the applicable slice of it. That way, clients could update arbitrary local state for every event. This brings up a couple challenges:

1. This could result in a great deal of over-pushing, if most of the schema hasn't changed in response to the new event.
2. There needs to be a way to ensure clients have never miss any information. Subscriptions don't natively support a first payload.

Less naive implementation on subscriptions

I think it may be possible to implement achieve this on top of subscriptions with some machinery.

Change types

For issue (1), there would need to be a way to represent changes to a schema. This could be done by representing every type A in a schema with a box type:

type Change[A] {
  change: A
} 

The box would be necessary for times when A is nullable, to differentiate between a change to null and no change.

The subscription representing a live query update would effectively be one giant "data model changed" event. It would need to provide a field that serves as the root of the live-updatable data. If the data is represented by type Data, this field would have a type of Change[Data]. Change[Data] would have a field of type Change[A] for every field of type A in Data.

This generic Change type doesn't need to literally exist, but it could be synthesized.

Server-side client model

To determine what changes to send, the server would either need to store the current client-side state (in the same way that it needs to store subscriptions queries) or its business logic would need to natively be capable of deriving diffs.

Array changes

This leaves out the question of efficient updates to arrays, which would naively be wrapped whole in a box. But that could still result in over-pushing and it might not play way well abstractions like Relay's connections.

Bootstrap events and consistency

For issue (2), there could be a synthetic event that the server fires immediately upon subscription success, which would have field containing the unaugmented payload of Data, which would allow the clients to bootstrap their local state. If the transport guarantees that either all messages are received or the connection fails (like Web Sockets), consistency would be guaranteed. Other transports would need some other system for ensuring consistency. Either way, applications would likely want to be able to depict interruptions of the connection.

So, how to do it?

The above approach is appealing in that it can conceivably be built on primitives that exist today, albeit with some schema augmentation and some GraphQL library machinery on the client and server sides. On the pro side, this would allow for event-based subscription payloads and live query updates to coexist in one protocol. On the con side, not having a first-class representation could make the pieces needed to achieve live queries feel disjointed.

I wanted to throw this out there as a rough proposal to get some comments and suggestions. If there's any interest, I can try to make it a proper RFC. My company is looking at adopting GraphQL for one of our real-time products, and I'm trying to look ahead to whether we'd be able to replace some bespoke Web Sockets stuff with something that's more of a standard. Maybe there's an opportunity for us to help push such a standard forward.