electric-sql
Inconsistent header names in the HTTP API response.
We have two headers in the HTTP API response:
electric-chunk-last-offsetelectric-next-cursor
Both provide values that are meant to be used by a client in the next HTTP request. It's a bit odd that one is the last and the other is the next. I suggest that either we normalise them to both be next or we just drop that part of the name, so the headers could be:
electric-chunk-offsetelectric-cursor
Or even:
electric-offsetelectric-cursor
What do you think?
N.b.: just eyeballing it in the terminal:
➜ ~ curl -v http://localhost:3000/v1/shape/items\?offset\=26760640_0\&shape_id\=56264740-1729877730504\&live\=true
* Trying 127.0.0.1:3000...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> GET /v1/shape/items?offset=26760640_0&shape_id=56264740-1729877730504&live=true HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/8.1.2
> Accept: */*
>
< HTTP/1.1 204 No Content
< date: Fri, 25 Oct 2024 17:38:29 GMT
< vary: accept-encoding
< cache-control: public, max-age=5, stale-while-revalidate=5
< x-request-id: GAHCxpxEwSjUx7gAAAkB
< server: ElectricSQL/0.7.6-12-g7de9f1d
< access-control-allow-origin: *
< access-control-expose-headers: *
< access-control-allow-methods: GET, HEAD, DELETE, OPTIONS
< content-type: application/json; charset=utf-8
< electric-shape-id: 56264740-1729877730504
< electric-chunk-last-offset: 26760640_0
< electric-chunk-up-to-date:
< electric-next-cursor: 1445900
The up-to-date header is also prefixed by chunk. So we should presumably also change that to electric-up-to-date if dropping chunk from offset.
I do think that electric-offset can be confusing, cause it could mean "the start offset for this data" or "the end offset for this data", which is why last was part of it.
I agree that chunk can be removed as a keyword, but also don't think that we can match the cursor and offset. The electric-next-cursor does mean "this is the cursor you should use for the next request", but electric-chunk-last-offset means "this is the last offset included in this response's data", which incidentally is also what you would use to request the next chunk of data but it is not strictly the same - the former is exclusively an instruction for subsequent requests, the latter is information about the current response (which is necessary to form the next request)
I think this can be solved in documentation. I.e.: use a terse name like electric-offset and electric-cursor in the headers and then explain in the documentation (we have the openapi spec, and the HTTP API doc page, and the "Writing your own client" guide) what it represents.
This is similar to using terse parameter names like cursor, offset.
On the topic, I also advocate that we simplify electric-shape-handle and shape_handle to electric-handle and handle. When making a request to GET /v1/shape/... then it's clear that we're getting a Shape resource. We don't use shape_offset, we use offset. So I think handle -- clearly documented about what it represents -- is a better and more consistent parameter name. Paired with electric-handle in the headers.
I have a PR in progress implementing all this. I'll push shortly.
On the topic, I also advocate that we simplify electric-shape-handle and shape_handle to electric-handle and handle.
Sounds good, fyi we have an open PR to refactor to shape_handle already, so if you need to refactor it might be worth doing it on that https://github.com/electric-sql/electric/pull/1796
I think we now have a well defined set of headers that we expose both in our OpenAPI spec and have appropriate errors for checking they are allowed etc - @thruflo happy to close this issue?