electric-sql
Implement backwards-compatible transition from 409 to 205 status code for shape expiry
Problem
Currently, when a shape expires and needs to be re-fetched, the server returns a 409 (Conflict) status code with a "must-refetch" control message. We've received feedback that users interpret 409 as an error condition, which creates confusion. Since this is a normal part of the shape synchronization protocol, not an error, we should use a 2xx success status code.
Solution
Implement a feature negotiation mechanism using an "accepts" query parameter that allows clients to opt into receiving 205 Reset Content instead of 409 for must-refetch scenarios, while maintaining backwards compatibility with existing clients.
Why 205: The 205 Reset Content status code is semantically perfect for this use case - it explicitly tells the client to reset the document view. While RFC 7231 specifies that 205 responses cannot have a body, we already provide the new shape handle via the electric-handle header, so the client has all the information needed. This also provides excellent observability as 205 responses are easy to track in metrics and logs.
Implementation Details
1. Server-side changes (Elixir)
a) Update parameter validation in /packages/sync-service/lib/electric/shapes/api/params.ex:
- Add `accepts` field to the embedded schema
- Parse accepts as a comma-separated list of feature strings
- Store parsed features in the request context
b) Update error response generation in /packages/sync-service/lib/electric/shapes/api/error.ex:
- Modify `must_refetch/1` to check if client accepts "reset-205" feature
- Return status 205 with no body if accepted
- Return 409 with must-refetch control message for backwards compatibility
- Always include the `electric-handle` header with the new shape handle
c) Update API endpoint handlers:
- Pass the accepts features through the request pipeline
- Ensure feature negotiation is available when generating shape expiry responses
- Set `Content-Length: 0` for 205 responses per RFC 7231
2. Client-side changes (TypeScript)
a) Update ShapeStream options in /packages/typescript-client/src/client.ts:
- Add `accepts` parameter to ShapeStreamOptions interface
- Default to `["reset-205"]` for new client versions
- Allow override for testing backwards compatibility
b) Update request URL building:
- Add accepts parameter to query string when present
- Format as comma-separated list (e.g., `accepts=reset-205` or `accepts=reset-205,other-feature`)
- Add "accepts" to ELECTRIC_PROTOCOL_QUERY_PARAMS for proxy forwarding
c) Update response handling:
- Handle both 409 and 205 status codes for shape expiry
- For 205: Read the `electric-handle` header and reset the shape
- For 409: Continue existing behavior with must-refetch control message
- Same reset behavior for both status codes
3. Update OpenAPI specification
Update /website/electric-api.yaml:
- Add `accepts` query parameter documentation
- Document 205 response for shape endpoint when shape expires
- Keep 409 response documentation for backwards compatibility
- Explain feature negotiation mechanism
Migration Strategy
- Phase 1: Deploy server support for `accepts` parameter (returns 205 when `reset-205` requested, 409 otherwise)
- Phase 2: Update TypeScript client to send `accepts=reset-205` by default
- Phase 3: Update other clients (Elixir, community clients) to adopt the new behavior
- Phase 4: After sufficient adoption, consider making 205 the default for clients that don't specify accepts
Testing Requirements
- Server returns 409 with must-refetch when accepts parameter is absent
- Server returns 409 with must-refetch when accepts doesn't include "reset-205"
- Server returns 205 with no body when accepts includes "reset-205"
- Server always includes `electric-handle` header on shape expiry (both 409 and 205)
- TypeScript client handles both 409 and 205 correctly for shape expiry
- Existing clients continue to work without changes
- Proxy configurations forward accepts parameter correctly
Documentation Updates
- Update HTTP API documentation to explain the accepts parameter
- Document that shape expiry is a normal protocol operation, not an error
- Update client documentation with new accepts option
- Explain why 205 is semantically appropriate for this use case
Implementation Example
Client request with new behavior: ``` GET /v1/shape?table=users&offset=10_5&handle=abc123&accepts=reset-205 ```
Server response when shape expired (new behavior): ``` HTTP/1.1 205 Reset Content electric-handle: new-handle-456 Content-Length: 0 ```
Server response when shape expired (legacy behavior): ``` HTTP/1.1 409 Conflict electric-handle: new-handle-456 Content-Type: application/json
[{"headers": {"control": "must-refetch"}}] ```
Benefits
- Perfect semantics: 205 explicitly means "reset your content" which is exactly what we want
- Better observability: 205 responses are easy to track in metrics/logs as shape resets
- Backwards compatible: Existing clients continue to work unchanged
- Future-proof: Establishes a pattern for protocol evolution
- No confusion: 2xx status clearly indicates expected protocol behavior, not an error
References
- Current 409 implementation: `/packages/sync-service/lib/electric/shapes/api/error.ex:20`
- TypeScript 409 handling: `/packages/typescript-client/src/client.ts` (line with 409 check)
- OpenAPI spec 409 response: `/website/electric-api.yaml` (409 response definition)
- RFC 7231 Section 6.3.6: 205 Reset Content specification
Minor but obvious thought, just to ask the question: could we use a header instead of a param?
I would see a header as more appropriate for client version / capability signalling. I appreciate it needs to be passed through the proxy and perhaps that'll be more automatic for a param, using our "pass through these params" list. Perhaps we could introduce a "pass through these headers" list?
People don't understand the 409s at first, but then they get it.
Is it a good moment to change this? It adds a bit of complexity to handle the keep compatibility with deprecated 409s. It would be abetter time to revise it when we look into improving our caching strategy.
could we use a header instead of a param
No — custom request headers trigger much stricter CORS handling which would trip up people a lot. Also, everything else in the protocol is a query param so there's no particular reason to change this.
People don't understand the 409s at first, but then they get it.
It's a lot of friction as each person looking at logs on their team and many more sophisticated users will have the same reaction (what's wrong with electric???) — nor is this much work to work around it & we need a way to evolve the protocol incrementally like this anyways.