MSC4186: Simplified Sliding Sync

[erikjohnston]

Rendered

[jmartinesp]

I've made a suggestion in Synapse to implement a membership field for Room objects. This would simplify the logic for the clients a lot, avoiding checks and assumptions, and make sure they'll have consistent behaviours since it'll be the server who explicitly sets this value.

[foresto]

It would be nice if room lists had a way to filter by room tags. For example, a client might want only rooms with the m.favourite tag, or to exclude rooms with the m.lowpriority tag. (In fact, I have a non-chat client that wants the latter by default.)

Tags are stored in a room's account data, though. I see that this proposal explicitly relegates account data retrieval to an extension (MSC3959). What about account data as a filtering condition?

I don't know how widely users are tagging their rooms today, but if it's common (or ever becomes so), this would seem a good opportunity to lighten the load on the servers and make clients feel faster. And having the capability here the core protocol, rather than an extension, would make clients more likely to implement it.

[jevolk]

This MSC is due for FCP and should be finalized without further delay. The following justifications apply:

1. The total time for this project and its predecessor is approaching four years which will be nearly one third of Matrix's entire existence.
2. It is the second evolution of the concept incorporating corrections; refining from experience with the first and no longer purely exploratory.
3. It is widely implemented with support in the notable servers, major language SDKs and popular clients. It may even be difficult to make substantive changes to the proposal at this point without significant disruption.
4. It only voluntarily deprecates legacy sync which remains specified and otherwise implemented as a cushion, rather than forcing any immediate obsolescence.
5. It is the sole method used by mobile clients like Element X due to matrix-rust-sdk not providing any fallback to legacy sync in its Kotlin bindings (et al). This already forces its implementation in existing production systems.

This MSC is already widely deployed in the production systems of business and government customers. Its status is approaching that of a de facto standard.

The actual Matrix specification must remain relevant, lest it become a set of open XEP's for everyone to pick and choose from. That's not what anybody wants to see.

[foresto]

This MSC is due for FCP and should be finalized without further delay.

What would be the point of a comment period on a spec that gets finalized without addressing the comments?

[erikjohnston]

I've reworked the MSC (fairly substantially) to a) include full API schemas, and b) clarify exact behaviour.

Currently, this MSC describes the Synapse implementation. I wanted to start from there so its easier to track any changes we make going forwards.

[erikjohnston]

I've made a bunch of minor breaking changes, see the changelog at the bottom of hte MSC. I have tried to address most of the open comments in them.

[erikjohnston]

I think this is ready for the SCT to read and weigh in on. In particular, attention should be paid to the changelog at the bottom that lists all the things that I propose we change from the current implementation (there are also some notes in the body of the text where things haven't been implemented by synapse).

I think the "controversial/open" questions that should get SCT agreement are the following, including my votes on the subjects (which reflect the current MSC):

1. Should we implement "sticky" lists? My vote is: yes no. Thread
2. Should we rename the pos parameters? My vote is: no. Thread
3. Should we implement the notification counts? My vote is: yes. Thread
4. Should we deduplicate heroes between heroes and required_state? My vote is: no. Thread
5. Should we replace timeline_limit expansion with a bulk /messages API? My vote is: no. Thread

@mscbot fcp merge

[mscbot]

Team member @mscbot has proposed to merge this. The next step is review by the rest of the tagged people:

• [ ] @clokep
• [x] @dbkr
• [x] @uhoreg
• [x] @turt2live
• [x] @ara4n
• [ ] @anoadragon453
• [ ] @richvdh
• [ ] @tulir
• [x] @erikjohnston
• [ ] @KitsuneRal

Concerns:

• ~~Pending complete implementation (action: remove needs-implementation label)~~
• unresolved threads

Once at least 75% of reviewers approve (and there are no outstanding concerns), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for information about what commands tagged team members can give me.

[erikjohnston]

MSCs proposed for Final Comment Period (FCP) should meet the requirements outlined in the checklist prior to being accepted into the spec. This checklist is a bit long, but aims to reduce the number of follow-on MSCs after a feature lands.

SCT members: please check off things you check for, and raise a concern against FCP if the checklist is incomplete. If an item doesn't apply, prefer to check it rather than remove it. Unchecking items is encouraged where applicable.

MSC authors: feel free to ask in a thread on your MSC or in the#matrix-spec:matrix.org room for clarification of any of these points.

• [x] Are appropriate implementation(s) specified in the MSC’s PR description?
• [x] Are all MSCs that this MSC depends on already accepted?
• [x] For each new endpoint that is introduced:
  • [x] Have authentication requirements been specified?
  • [x] Have rate-limiting requirements been specified?
  • [x] Have guest access requirements been specified?
  • [x] Are error responses specified?
    • [x] Does each error case have a specified errcode (e.g. M_FORBIDDEN) and HTTP status code?
      • [x] If a new errcode is introduced, is it clear that it is new?
• [x] Will the MSC require a new room version, and if so, has that been made clear?
  • [x] Is the reason for a new room version clearly stated? For example, modifying the set of redacted fields changes how event IDs are calculated, thus requiring a new room version.
• [x] Are backwards-compatibility concerns appropriately addressed?
• [x] Are the endpoint conventions honoured?
  • [x] Do HTTP endpoints use_underscores_like_this?
  • [x] Will the endpoint return unbounded data? If so, has pagination been considered?
  • [x] If the endpoint utilises pagination, is it consistent with the appendices?
• [x] An introduction exists and clearly outlines the problem being solved. Ideally, the first paragraph should be understandable by a non-technical audience.
• [ ] All outstanding threads are resolved
  • [ ] All feedback is incorporated into the proposal text itself, either as a fix or noted as an alternative
• [x] While the exact sections do not need to be present, the details implied by the proposal template are covered. Namely:
  • [x] Introduction
  • [x] Proposal text
  • [x] Potential issues
  • [x] Alternatives
  • [x] Dependencies
• [x] Stable identifiers are used throughout the proposal, except for the unstable prefix section
  • [x] Unstable prefixes consider the awkward accepted-but-not-merged state
  • [x] Chosen unstable prefixes do not pollute any global namespace (use “org.matrix.mscXXXX”, not “org.matrix”).
• [x] Changes have applicable Sign Off from all authors/editors/contributors
• [x] There is a dedicated "Security Considerations" section which detail any possible attacks/vulnerabilities this proposal may introduce, even if this is "None.". See RFC3552 for things to think about, but in particular pay attention to the OWASP Top Ten.

[clokep]

I think the "controversial/open" questions that should get SCT agreement are the following, including my votes on the subjects (which reflect the current MSC):

I've already given a bunch of reviews on here, so I thought highlighting my stance on these questions might be the most useful thing.

1. Should we implement "sticky" lists? My vote is: yes. Thread

This feels like a thing we can add in later easily by having clients opt into a list being sticky. I'm not seeing a great cost savings by doing this right now (at further complexity to the API).

2. Should we rename the pos parameters? My vote is: no. Thread

Yes. It is completely inconsistent with our current APIs. See matrix-org/matrix-spec#125, matrix-org/matrix-spec#1280.

3. Should we implement the notification counts? My vote is: yes. Thread

I don't see why not? Is the concern performance?

4. Should we deduplicate heroes between heroes and required_state? My vote is: no. Thread

I don't have a strong opinion, but I think no -- that's pushing complexity onto the client.

5. Should we replace timeline_limit expansion with a bulk /messages API? My vote is: no. Thread

I'm not sure, it feels like it would simplify this API a lot and there's a bunch of good alternatives that don't have their pros/cons explored in the MSC. I think there's something missing here.

[turt2live]

@mscbot concern Pending complete implementation (action: remove needs-implementation label)

Note: anyone on the SCT should be able to @mscbot resolve this concern. Just highlighting that implementation isn't quite there yet.

[erikjohnston]

While not everything has been implemented, the vast majority of it has been. I believe that the stuff not implemented are either a) sufficiently minor to not worry about, or b) features that we have in sync v2 and so know are implementable server-side.

@mscbot resolve Pending complete implementation (action: remove needs-implementation label)

[richvdh]

@mscbot concern unresolved threads