Event cache: Move decryption retry in there

[bnjbvr]

Right now, it's the Timeline API that takes care of retrying decryption of an event. This has a few bad consequences:

• retrying decryption, while it could be done in other contexts than the Timeline API, can only happen there
• some observers (e.g. the UTD hook) want to know when a previously-UTD event could be decrypted; this happens in the Timeline code too, which is an abstraction leak
• some other code basically retries decryption on its own: the latest event code in the base crate does that

I'm proposing that we move all of this over to the Event Cache. An event in a linked chunk / on disk would be replaced by its decrypted equivalent, if needs be, and we can implement a few enhancements mentioned in a few other issues too.

High-level rough plan

Names to be refined.

• We have the UTD hook / manager, which helps notifying whenever there's a UTD.
  • It should be moved in a central location, could be in the Client for instance.
• The timeline needs a way to call into the UTD hook, to report that a UTD event has been displayed.
• I propose to have a new Redecryptor component which role is to react to
  1. new UTD events coming from sync
  2. new interesting events / notifications that some UTDs may be decryptable now
  • generic over a new trait RedecryptorCtx, which should help testing the component in isolation:

trait RedecryptorCtx {
  /// Returns the current list of UTD events known by the context.
  /// For instance, the event cache can load those from the database, deserializing only the event type.
  async fn current_utd_events(&self) -> Result<Vec<Event>, SomeErrorType>;

  /// Called back whenever UTD events have been decrypted by the `Redecryptor`.
  async fn on_resolved_utds(&self, events: Vec<Event>) -> Result<(), SomeErrorType>;
}

• The event cache will implement the above trait.
  • on_resolved_utds will replace now decrypted events in the linked chunk and/or in memory.
    • also trigger an update for observers like the timeline (using RoomEvents::replace_event_at + Update::ReplaceItem for the store, as it's done here)
  • current_utd_events can either load each time from the DB and deserialize only the event type… or do something better
    • we could index the event type in the database, as a separate column in the events table
    • we could maintain a list of UTDs: load it from DB first, then keep it in sync (after we received new UTDs from sync, or we successfully decrypted some previous UTD)
    • this could be composed with the UTD hook, so that a resolved UTD also is taken into account by the UTD hook immediately. I think the above trait would lend itself to composition in a nice way, by having a UtdHookRedecryptorCtx wrap another RedecryptorCtx.
• the Redecryptor would allow creating a task that listen to interesting events / notifications from other mechanisms (e.g. backup), and call the RedecryptorCtx as suited.
  • Ideally, there would be at most one single task to listen to all these things for all the rooms, using tokio::select! and global event handlers. We should not have a task per room, or multiple tasks, if we can.
  • The code to create the task should live alongside the Redecryptor, but the instantiated task can be owned by the EventCache or the Client themselves, for simplicity.

Flow

• The event cache creates a RedecryptorCtx instance, then creates the Redecryptor listening task
• Some events may come from sync: when they do, they're stored as such in the event cache, and propagated to the timeline.
  • When the timeline renders one such UTD, it calls back into the UTD hook.
• The listening task observes interesting events / notifications.
  • When it considers it has new information, it calls into RedecryptorCtx::current_utd_events()
  • The event cache's implementation will load UTD events from the database (or from the up-to-date list)
  • If the Redecryptor managed to, well, re-decrypt, it then calls back RedecryptorCtx::on_resolved_utds with the decrypted events
    • The event cache's implementation stores those decrypted events in the in-memory chunk / database, and emits updates for observers
    • The UTD hook wrapper implementation of the trait takes that into consideration, and choose whether it should consider those late-decrypt or actual UTDs, etc.

Future improvements

• [x] https://github.com/matrix-org/matrix-rust-sdk/issues/3768#issuecomment-2252779570
• [ ] https://github.com/matrix-org/matrix-rust-sdk/issues/4106

Part of #3058.