Added event cache

[alanpoon]

Fixes #442, #435 Allows displaying of offline cached messages and room list.

Tested:

• [X] Starting Robrix in offline mode and reconnecting to internet. But the performance is bad, because all the rooms are synced together, causing the UI to freeze
• [ ] UI to clear room cache

[alanpoon]

What are the performance problems you're referring to? Can you elaborate on that?

Also, I'm confused, because I thought the SDK did not fully support saving/restoring the event cache to/from disk. I asked about this previously but I think you might have missed it. Last I heard, the SDK implementation of event caching was not complete; is it fully working yet?

Finally, Robrix currently assumes that there will not be any existing timeline content when syncing and displaying rooms. What happens now that we restore old timeline content to each room? At a minimum, we'll encounter the problem of needing to support forward pagination, because there will very likely be a gap between the old events from the event cache and the newest events fetched via sliding sync. We don't have a way to handle that yet.

Added Forward pagination after internet reconnection.

[kevinaboos]

Finally, Robrix currently assumes that there will not be any existing timeline content when syncing and displaying rooms. What happens now that we restore old timeline content to each room? At a minimum, we'll encounter the problem of needing to support forward pagination, because there will very likely be a gap between the old events from the event cache and the newest events fetched via sliding sync. We don't have a way to handle that yet.

Added Forward pagination after internet reconnection.

I appreciate that, but .... it's not that simple. First, the code you added will unconditionally run forward pagination on all rooms in the entire client. We never want to do that, as a user might have thousands of rooms, the vast majority of which will not be opened by them. Doing something like that would be incredibly wasteful of CPU, memory, disk, and network resources on both the client and server.

Second, and most importantly, we need significant infrastructure across multiple parts of Robrix in order to be able to handle forward pagination correctly. All of the pagination-related code we have thus far assumes only backwards pagination, so running a forward pagination request will not work as well as you expect. For example, we don't efficiently handle the case of merging in new items from a recent forward-pagination request into the existing timeline. We also don't have a way to represent that a timeline may have a gap of missing events at any point within it, rather than just before it's current starting point. There are many other cases that we also have to consider beyond these.

This requires serious modifications/improvements to Robrix's code. I would suggest that we pause work on enabling the event cache until those tasks are complete.

[alanpoon]

Finally, Robrix currently assumes that there will not be any existing timeline content when syncing and displaying rooms. What happens now that we restore old timeline content to each room? At a minimum, we'll encounter the problem of needing to support forward pagination, because there will very likely be a gap between the old events from the event cache and the newest events fetched via sliding sync. We don't have a way to handle that yet.

Added Forward pagination after internet reconnection.

I appreciate that, but .... it's not that simple. First, the code you added will unconditionally run forward pagination on all rooms in the entire client. We never want to do that, as a user might have thousands of rooms, the vast majority of which will not be opened by them. Doing something like that would be incredibly wasteful of CPU, memory, disk, and network resources on both the client and server.

Second, and most importantly, we need significant infrastructure across multiple parts of Robrix in order to be able to handle forward pagination correctly. All of the pagination-related code we have thus far assumes only backwards pagination, so running a forward pagination request will not work as well as you expect. For example, we don't efficiently handle the case of merging in new items from a recent forward-pagination request into the existing timeline. We also don't have a way to represent that a timeline may have a gap of missing events at any point within it, rather than just before it's current starting point. There are many other cases that we also have to consider beyond these.

This requires serious modifications/improvements to Robrix's code. I would suggest that we pause work on enabling the event cache until those tasks are complete.

I have tested, there is forward synchronization after internet reconnection after opening my proxy.

https://github.com/user-attachments/assets/5b3318cd-fafb-4d81-878b-854b5f4d66d1

[alanpoon]

I have tested, there is forward synchronization after internet reconnection after opening my proxy.

That's good, but that's only part of what I was wondering about.

The real test is the behavior when you run Robrix for a while, close it, and then re-open it without any internet connection. What happens when you finally reconnect to the internet and there is a lot of history missing, e.g., multiple weeks of messages?

For example, if you were offline for a month, there will be a gap in the timeline, like this:

Messages from May 21
Messages from May 22
Messages from May 23
...
... (offline)
...
...
Messages from June 22
Messages from June 23

The messages from May will be in your previous event cache, and then the messages from June would be pulled down when you reconnect. But since Robrix only currently supports the TimelineFocus::Live mode, it expects there to be no gaps in the message history (because it doesn't expect anything older than its current message history to exist).

Before enabling the offline/cached mode, I assumed that we would need to support both (1) timelines with gaps, and (2) different TimelineFocus modes where we are able to load both earlier and later messages at a given point in the timeline.

Is this not the case?

Manually handling ChunkContent::Gap was in my first PR https://github.com/project-robius/robrix/pull/442. I don't think it is required for newer matrix version.

[kevinaboos]

Thanks Alan!

Now that this is merged, we'll have to re-work the login flow such that we allow the user to see the main room content before they have logged in. Currently, we hide everything until the user logs in, which means that if they start Robrix in an offline state, they'll never be able to get past the login screen.

So we need to change the logic to only show the login screen if there was no previously-logged-in session. We'll also need to show the main room content immediately upon starting Robrix (if there was a previous session), and perform the automatic re-login process in the background.