New scheme for room (replace ArbitraryStorage table and datastore)

[mahibi]

Currently the app uses datastore and room, while room only has 2 tables (User and ArbitraryStorage). It's currently not clear when to use datastore(via AppPreferencesImpl) or ArbitraryStorage.

There should be only room used to store any data. ArbitraryStorage should be replaced as it contains mixed usecases which is not good (user settings and conversation settings).

A new scheme must be created which for the beginning contains tables for:

• app wide settings
• user wide settings
• conversation settings

while always having in mind that in the long term we also want to have full offline support.

Chat

• [x] Develop offline-first data layer for chat
  • #3630
• [ ] Devise and implement DB Schema to enable reading from chat offline
  • #3928
• [x] Create Background job to monitor network connections
• [x] Implement Pull-Based Synchronization to update DB on internet connection.

Conversations

• [ ] todo

User Settings

• [ ] todo

App Settings

• [ ] todo

[rapterjet2004]

@AndyScherzinger @nickvergessen - Can you give me some insight into how federation might effect this?

With federation being a priority, I'll procrastinate on developing offline-writing in favor of just read only.

docs

• app wide settings
• user wide settings
• conversation settings
• chat settings

Based off my reading, this will need a full MVVM implementations of all the above. In this way, a common repository will be able to support 2 data sources for the application in the data layer. See chart.

Fortunately, we already have MVVM infrastructure for the Chat and Conversations. NGL ArbitaryStorage is pretty spaghetti, so that's going to be a little bit of a pain to deal with. At minimum, I want to be able to model every Conversation and ChatMessage, as to enable reading without access to internet. Writing will be a different challenge, as offline-first apps write locally -> perform synchronization with server on network connection. See chart,

I will need to maintain a background service that monitors the Network connection.

I think the pull-based method of conflict resolution works best for talk-android. I feel that most users will be using this at home/office and only venture outside away from data/WiFi for short periods of time. From the docs

In pull-based synchronization, the app reaches out to the network to read the latest application data on demand. A common heuristic for this approach is navigation-based, where the app only fetches data just before it presents it to the user.

This approach works best when the app expects brief to intermediate periods of no network connectivity. This is because data refresh is opportunistic, and long periods of no connectivity increase the chance that the user attempts to visit app destinations with a cache that is either stale or empty.

Advantages	Disadvantages
Relatively easy to implement.	Prone to heavy data use. This is because repeated visits to a navigation destination triggers unnecessary refetching of unchanged information. You can mitigate this through proper caching. This can be done in the UI layer with the cachedIn operator, or in the network layer with a HTTP cache.
Data that is not needed will never be fetched.	Does not scale well with relational data since the model pulled needs to be self sufficient. If the model being synchronized depends on other models to be fetched to populate itself, the heavy data use problem mentioned earlier will be made even more significant. Furthermore, it may cause dependencies between repositories of the parent model and repositories of the nested model. Need to keep this in mind when designing the Schema - @rapterjet2004

[nickvergessen]

So there is https://github.com/nextcloud/spreed/issues/10680 Maybe that helps?