Megolm Backup | Client should always have a local copy of all their megolm keys to limit the possibility of key loss

[BillCarsonFr]

Context

We implemented a recent change to the way we get the room keys from backup. Previously, the full backup was downloaded after verification, as soon as the backup decryption key was known, but that did not result in a good user experience when your backup was too big. The time to download the full backup and decrypt it could be taking 10s of minutes depending on the platform.

In order to improve that (and in the absence of pagination for the backup), keys are now downloaded on-demand when we try to decrypt an historical message. This results in a better user experience, as the messages get decrypted faster after a new login.

Problem

With this new way of getting the keys, all user sessions will now have only a partial set of the backed up keys. This can cause key loss if for example a user tries to reset his backup (without importing first from the previous backup).

Notice that the previous implementation of full backup import has several issues:

• Performance
• Lack of pause/stop/resume. Particularly on mobile, if the app was put in background and terminated by the OS before the end of import, it was not restarted.
• Was possible to start several import in parallel

Solution

We want to ensure that each new client gets a copy of all the keys in backup as soon as it can. The operation should be resilient to app restart and should have an API [richvdh: or, more to the point: a UI?] to report progress.

• EAX/EIX:
  • [ ] rust-sdk: Ensure that all the keys from a given backup version are downloaded
  • [ ] EIX: UI feedback for ongoing import
  • [ ] EAX: UI feedback for ongoing import

• Web (https://github.com/element-hq/element-web/issues/27009):
  • [ ] js-sdk: Ensure that all the keys from a given backup version are downloaded
  • [ ] https://github.com/element-hq/element-web/issues/17839
• Element Android:
• Element iOS

Related/Existing issues

• https://github.com/element-hq/element-web/issues/27009
• https://github.com/element-hq/element-web/issues/20694

[richvdh]

I wrote some notes on how we might do this for web at https://github.com/element-hq/element-web/issues/27009.

Notes:

• In a naive implementation, if the app is killed during the recovery, we'll restart from the beginning next time. Anything else is somewhat annoying without a fix to https://github.com/element-hq/element-meta/issues/2176.
• If your backup is too big, it's possible that we'll never successfully download it; the client will just keep retrying.

[richvdh]

Question: is there any value in attempting to download individual keys on-demand once the full backup has been successfully downloaded? I'd think not?

[andybalaam]

@richvdh will remodel this item based on https://docs.google.com/document/d/1eEj_BagixDuny_fhF_WjoKEm0SD538mTXA3s436-c2I/edit - we chose the "Update Rust backup manager, and refactor EW to use it" option

[BillCarsonFr]

Question: is there any value in attempting to download individual keys on-demand once the full backup has been successfully downloaded? I'd think not?

Short answer: Once it has been successfully downloaded, no. But while it's being downloaded definitely yes. If not it will take ages before I can see history after a login (unless we have this).

Longer answer:

• Maybe a quick race where when you start downloading, some new keys are being uploaded and you'd want to get them after?
• Maybe in case of identity reset some devices might become unsigned for some time and then be excluded from key distribution, but when they are verified again they could re-try the backup?