Improve performance of encrypted sqlite DB

[kegsay]

As seen in https://github.com/matrix-org/complement-crypto/pull/122 which times out due to how slow it gets with encryption enabled via .Passphrase being set on the client_builder.

Example timings:

• TestAliceBobEncryptionWorks/{rust_hs1}|{rust_hs1} - this creates 2 clients, 1 room and Alice sends 1 message to Bob. It's the simplest E2EE test. 910ms without encrypted DB, 4.86s with.
• TestFallbackKeyIsUsedIfOneTimeKeysRunOut - this consumes all OTKs (blocking the client uploading more) then claims the fallback key, so it will cause the client to re-upload 50 OTKs. 2.52s without, 8.43s with.
• Overall: 3m29s to run without, 9m28s to run with.

This is 3-5x worse.

This means I can't realistically enable encrypted DBs in tests, which is desirable in order to match EX.

[poljar]

I think that this is mostly due to our PBKDF usage in the default case when we encrypt the store. The usage of a random key is possible with our primitive itself:

https://github.com/matrix-org/matrix-rust-sdk/blob/7a85b7abdcfc05881ccbddf54a2094273be90110/crates/matrix-sdk-store-encryption/src/lib.rs#L151-L182

But it's not easily exposed in the ClientBuilder like the passphrase based mechanism is:

https://github.com/matrix-org/matrix-rust-sdk/blob/7a85b7abdcfc05881ccbddf54a2094273be90110/crates/matrix-sdk/src/client/builder.rs#L216-L228

We would need a new method to let us create stores with a random key instead.

[multisme]

Hi, can you extend more on the need here? Are you talking about adding to BuilderStoreConfig a Option field that would contain the encryption key?

[poljar]

Yes, instead of requiring the usage of a passphrase that is used to derive the encryption key, we should let users provide an encryption key themselves.

This avoids the slow PBKDF step.