Support to export into libolm pickles

[poljar]

[codecov[bot]]

Codecov Report

Base: 87.58% // Head: 88.16% // Increases project coverage by +0.58% :tada:

Coverage data is based on head (b5dab00) compared to base (5ece305). Patch coverage: 87.84% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #95      +/-   ##
==========================================
+ Coverage   87.58%   88.16%   +0.58%     
==========================================
  Files          32       32              
  Lines        1603     1724     +121     
==========================================
+ Hits         1404     1520     +116     
- Misses        199      204       +5     

Impacted Files	Coverage Δ
src/lib.rs	86.66% <ø> (-13.34%)	:arrow_down:
src/olm/session_keys.rs	0.00% <ø> (ø)
src/sas.rs	94.64% <ø> (ø)
src/utilities/mod.rs	98.00% <ø> (ø)
src/olm/session/chain_key.rs	92.30% <25.00%> (-1.98%)	:arrow_down:
src/olm/session/ratchet.rs	81.48% <25.00%> (-1.13%)	:arrow_down:
src/olm/session/root_key.rs	96.42% <50.00%> (-3.58%)	:arrow_down:
src/olm/account/mod.rs	90.25% <82.85%> (-2.18%)	:arrow_down:
src/olm/session/mod.rs	90.24% <88.88%> (+6.11%)	:arrow_up:
src/megolm/group_session.rs	95.83% <100.00%> (+0.48%)	:arrow_up:
... and 12 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[dkasak]

We also need to update the README to remove the notice that libolm pickle support is "read-only".

[nico-famedly]

Is there anything blocking this or something I could help to push this forward? I would like to support both libolm and vodozemac for a while in our SDK and if it was possible to switch between both, that would make a few things easier until we migrate away from libolm entirely.

[poljar]

Is there anything blocking this or something I could help to push this forward? I would like to support both libolm and vodozemac for a while in our SDK and if it was possible to switch between both, that would make a few things easier until we migrate away from libolm entirel

I think it's just a lack of time due to a FOSDEM crunch.

Beware that the transition from libolm to vodozemac is losless, but vodozemac has larger buffers for one-time keys and some other objects.

This makes the transition from vodozemac back to libolm lossy.

[nico-famedly]

Beware that the transition from libolm to vodozemac is losless, but vodozemac has larger buffers for one-time keys and some other objects.

Yes, I am aware and partially documented it already internally. We generally wouldn't want to transition back, but we want to somewhat gracefully handle the cases where someone might start an older version of the app and similar for a while.

I think it's just a lack of time due to a FOSDEM crunch.

Okay, that sounds absolutely fair. I was mostly just thinking this would be useful for me during the migration and wanted to make sure it couldn't use some help. Thank you for the reply!

[tulir]

Any chance of merging this? I'm thinking of switching mautrix-python to use vodozemac, but the database needs to remain compatible with mautrix-go, which uses libolm or goolm. Longer term goolm could get vodozemac pickling support, but probably won't happen in the near future

[poljar]

Let's see if I can resurrect this PR.

[codecov-commenter]

Codecov Report

:x: Patch coverage is 83.91304% with 37 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 91.03%. Comparing base (7d36b9c) to head (470ab7d). :warning: Report is 2 commits behind head on main. :white_check_mark: All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/olm/session/mod.rs	75.00%	22 Missing and 4 partials :warning:
src/megolm/mod.rs	83.87%	3 Missing and 2 partials :warning:
src/olm/session/ratchet.rs	70.00%	3 Missing :warning:
src/megolm/group_session.rs	92.85%	1 Missing :warning:
src/megolm/inbound_group_session.rs	92.85%	1 Missing :warning:
src/olm/session_keys.rs	0.00%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #95      +/-   ##
==========================================
+ Coverage   90.64%   91.03%   +0.38%     
==========================================
  Files          34       34              
  Lines        4940     5154     +214     
  Branches     4940     5154     +214     
==========================================
+ Hits         4478     4692     +214     
+ Misses        297      292       -5     
- Partials      165      170       +5     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #95 will not alter performance

_{Comparing poljar/libolm-pickle-encode-support (470ab7d) with main (99aebd6)}

Summary

✅ 6 untouched

[poljar]

Alright, the PR is ready for review, that being said:

I'm thinking of switching mautrix-python to use vodozemac, but the database needs to remain compatible with mautrix-go, which uses libolm or goolm.

This isn't a particularly great idea, since the conversion to a libolm pickle will always be a lossy operation. One of the reasons for this is because vodozemac can store much more one-time keys compared to libolm.

[tulir]

I assume it'll throw away the same keys that libolm would've thrown away at generation time? (as in the behavior is exactly the same either way)

I shouldn't have any need for more than 100 one-time keys nor more than 2 billion messages in an olm session