Support for end-to-end encryption

[iXo]

Hi, I am sorry to use the issue for that, but how can I decipher messages went I have a RoomEncryptedContent instance ?

Does the sdk provides a way to do that ?

[ma1uta]

Not yet, but planned.

[brevilo]

I wonder, is this issue meant to cover support for end-to-end encryption (and thus also libolm) or should I open a new issue for that? I'd like to start the discussion on how to best get there...

[iXo]

My ticket was indeed a request for a way to had libolm integration to be able to process e2e messages, and generate encrypted messages to be able to post.

[brevilo]

Ok, thought so. How about renaming the issue accordingly, e.g. "Support for end-to-end encryption"? We can then start the discussion here on how to proceed.

[brevilo]

A few notes to get things started.

I had a look at the official guide on E2EE and, fortunately enough, all crypto primitives have already been dealt with and it's usually best practice to not roll your own. This means we need to look into integrating the olm library into jmsdk. Unfortunately there aren't yet any Java bindings available. The closest match would be matrix-kt which provides Kotlin bindings. Kotlin can in principle be used from Java but that usually comes with its own challenges (like additional required patches for upstream), so I'm not sure this would be the best way forward. However, should we decide to integrate olm directly, matrix-kt can certainly serve as guidance (or even a template).

That said, I think these should be the next steps:

• Should jmsdk integrate olm directly or use a 3rd-party integration like matrix-kt?
• If directly:
  • How to integrate olm (C library)? JNA?
  • Where to integrate it? jeon vs. jmsdk / common vs. client / API vs. impl
  • Maybe it should even be a separate project such that others can integrate it easier (like matrix-kt)
• If 3rd-party:
  • Can and should matrix-kt be the way forward?
  • Any alternative integrations available?
• Who can do what?

Last but not least: @ma1uta, what did you have "planned" so far?

Cheers

[ma1uta]

Currently jmsdk doesn't support decrypt/encrypt methods. You can use JNA.

I think to have pure java implementation of the olm/megolm.

[brevilo]

I think to have pure java implementation of the olm/megolm.

What precisely do you mean by that? Do you mean pure Java bindings to the existing olm C-library (like I mentioned above), or do you mean a (roll-your-own) re-implementation of the olm's underlying crypto (double-ratchet) in pure Java?

[iXo]

Ideally, I think that pure olm implementation in java would allow it to works everywhere without any external dependencies. But someone needs to understand well how olm works, and that's why I did create this ticket, as my knowledge don't go that far. I was able to bypass that by using an hybrid solution with a small Go client that will only decode and encode messages, the communication between my Java program and the Go program is done by using Apache Thrift, but it is not very ideal.

[brevilo]

pure olm implementation in java would allow it to works everywhere without any external dependencies

I understand the motivation but I don't support the conclusion for three reasons:

1. Experience shows that it's not a good idea to roll you own crypto. There are countless examples where that approach failed time and time again. The existing olm/megolm C-library is mature, well supported and already underwent a security audit. I don't see how reinventing the wheel by doing all of that again would pay off.
2. We have a very limited number of people to implement whatever we decide upon. If I'm not mistaken we are 2-3 people currently interested in such an effort. I'd argue we need to focus on what we need (Java bindings) while reusing well-tested implementations (olm) in order to get things done in a reasonable amount of time.
3. C can also be compiled everywhere and, being of such central importance, the E2EE implementation should be performing as good as possible, so native optimized C might be a better choice for the low-level computations than Java.

[Polve]

Is it possible to use BouncyCastle to avoid running your own crypto libraries?

[brevilo]

Hm, I don't think so. BountyCastle looks like a set of crypto primitives (like AES) and protocols (like TLS) while olm is a Matrix-specific implementation of Signal's double-ratchet algorithm, by the Matrix team itself.

That said, olm is already the official best practice implementation. All that's missing for us to use it are Java bindings.

[iXo]

pure olm implementation in java would allow it to works everywhere without any external dependencies

I understand the motivation but I don't support the conclusion for three reasons:

1. Experience shows that it's not a good idea to roll you own crypto. There are countless examples where that approach failed time and time again. The existing olm/megolm C-library is mature, [well supported](https://gitlab.matrix.org/matrix-org/olm/-/project_members) and already underwent a [security audit](https://gitlab.matrix.org/matrix-org/olm#security-assessment). I don't see how reinventing the wheel by doing all of that again would pay off.

2. We have a very limited number of people to implement whatever we decide upon. If I'm not mistaken we are 2-3 people currently interested in such an effort. I'd argue we need to focus on what we need (Java bindings) while reusing well-tested implementations (olm) in order to get things done in a reasonable amount of time.

3. C can also be compiled everywhere and, being of such central importance, the E2EE implementation should be performing as good as possible, so native optimized C might be a better choice for the low-level computations than Java.

I was just giving my point of view, in general I prefer not relying on JNA / JNI, but either method used will be ok for me.

[brevilo]

Sure, no worries. This is meant as an open discussion and I was also just presenting my reasoning why I think I'm in favor of using libolm.

That said, I gave matrix-kt a try and ran into a number of obstacles, that lead me to create a proof-of-concept for pure Java bindings to libolm, using JNA. The JNA wrapping is already done and I'm fleshing out the tooling around it right now. I have a test app going that solely uses jmsdk and my new bindings. While the former is probably going to end up in a dedicated project, the latter could eventually become part of jmsdk's client, I think.

[brevilo]

There you go: jOlm (feedback and PRs welcome)

Probably not (yet) the most beautiful codebase, but certainly a decent start and it does the job 😏

My current test "client" is too convoluted for jmsdk integration but I'll try and come up with a less complex example. Apart from an example we still need to decide how far we should integrate E2EE in jmsdk's client implementation. I'd argue it would be great see the following implemented:

• Send/receive encrypted P2P messages
• Send/receive encrypted room messages
• Provide support for device verification (SAS)

Opinions?

[brevilo]

Oh, and @ma1uta, please review the open Jeon PRs/issues that came up during development. Would be great to see a new release with those things fixed.

Cheers!