Streaming encryption/decryption

[dholms]

Problem

WebCrypto does not support streaming encryption/decryption. This means that if we want to display a large file in the browser, we need to read the entire file into memory, decrypt it, and the stream it to the renderer. https://github.com/w3c/webcrypto/issues/73

Solution

We're using AES-CTR, so after calculating the nonce, the counter is predictable for each block.

W'll need to read one block at a time, encrypt, write to outStream & increment the counter

[expede]

@dholms how much effort would you expect this to take?

@bmann can you add a few words on why this got prioritized so high?

[bmann]

I didn't move it, so it just stayed wherever it is. I've Icebox'd it for now.

[expede]

@bmann cool cool thanks :)

[dholms]

I'd probably budget a day for this. Like with a lot of crypto stuff, it's sort of hard to tell. Theoretically it shouldn't be too hard, but if you run into any issues, it's very hard to debug because you're just dealing with nonsensical bytes.

But yeah icebox for now :+1: . This doesn't matter much until we start streaming large private files (private videos for instance)

[icidasset]

I might try to tackle this one soon-ish, so we can add full Fission support to Diffuse 🙏 We'll need this to stream any media via the standard HTML elements.

@dholms I don't think I'll be able to do in one day. So any pointers are welcome 😄

[dholms]

:+1:

Unfortunately it might be a bit of a pain to do...

Basically:

• WebCrypto itself doesn't support streaming. It just takes in-memory data & encrypts/decrypts it (which seems really silly for a crypto library, I wonder if they'll add it at some point)
• You never want to use the exact same AES key twice (security vulnerabilities). You get around this by adding a nonce or counter or some piece of info to the key before each operation
• AES encrypts/decrypts in chunks (let's say 256 bits). So to kick off a normal encryption, you use some nonce, encrypt 256 bits, use that encrypted 256 bits to generate another nonce, encrypt the next chunk, etc
• What we want to do is essentially "fake stream" (read 256 bits off the incoming stream into memory, encrypt it with WebCrypto, write the encrypted bits to the outgoing stream)
• So we either need to manually calculate that nonce after each block or (my proposed plan), use AES-CTR (CTR is short for "counter") which uses a predictable nonce
• AES-CTR basically just increments the nonce for each subsequent block. So you'll generate an arbitrary nonce for the first block (but for simplicity let's say you generate 1), then the next block will use 2, then 3, then 4 etc. So this is predictable and we should be able to fake streaming this way.
• I don't think this should add any performance overhead

The other option is to just leave the library the way it is, fully read info into memory, encrypt/decrypt it, and then just write that to a stream. This is pretty memory intensive & won't work well for large files. But if the goal is just a streaming API and you're working with smaller files (I think songs would be fine :thinking:) then this would work as a stopgap

Let me know if this makes sense & if you want any clarity on anything or if a call might help!

[expede]

Posted in Discord, but relevant here: https://security.stackexchange.com/questions/65085/using-aes-as-a-stream-cipher

TL;DR they agree with @dholms on the AES-CTR idea, and give similar tradeoffs.

Importantly:

Within the limits expressed above, CTR mode is as good as such things can get.