Secret stream state initialization; Continue interrupted upload of encrypted stream;

[tomekit]

Secret stream documentation: https://libsodium.gitbook.io/doc/secret-key_cryptography/secretstream says: "Since the length of the stream is not limited, it can also be used to encrypt files regardless of their size."

In order to reliably upload multiple GB over the HTTP it is safe to assume that this will take a while, connection might drop or even system might crash over the period of hours / days.

I am looking for a way to start secretstream encryption from last saved checkpoint before failure took place.

In C API I believe that state is kept externally and can always be restored if needed e.g.

crypto_secretstream_xchacha20poly1305_state state;
crypto_secretstream_xchacha20poly1305_init_push(&state, header, key);

In Dart bindings I've found comment in the: secret_stream.dart:

  /// Unlike the C API, which requires manual state management etc., this API
  /// combines all the required parts into a single, simple dart stream. As a
  /// first step, crypto_secretstream_xchacha20poly1305_init_push is used to
  /// generate the stream state from the [key] and the created header is sent
  /// as the first message of the returned [SecretExStream].

Is it possible to initialize state manually with libsodium_dart_bindings in order to start encryption stream from a certain point?

[Skycoder42]

This is indeed a use case I have not thought of, but definitely something useful to look in to. The dart stream API makes it somewhat hard to do that. Initializing from a previously saved state is easy, but getting that state is hard.

The main problem is that dart streams, as well as all IO (including HTTP) is asynchronous and thus we cannot wait for individual chunks to be sent before "advancing" the state. I will however think about this a little more and see how it might be integrated into the existing APIs.

Also, while not ideal, you could directly use the C-Bindings, they are exported from the library and allow you direct access to the C API. However, then you have to take care of pointer-management etc. yourself as well, so I'd only recommend that as a last resort.

[tomekit]

I do appreciate your quick response on that. Even though the use case as described is valid and worth having in mind in the future, we've actually concluded that secretstream isn't for us, so if by any chance I've created some feeling of urgency it's definitely no longer important to me at least.

In the meantime I've received brilliant summary response regarding the streaming ciphers: https://crypto.stackexchange.com/a/106992/70896 which made us pursue other streaming protocol, because of lack of seeking in secretsteam:

it doesn't support random-access decryption because of the nonce XOR

[Skycoder42]

Okay. I think I will still leave this issue open, as it might be relevant for others as well.