flowtide icon indicating copy to clipboard operation
flowtide copied to clipboard

State encryption

Open Ulimo opened this issue 9 months ago • 0 comments

This issue may or may not be done depending on actual requirements.

State encryption may be required in certain situations if one does not trust hardware encryption or where an IT administrator may require access to the storage for administrative tasks but should not be able to see the actual data.

For this to be fully compliant it should support key rotation as well where the entire state is re-encrypted with the new key.

The initial plan would be to add an encryption factory that can give create symmetric algorithm encryptors for the different storage clients (where each encryptor is used single threaded to allow potential caching such as buffers).

The factory would also need to be able to fetch all available pages from the current storage solution and fetch them to do key rotation. To also trigger a save of the data it needs the possibility to enqueue checkpoints. It could then for instance rotate 100 000 pages per checkpoint to still allow the stream to continue responding and trigger a new checkpoint directly after.

At this point there is no method to list all stored pages in a storage solution, so this would need to be added to all existing storage solutions.

Ulimo avatar Mar 06 '25 15:03 Ulimo