zed icon indicating copy to clipboard operation
zed copied to clipboard

Lake/pool level encryption

Open philrz opened this issue 3 years ago • 0 comments

As this issue is being filed, Zed is currently at commit 28f95a3.

A community zync user has stated that changes in government regulations make it such that they need to have all personal data encrypted by September of 2023, and therefore would need the ability to encrypt the data in their Zed lake/pools. At a high level, we confirmed that this user would be ok with an approach that simply encrypts everything at the storage layer, and we might get more fine-grained later. We know encryption would have some impact on performance, but we could start by implementing a simple approach and measuring the impact. The perf would get some help from the vector cache when it's finished since that would live in memory and not have to be encrypted.

In brainstorming about implementation (key handling, etc.), we recognized that we should do some research into how other open source systems solve this problem.

philrz avatar Sep 28 '22 17:09 philrz