pyspark-ai
pyspark-ai copied to clipboard
Signed cache
It is convenient for a developer to ship code in English and the cache data together to optimize speed, save cost, and more importantly reproduce the result. However, we need a way for end users to trust the cache data, which contains "compiled" code and could be modified by others between the user and the trusted developer. One way to do this is to let the developer sign the cache entries with a private key and share the public key with end users to verify the cached content.