hashgraph
Deterministic compression
Problem Description:
In some cases, before saving data to IPFS (e.g., Tools), Guardian creates a folder structure, transforms the data into files distributed within that structure, compresses the root folder, and saves the resulting file.
Because compression is currently not a deterministic operation, this approach can currently lead to an inability to prove that two components are equal, even though they are. It's just their compressed format that differs.
This is especially critical in some cases, such as when working with Tools.
Tools are published to IPFS after compression, and exporting them also involves compression into a zip file. This means that when you publish a policy or schema referencing a tool, there is no way to recover that information unless the exact file is available on IPFS.
For example, if tools are not available during the import process, the import will not end correctly.
With deterministic compression, users will be able to export, import, and share the exported/used files via p2p networks like IPFS or traditional sharing mechanisms. They will also be able to prove that they are using the same components.
Moreover, uploading that file to IPFS will enable an isolated or un-synced IPFS node to correctly serve that information to the Guardian. Currently, that is impossible unless the IPFS node is synced and the original author is still serving the file.
Also, for example, this will allow all the tools in the Methodology Library folder to be loaded into a local IPFS node at startup, and this allows them to be referenced and served correctly in all policies and schemas.
Requirements:
Implement deterministic compression for all cases in which compression is applied by Guardian.
Definition of done
When compression is applied twice to the same data, the resulting compressed files always have the same SHA-256 hash.
