desci-labs
Standardizing DAG builder parameters
In the context of persistence, there is an issue in the IPFS network implementation described in the following discussions:
- https://stackoverflow.com/a/59184086/9560245
- https://discuss.ipfs.tech/t/does-the-ipfs-chunking-change-the-cid-for-the-same-file-chunked-differently/11608
- https://discuss.ipfs.tech/t/why-does-the-same-file-result-in-different-sha256-in-cid/14474
- ...and so one
This behaviour is natural and depends on the IPFS DAG builder and its defaults. It means the same initial bitstream may have different valid CIDs if built based on differently structured DAGs. To eliminate any possibility of non-deterministic resolution of data, the DAG builder parameters we are using while minting the dPID could be standardized. The exact parameters (chunker, hash function, CID structure) should also be openly declared in the documentation. This will eliminate the possibility of unexpected data duplication on the network, and allow the external users to report dPIDs in cases of dishonestly used libraries.
