boost
boost copied to clipboard
Consistently handle carv2 files during the deal making flows
Summary
While doing some retrieval testing I was generating random, dense DAG car files to store. The default tooling resulted in these being CARv2 files, instead of Boosts preferred CARv1 files. The boostx commp and Boost deal flow, local commp verification do not produce the same commp. The code flow also seems to be different for the Lotus commp validation, making it likely impossible to make a deal with a CARv2.
While we do prefer CARv1 for deal making flows, we should support CARv2 as a lot of existing tooling in the ecosystem generates them by default (go-car for example).
Steps to reproduce
- Generate a CARv2 file using go-car (or the random car utility command, currently at https://github.com/filecoin-project/boost/commit/b0811c758f2d4686d3d263c7d2cdb8d85499414f)
- Calculate commp with
boostx commp - Make a deal against the local devnet, commp validation will fail as it doesnt match
Note: It also appears that the size check during commp validation is nondeterministic. In the screenshot below the car file is the same, but the "written" byte count varied between two deal attempts:

References
- Relates to https://github.com/filecoin-project/boost/issues/673