libipld icon indicating copy to clipboard operation
libipld copied to clipboard

Make trait for Schema's without being tied to DagCbor

Open ProofOfKeags opened this issue 2 years ago • 3 comments

As far as I can tell the only code in this library related to schemas is directly tied to the DagCbor codec. As far as I understand, there is no fundamental reason these things should be tied together as Schemas are related to how the IPLD Data Model.

From the IPLD Schema Docs:

IPLD Schemas are, of course, based on IPLD. Because we get to reuse the layers of IPLD's Data Model and IPLD's Codecs, IPLD Schemas are a design language that can be used with many different serial formats.

So disentangling the DagCbor codec from the schema implementations seems somewhat imperative if this library wants to faithfully implement the IPLD design/spec.

ProofOfKeags avatar Nov 11 '22 22:11 ProofOfKeags

Think of it as one possible implementation. The advantage of this implementation is, that you can directly go from Rust types to the serialized version without an intermediate step.

I hope that the current DAG-CBOR codec from this library is replaced in the future with https://github.com/ipld/serde_ipld_dagcbor. Those schema-like transformations would then happen on the Serde level.

If you're interested in IPLD Schemas, there's also another IPLD implementation which focuses on schemas, which might also worth a looks: https://github.com/datalove-app/ipld/pull/2.

vmx avatar Nov 14 '22 10:11 vmx

I feel like I have to be missing something really important. Can you explain why IPLD should have anything to do with serde? serde has an entirely separate data model and IPLD's data model is what guarantees it's canonicalization which is what makes content addressing something that is viable. Without going through the IPLD data model I'm not sure how IPLD actually gives us anything. Serde already has a means of serializing out to different "codecs" (hence serde_json and serde_cbor).

ProofOfKeags avatar Nov 14 '22 18:11 ProofOfKeags

IPLD itself doesn't necessarily need to have anything to do with Serde. serde_ipld_dagcbor is using Serde to (de)serialize data, kind of only a subset of what Serde's power is. This custom codec makes sure that the result is valid data according to the IPLD specs. So in a sense it kind of is going through the IPLD Data Model implicitly. This is why we need a custom implementation like serde_ipld_dagcbor and cannot use serde_cbor (or similar general purpose Serde CBOR implementaitons) directly.

vmx avatar Nov 15 '22 11:11 vmx