specs icon indicating copy to clipboard operation
specs copied to clipboard

unixfs: suggest, but do not strictly mandate unixfs<>dagpb

Open rvagg opened this issue 3 years ago • 2 comments

The strict coupling poses some problems for IPLD implementations that:

  1. do not retain codec information between the serialization and UnixFS (ADL) reification; and/or
  2. do not have a mechanism to strictly mandate that UnixFS data be only encoded in a particular codec.

A mandate in the specification that strictly defines the layering on top of the codec makes it difficult to implement it as an ADL, which also presents difficulty for using tooling that builds on IPLD lens-style layering.

Ref: https://github.com/ipfs/go-unixfsnode/pull/27 Ref: https://github.com/ipfs/specs/pull/271 Ref: https://github.com/ipld/go-car/pull/304


So, this is basically a take-two on https://github.com/ipfs/specs/pull/271, but with an extra helping of nuance in an attempt to open the door slightly, but not too far that this complicates life for existing implementations. The main aim here is to allow for the kind of layering approach that we have with go-ipld-prime + ADLs. As per https://github.com/ipld/go-car/pull/304 we already have situations where we have enough distance between block decode and ADL interpret that we can't properly apply a type check. This gives us wriggle-room to pass the data through the generic go-ipld-prime "lens" paradigm and let it come out as UnixFS; even if in practice we tightly couple it when encoding the data and may impose an expectation when decoding, where possible.

But also, it'd be good to have at least the potential for separation so we can treat the UnixFS ADL as a full ADL and not a special-cased beast.

At the same time this PR updates the language of this section to current IPLD parlance.

Any stomach for nuance?

rvagg avatar Jun 27 '22 11:06 rvagg

As in #271 I would prefer that the ADL works as an ADL, so i would be in favor of this spec change.

willscott avatar Jun 27 '22 11:06 willscott

what else is needed to merge this?

willscott avatar Jul 21 '22 07:07 willscott