Reconsider the project with standardization of OCI Distribution Spec

[AkihiroSuda]

Originally I designed FILEgrain to be agnostic to distribution protocols, because the standardization of distribution was out of the scope of OCI's mission at that time. But the situation has changed now.

If vnd.oci.image.layer.v1.tar (not tar+gzip) blobs are pushed to a Docker/OCI registry (via Transfer-Encoding: gzip), and the registry supports HTTP Range Requests, it is not difficult to implement deduplication and lazy-pulling in arbitrary granularity, without introducing FILEgrain.

Apparently, no change is needed on the distribution spec.

However, as an optimization, we might be able to define some extension spec for fetching tar headers at once.

e.g.

HEAD /v2/foo/blobs/sha256:deadbeef

200 OK
Content-Type: application/vnd.oci.image.layer.v1.tar
LazyPull-Digest: sha256:cafebabe

GET /v2/foo/blobs/sha256:cafebabe

200 OK
Content-Type: application/vnd.lazypull.manifest

{
  {
    // can be continuity manifest, but using raw TarHdr bytes
    // might be beneficial for deduplication on registry-side.
    bytes TarHdr = 1;
    // clients can invoke HTTP Range Requests with this offset for lazy-pulling.
    int64 payloadOffsetInOriginalTar = 2;
  },
  {
    ...
  },
  ...
}

cc @stevvooe @dmcgowan @tonistiigi