refactor `content/oci` with `fs.FS`

[shizhMSFT]

Refactor the content/oci package so that it is based on io/fs.FS.

With such refactoring, we can have something like

func NewFromFS(fsys fs.FS) *ReadOnlyStore
func NewStorageFromFS(fsys fs.FS) *ReadOnlyStorage

Powered by fs.FS, we can further leverage embed.FS and zip.Reader to read OCI layout from various file systems.

If we implement fs.FS for tarballs, we can directly read OCI layout from a tarball generated by docker buildx build --output=type=oci,dest=oci.tar.

$ tar xvf oci.tar
blobs/
blobs/sha256/
blobs/sha256/4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
blobs/sha256/643c10cea5330d08b25f85e99a1f9158efae4a2262125daf3e815a2fddb97ac7
blobs/sha256/8c356d8c5bb84bb001fc8e6c77474833d2b064d21de8610fa26a3b1da16a7921
blobs/sha256/a9c898c2b650367087223a0e5a12260ab88d309af8461fbfaac4ece4f4fc3c71
blobs/sha256/c2eb24acba5f5d4e6a231bff0b8e97050aa081be4dae2fad620896766617ae0e
blobs/sha256/df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139
blobs/sha256/f4612cfdf7a5b92df210c3ce1f0338b651b148023215f40b5b714f0444a0e823
index.json
manifest.json
oci-layout

Reference: Go 1.16 Release Notes

[shizhMSFT]

@Wwwsylvia You might be benefited from the code below for indexing a tarball for random access.

package main

import (
	"archive/tar"
	"fmt"
	"io"
	"log"
	"os"
	"strings"
)

func run() error {
	file, err := os.Open("test.tar")
	if err != nil {
		return err
	}
	defer file.Close()

	index := make(map[string]int64)
	r := tar.NewReader(file)
	for {
		header, err := r.Next()
		if err != nil {
			if err == io.EOF {
				break
			}
			return err
		}

		pos, err := file.Seek(0, io.SeekCurrent)
		if err != nil {
			return err
		}
		index[header.Name] = pos - 512 // 512 is the fixed header block size
	}

	for name, pos := range index {
		fmt.Println(name)
		fmt.Println(">>>", "pos:", pos)
		if _, err := file.Seek(pos, io.SeekStart); err != nil {
			return err
		}
		r = tar.NewReader(file)
		header, err := r.Next()
		if err != nil {
			return err
		}
		fmt.Println(">>>", "size:", header.Size)
		if header.Typeflag == tar.TypeReg && strings.HasSuffix(name, ".txt") {
			content, err := io.ReadAll(r)
			if err != nil {
				return err
			}
			formatted := strings.TrimSpace(string(content))
			fmt.Println(">>>", "content:", formatted)
		}
	}

	return nil
}

func main() {
	if err := run(); err != nil {
		log.Println(err)
		os.Exit(1)
	}
}

[qweeah]

Why the example has a docker layout file manifest.json?

[shizhMSFT]

Why the example has a docker layout file manifest.json?

That's a docker buildx specific file.