[DO NOT MERGE] test composefs

[edsantiago]

Signed-off-by: Ed Santiago [email protected]

None

[rhatdan]

Thanks @edsantiago LGTM

[packit-as-a-service[bot]]

Ephemeral COPR build failed. @containers/packit-build please check.

[edsantiago]

Failures so far:

• Swagger:

mkcomposefs: Invalid integer 628

• int (all of them):

# podman [options] save -o /tmp/quay.io-libpod-cirros-latest.tar quay.io/libpod/cirros:latest
Copying blob sha256:b993cfcfd8fdc2731b308222fce8b5ce7a681780ab276b6a23e1b0fe3ce06bbb
Copying blob sha256:aa107a40759291c596cd432ca14cf15e781b91cf82121dc5a335eb734cafe1c4
Copying blob sha256:858d98ac489379d0c34c20811558091180c724d7d185693c3a601ba8c0a1cedf
Error: writing blob: archive/tar: write too long

• sys remote:

# podman-remote --url unix:///tmp/podman_tmp_ijkO run -d --name myweb -p 5054:80 -v /tmp/podman_bats.CpsvZW/myimage-D4TDFZjD.tar:/var/www/image.tar:Z -w /var/www quay.io/libpod/testimage:20240123 /bin/busybox-extras httpd -f -p 80
Trying to pull quay.io/libpod/testimage:20240123...
...
Writing manifest to image destination
Error: committing the finished image: \
     failed to put layer using a partial pull: \
          rename /var/lib/containers/storage/overlay/staging/2508350620 /var/lib/containers/storage/overlay/SHA/diff: no such file or directory

[Luap99]

Failures so far:

* [Swagger](https://cirrus-ci.com/task/5151327842467840):

mkcomposefs: Invalid integer 628

Tacked in https://github.com/containers/podman/issues/22422

Maybe file new issues for the other ones

[edsantiago]

mkcomposefs: Invalid integer 628

I'm like 90% sure that 628 is a valid integer, but I don't have enough fingers and toes to confirm.

[packit-as-a-service[bot]]

Cockpit tests failed for commit e2b064ca70249a5b869189d7faf61c9752c5f2ae. @martinpitt, @jelly, @mvollmer please check.

[packit-as-a-service[bot]]

Ephemeral COPR build failed. @containers/packit-build please check.

[packit-as-a-service[bot]]

Cockpit tests failed for commit 15dd82be1a476497189bc6f0d11cfc150975f901. @martinpitt, @jelly, @mvollmer please check.

[packit-as-a-service[bot]]

Ephemeral COPR build failed. @containers/packit-build please check.

[packit-as-a-service[bot]]

Cockpit tests failed for commit a5b2535ec68480dccc906727963afb157ccb006d. @martinpitt, @jelly, @mvollmer please check.

[packit-as-a-service[bot]]

Ephemeral COPR build failed. @containers/packit-build please check.

[packit-as-a-service[bot]]

Cockpit tests failed for commit b316e5a55aad0cbe3b23f532ff3ee7c4a1d4cfbc. @martinpitt, @jelly, @mvollmer please check.

[edsantiago]

@containers/podman-maintainers I'm finally confident that we can start testing with composefs. Now what? Proposal:

1. vendor in c-storage (should be safe to do independently of this PR or anything to do with composefs)
2. Enable composefs in our f41 VM??? (See commit 1 in this PR)
3. Write new tests that exercise composefs???
4. Profit!

[packit-as-a-service[bot]]

Ephemeral COPR build failed. @containers/packit-build please check.

[edsantiago]

@containers/podman-maintainers remember when I said composefs was looking good? I was way way wrong. I had forgotten the "specialness" of e2e tests, where they need everything spelled out in command-line options. This is nowhere even close to working.

[edsantiago]

@containers/podman-maintainers I need a way to convert this containers.conf entry:

[storage.options]
pull_options = {enable_partial_images = "true", use_hard_links = "false", ostree_repos="", convert_images = "true"}

...to a command-line option suitable for e2e tests:

podman ... --storage-opt "something something pull_options something something"

[vrothberg]

@edsantiago, could you move it into a separate containers.conf and use podman --module containers.conf?

[edsantiago]

Just at first glance it looks tricky but I'll try.

[Luap99]

@edsantiago, could you move it into a separate containers.conf and use podman --module containers.conf?

This cannot work because these are storage.conf options not containers.conf options. And looking at the c/storage code there is no way to set pull_options from the cli at all AFAICT

[edsantiago]

@giuseppe this is one of the errors seen when only use_composefs=true is given but without the pull_etc options:

           # podman [options] save -o /var/tmp/quay.io-libpod-alpine_nginx-latest.tar quay.io/libpod/alpine_nginx:latest
           Copying blob sha256:4fc242d58285699eca05db3cc7c7122a2b8e014d9481f323bd9277baacfa0628
           Copying blob sha256:2edf982224acf5533abc9a2d08d571244086ccda36c2b8739634ec28aea8ee43
           Copying blob sha256:07e77d813a2b982b9d47f55345f052520ea49a080fb9e7ae4844e7c46ebe576c
           Error: reading blob sha256:2edf982224acf5533abc9a2d08d571244086ccda36c2b8739634ec28aea8ee43: open /tmp/CI_b8Hq/podman-e2e-2188729336/image-init/root/overlay/6a93b03276587c680cbf9e8f6eac345c1bc7607fab5a171a1c368376764caa52/merged/usr/share/ca-certificates/mozilla/NetLock_Arany_=Class_Gold=_Főtanúsítvány.crt: no such file or directory

[giuseppe]

thanks, I think this is just something broken in composefs.

I'll take a look

[edsantiago]

Failing rootless:

Error: configure storage: composefs is not supported in user namespaces

This is with c-storage @ main as of this morning.

[giuseppe]

Failing rootless:

Error: configure storage: composefs is not supported in user namespaces

This is with c-storage @ main as of this morning.

that is expected. composefs doesn't work with rootless as it needs to mount EROFS

[edsantiago]

Vendoring in c-image, storage, common @ main yields:

CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build \
         \
        -ldflags '-X github.com/containers/podman/v5/libpod/define.gitCommit=cb98e769e6be7dc22662889580e49b41ae64817b -X github.com/containers/podman/v5/libpod/define.buildInfo=1720019889 -X github.com/containers/podman/v5/libpod/config._installPrefix=/usr/local -X github.com/containers/podman/v5/libpod/config._etcDir=/etc -X github.com/containers/podman/v5/pkg/systemd/quadlet._binDir=/home/esm/bin/x86_64 -X github.com/containers/common/pkg/config.additionalHelperBinariesDir= ' \
        -tags "   systemd libsubid exclude_graphdriver_devicemapper seccomp " \
        -o bin/podman ./cmd/podman
# github.com/containers/podman/v5/pkg/api/handlers/compat
pkg/api/handlers/compat/containers.go:394:16: cannot use struct{NetworkMode string `json:",omitempty"`}{…} (value of type struct{NetworkMode string "json:\",omitempty\""}) as struct{NetworkMode string "json:\",omitempty\""; Annotations map[string]string "json:\",omitempty\""} value in struct literal
pkg/api/handlers/compat/networks.go:201:24: cannot use networkCreate.EnableIPv6 (variable of type *bool) as bool value in assignment
pkg/api/handlers/compat/system.go:60:16: cannot use struct{NetworkMode string `json:",omitempty"`}{} (value of type struct{NetworkMode string "json:\",omitempty\""}) as struct{NetworkMode string "json:\",omitempty\""; Annotations map[string]string "json:\",omitempty\""} value in struct literal
make: *** [Makefile:380: bin/podman] Error 1

cc @mheon

[Luap99]

That is the docker update: https://github.com/containers/podman/pull/23093

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago, giuseppe, Luap99

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [Luap99,edsantiago,giuseppe]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment