overlay: lock staging directories

[giuseppe]

lock any staging directory while it is being used so that another process cannot delete it.

Now the Cleanup() function deletes only the staging directories that are not locked by any other user.

Closes: https://github.com/containers/storage/issues/1915

Signed-off-by: Giuseppe Scrivano [email protected]

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: giuseppe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [giuseppe]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[edsantiago]

I vendored in your PR and ran podman tests. tons of failures, all in seccomp, all timeouts. Too many failures to be coincidence, I think.

[giuseppe]

I vendored in your PR and ran podman tests. tons of failures, all in seccomp, all timeouts. Too many failures to be coincidence, I think.

sorry for wasting your time on this.

I've updated the PR and ran the Podman tests: https://github.com/containers/podman/pull/22573 which are green now

[edsantiago]

Thanks! podman + composefs test now in progress, https://github.com/containers/podman/pull/22425

[edsantiago]

Same thing: timeouts in seccomp-policy. Common factor seems to be podman-local (not remote) + sqlite (not boltdb) + fedora (not debian).

I also vendored c-common, so maybe the bug is there?

[edsantiago]

Confirmed.

# cat /etc/containers/storage.conf 
[storage]
driver = "overlay"
runroot = "/run/containers/storage"
graphroot = "/var/lib/containers/storage"

[storage.options]
pull_options = {enable_partial_images = "true", use_hard_links = "false", ostree_repos="", convert_images = "true"}

[storage.options.overlay]
use_composefs = "true"

# go mod edit --replace github.com/containers/storage=github.com/giuseppe/storage@a5c63583a5f1ae8d9495716f2fd8f84755c64feb
...
# make
...

# bin/podman run --rm --seccomp-policy '' quay.io/libpod/alpine-with-seccomp:label ls
bin
dev
...
HANG!

[giuseppe]

ah no, this is another issue in the PR. I've fixed it now. It happens only when the registry rejects the range request, and I am going to debug now why quay does that with the quay.io/libpod/alpine-with-seccomp:labels image

[giuseppe]

ah no, this is another issue in the PR. I've fixed it now. It happens only when the registry rejects the range request, and I am going to debug now why quay does that with the quay.io/libpod/alpine-with-seccomp:labels image

opened a PR in c/image to address it: https://github.com/containers/image/pull/2391

[edsantiago]

Well, FWIW, with this and the c-i PR podman tests now pass

[giuseppe]

pushed a new version that addresses all the comments.

[rhatdan]

@mtrmac waiting on you.

[giuseppe]

@mohanboddu should the jira label create automatically the issue?

[giuseppe]

I'd like to get this into the next release, can we move it forward?

[giuseppe]

Just to unblock progress, not a careful review I’m afraid.

Thanks. Fixed the comments and pushed a new version

[edsantiago]

Successful CI run with composefs in #22425, with latest (this morning's) main. Also a successful run this weekend but that was before the giant vendor merge. Either way, tentative LGTM from the passing-tests front.

[rhatdan]

/lgtm

[giuseppe]

@mtrmac thanks for the review, I've opened a PR: https://github.com/containers/storage/pull/1926