imagebuildah: make traditional volume handling not the default

[nalind]

What type of PR is this?

/kind api-change

What this PR does / why we need it:

The traditional handling of volumes (where they're "frozen" and can only be modified by ADD or COPY, which requires that we cache their contents and save/restore them before/after RUN instructions) is not what newer BuildKit-based builds do, and it tends to confuse people, particularly when the volume is inherited from a base image that they're using for their own build.

This PR makes that an optional behavior which can be enabled with a build option. When the previous behavior is enabled, it corrects the handling of those inherited volumes to match the way it should have worked.

How to verify it

New and expanded conformance tests!

Which issue(s) this PR fixes:

Fixes #4247 Fixes #4407

Special notes for your reviewer:

I expect this to be more of interest to API consumers who want to control when this behavior changes.

Does this PR introduce a user-facing change?

When building an image using `buildah build`, the contents of directories marked as volumes (defined by VOLUME instructions), whether as part of the build, or inherited from a base image, will no longer be saved before, and restored after, processing RUN instructions.  The `--compat-volumes` flag can be used to re-enable this behavior.

[packit-as-a-service[bot]]

Ephemeral COPR build failed. @containers/packit-build please check.

[rhatdan]

Are you now compatible with docker/buildkit by default?

[nalind]

That is the goal, once all the tests pass.

[nalind]

The TestConformance/chown-volume test failures in the "Debian Conformance w/ vfs" context are fixed by https://github.com/containers/storage/pull/1962 and https://github.com/containers/storage/pull/1968.

[rhatdan]

Can you update the vendor of containers/storage?

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: flouthoc, nalind

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [flouthoc,nalind]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[rhatdan]

Tests still failing?

[flouthoc]

I see again mismatch of mtime or is there somthing more ? @nalind Should we instrument conformance test to ignore checking mtime ?

[nalind]

The TestConformance/chown-volume test failures in the "Debian Conformance w/ vfs" context are fixed by containers/storage#1962 and containers/storage#1968.

[nalind]

The TestConformance/chown-volume test failures in the "Debian Conformance w/ vfs" context are fixed by containers/storage#1962 and containers/storage#1968.

https://github.com/containers/buildah/pull/5585 brought those in. Rebased.

[rhatdan]

/lgtm