podman
podman copied to clipboard
containers
DO NOT MERGE: buildah vendor treadmill
Followup to Cabal discussion 2022-04-07, re: letting us vendor in buildah on a moment's notice.
Instead of cron'ing this, let's see if this works: keep this PR open perpetually, with daily updates to get latest podman and buildah.
See individual commit messages for details.
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: edsantiago
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [edsantiago]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
![openshift-ci[bot] avatar](https://avatars.githubusercontent.com/in/91280?v=4 "Published by a pub.dev verified publisher"){kind=small}
The first commit is the important one. This one must be hand-crafted by a human, there is no possible way to automate it. This is the one that, in case of emergency vendor, can be grabbed into a real PR.
The second commit is a throwaway, ignore it. That's just automated fetch-the-latest-buildah.
bud test failed, remote only, because the new --identity-label=false option is not getting passed through. Probably an easy fix, but not this late in my day.
This is still a proof-of-concept PR, and my feeling so far is positive: it has quickly identified multiple problems in the buildah merge that are easy to solve iteratively. A nightly cron job, or a non-gating step in buildah CI, would be utterly useless because neither one would have the crucial first-commit. Without that, there's no way to make it past the first or second or third failure.
By a human, presumably me. There is actually no other way: this is not possible to solve via cron or PRs.
I am working on a script to automate as much of it as possible.
Followup, because I don't want anyone to think this is abandonware: it's not. I'm running my script at least twice a day, one of those times in the evening. One of the nice features I added is:
+---> Podman has bumped, but Buildah is unchanged. There's probably not much point to testing this.
So I'm choosing to save CI cycles by not re-pushing today. Should anyone need to re-vendor buildah tomorrow or this weekend (when I'm unavailable), just cherry-pick the first commit from this PR. It's still valid.
Yippee, script has caught a bug! https://github.com/containers/buildah/pull/3919 breaks when vendoring into podman:
+---> Running 'make' to confirm that podman builds cleanly...
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build \
-mod=vendor \
-ldflags '-X github.com/containers/podman/v4/libpod/define.gitCommit=3bf05d3113df29addd392032797f4fdf5dae56b7 -X github.com/containers/podman/v4/libpod/define.buildInfo=1650539506 -X github.com/containers/podman/v4/libpod/config._installPrefix=/usr/local -X github.com/containers/podman/v4/libpod/config._etcDir=/usr/local/etc -X github.com/containers/common/pkg/config.additionalHelperBinariesDir= ' \
-tags " selinux systemd exclude_graphdriver_devicemapper seccomp" \
-o bin/podman ./cmd/podman
# github.com/containers/podman/v4/libpod
libpod/events.go:18:3: cannot use r.config.Engine.EventsLogFileMaxSize (type "github.com/containers/common/pkg/config".eventsLogMaxSize) as type uint64 in field value
make: *** [Makefile:321: bin/podman] Error 2
buildah-vendor-treadmill-x: 'make' failed with new buildah. Cannot continue.
I could just force-push right now and let y'all see the breakage in CI, or I could just report it like I am right here. Or I could file an issue in containers/common since that's where the offending code seems to be?
Anyhow, @rhatdan et al, PTAL because this is not going to vendor into podman.
@edsantiago this is expected, we have to update c/common in podman https://github.com/containers/podman/pull/13945
What the heck, I pushed anyway in case someone wants to pull & play.
Apologies, @edsantiago!
No need for apologies, this is a useful test of the script. It also helps me understand, because I thought the script's vendor-new-buildah would also pull in new c-c. It does, v0.47.5-0.20220420095823-d822f53650b2, but your PR pulls in an even newer one, and I assume everything will work again as soon as your PR merges.
@edsantiago It pulls in the new one but this version is incompatible with podman that is why it fails at compile time. We have to tweak the code in podman before it will work.
Today's push is failing, seems to be related to the new etchosts code in buildah. I'm going to assume that this is Someone Else's Problem, likely one that they already know about, and it will be fixed in a day or two.
I have no idea why it would fail, however I am sure it will be fixed once #13918 is merged
@containers/podman-maintainers heads-up:
* bin/podman grew by 63360 bytes; max allowed is 51200.
I have temporarily bypassed in order to run tests. I am sending this as an early notice for your awareness. The culprit is https://github.com/containers/buildah/pull/3949 . (We don't have bloat checks in buildah, but it's safe to assume that this vendor update caused similar growth in the buildah binary).
See my comment in the recent containers-storage vendor PR. Yes, 45K was that PR, the rest must've been buildah. There is no way to know beyond that.
Latest buildah fails:
xref-helpmsgs-manpages: 'podman build --help' lists '--env', which is not in docs/source/markdown/podman-build.1.md
xref-helpmsgs-manpages: 'podman build --help' lists '--os-feature', which is not in docs/source/markdown/podman-build.1.md
xref-helpmsgs-manpages: 'podman build --help' lists '--os-version', which is not in docs/source/markdown/podman-build.1.md
xref-helpmsgs-manpages: 'podman image build --help' lists '--env', which is not in docs/source/markdown/podman-build.1.md
xref-helpmsgs-manpages: 'podman image build --help' lists '--os-feature', which is not in docs/source/markdown/podman-build.1.md
xref-helpmsgs-manpages: 'podman image build --help' lists '--os-version', which is not in docs/source/markdown/podman-build.1.md
Caught by the treadmill script, and normally this is something I'd easily fix, but I'm on PTO. If you're gonna be vendoring in buildah this week, I've got some of the work done for you here (use the treadmill script) but this new --env one, you're on your own, sorry.
Heads-up that this morning's treadmill run failed in a way I've never seen before:
+---> Vendoring in buildah...
go mod tidy
warning: ignoring symlink /home/esm/src/atomic/2018-02.podman/libpod/contrib/systemd/user
warning: ignoring symlink /home/esm/src/atomic/2018-02.podman/libpod/docs/build/man/links
warning: ignoring symlink /home/esm/src/atomic/2018-02.podman/libpod/src
go: downloading github.com/containers/common v0.48.1-0.20220608111710-dbecabbe82c9
go: downloading github.com/containers/buildah v1.26.1-0.20220609034125-a818b5527f02
go: finding module for package github.com/docker/libnetwork/ipamutils
go: downloading github.com/docker/libnetwork v0.5.6
go: found github.com/docker/libnetwork/ipamutils in github.com/docker/libnetwork v0.5.6
warning: ignoring symlink /home/esm/src/atomic/2018-02.podman/libpod/contrib/systemd/user
warning: ignoring symlink /home/esm/src/atomic/2018-02.podman/libpod/docs/build/man/links
warning: ignoring symlink /home/esm/src/atomic/2018-02.podman/libpod/src
go: finding module for package github.com/Sirupsen/logrus
go: downloading github.com/Sirupsen/logrus v1.8.1
go: found github.com/Sirupsen/logrus in github.com/Sirupsen/logrus v1.8.1
go: github.com/containers/podman/v4/pkg/specgenutil imports
github.com/docker/docker/opts imports
github.com/docker/libnetwork/ipamutils imports
github.com/docker/libnetwork/osl imports
github.com/Sirupsen/logrus: github.com/Sirupsen/[email protected]: parsing go.mod:
module declares its path as: github.com/sirupsen/logrus
but was required as: github.com/Sirupsen/logrus
make: *** [Makefile:255: vendor] Error 1
It worked fine yesterday, so this is something that changed between yesterday and right now. Thought y'all might want the early alert.
@containers/podman-maintainers heads up, treadmill is reporting that the next buildah vendor will trigger the bloat block.
- bin/podman grew by 56712 bytes; max allowed is 51200.
Yo people, bloat is getting heavier. It was 56712 just two days ago, now it's:
- bin/podman grew by 77216 bytes; max allowed is 51200.
bloat is now:
bin/podman grew by 83048 bytes; max allowed is 51200.
pkg/hooks was moved to c/common and is used by buildah so we need to remove it from podman and update the import paths. I have not looked at how big that package is but it should help at least a bit. cc @flouthoc
I'll remove it from
podman.
PR here: https://github.com/containers/podman/pull/14907
@containers/podman-maintainers, @flouthoc, all the int tests are failing in podman run from manifest list . This is a new failure. At first glance I don't see why treadmilling in a new buildah could cause this failure, but I haven't looked deeply. That is a new test (from last week) so it could just be a bad interaction between podman and buildah. PTAL, get this fixed early before a real buildah vendor is necessary.