gitian-builder
gitian-builder copied to clipboard
devrandom
make-base-vm should upgrade OS before writing base image
Another possibility that comes to mind would be to simply have a way of running the upgrade within the base image... Any reason that wouldn't be feasible?
Any progress on this? The Bitcoin Core 0.11.0 release cycle has started, with the first RC tagged, and if at all possible I'd like to have this fixed (and ideally a way to upgrade an existing base, but that's less important) before I start doing 3 builds for each RC and then for the release.
A simple workaround for now is to execute the following after an LXC gbuild:
cp -a --sparse=always target-trusty-amd64 base-trusty-amd64
(this is equivalent to running gbuild -i BTW, so it's not as clean... the build artifacts will remain, but that normally shouldn't affect future builds)
That's not a good solution, because it's the state of a VM after a whole
build, with some packages installed and files present. That's a problem --
I've tried using -i after builds targeting the same OS, and it didn't work
unless I on-targeted in and deleted everything manually. Even a gbuild
with a special gitian descriptor that does nothing might not work, since I
know a whole bunch of stuff gets done to the target after it's copied and
before the build-specific stuff works...
On Thu, Jun 4, 2015 at 10:14 PM, Dev Random [email protected] wrote:
A simple workaround for now is to execute the following after an LXC gbuild:
cp -a --sparse=always target-trusty-amd64 base-trusty-amd64
— Reply to this email directly or view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-109016337 .
Actually, what about doing that copy after a manual make-clean-vm plus a manual package upgrade?
On Thu, Jun 4, 2015 at 10:17 PM, Micha Bailey [email protected] wrote:
That's not a good solution, because it's the state of a VM after a whole build, with some packages installed and files present. That's a problem -- I've tried using -i after builds targeting the same OS, and it didn't work unless I
on-targeted in and deleted everything manually. Even a gbuild with a special gitian descriptor that does nothing might not work, since I know a whole bunch of stuff gets done to the target after it's copied and before the build-specific stuff works...On Thu, Jun 4, 2015 at 10:14 PM, Dev Random [email protected] wrote:
A simple workaround for now is to execute the following after an LXC gbuild:
cp -a --sparse=always target-trusty-amd64 base-trusty-amd64
— Reply to this email directly or view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-109016337 .
I was just going to suggest that. Here's the outline of the commands:
export SUITE=trusty
export ARCH=amd64
make-clean-vm
on-target -u root apt-get update
on-target -u root bash < target-bin/upgrade-system.sh
cp -a --sparse=always target-trusty-amd64 base-trusty-amd64
Is there anything in make-clean-vm (or rather, in
target-bin/bootstrap-fixup) that shouldn't be repeated?
On Thu, Jun 4, 2015 at 10:35 PM, Dev Random [email protected] wrote:
I was just going to suggest that. Here's the outline of the commands:
export SUITE=trusty export ARCH=amd64 make-clean-vm on-target -u root apt-get update on-target -u root bash < target-bin/upgrade-system.sh cp -a --sparse=always target-trusty-amd64 base-trusty-amd64
— Reply to this email directly or view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-109021479 .
That's a good point. I just pushed a commit to make it idempotent: https://github.com/devrandom/gitian-builder/commit/7219c490cccf332837753690c6e21bfc8041ed0c
I seem to be getting an error, something about already having added a user, but it appears to be harmless.
Right, I have a || true on that line to make it idempotent.
On 2015-06-05 07:22 AM, Michagogo wrote:
I seem to be getting an error, something about already having added a user, but it appears to be harmless.
— Reply to this email directly or view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-109307928.
devrandom / Miron
Hm. So now every time some packages are updated, gbuild fails unless I go through that process of upgrading the base container again.
I don't have it written down. I think it failed when running grab-packages or something like that, but I can't recreate it now until Ubuntu releases more updates.
On Wednesday, July 1, 2015, Dev Random [email protected] wrote:
What is the exact error?
— Reply to this email directly or view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-117359468 .
Agreed - a command or flag to upgrade the base image would be great. Or it could do the upgrading in the base image before making the copy.
Seems that debootstrap installs the earliest version of every package, so when the distribution grows older, more and more packages have to be updated every time, making the build take longer than necessary.
No -- it's still something that needs to be done on each build, and it takes a very long time. There's a workaround to manually update the base, but it's still something that shouldn't have to be done long-term.
On Monday, October 10, 2016, Joseph Bisch [email protected] wrote:
Can this be closed, because gbuild has an upgrade flag?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-252523799, or mute the thread https://github.com/notifications/unsubscribe-auth/ACcNnk-LU6JcFkC_RDVIdicc3J4TzTX7ks5qyYgsgaJpZM4EfsUO .
I agree. There is a workaround for this, but having a built-in command to upgrade the base image (instead of the copy) would be nice. This costs a lot of time every run, especially with LXC which starts off with a un-upgraded distribution.
(#120 would help with that last part. I meant to research and figure out how to get that to work - or even better, to use LXC's tools to create a container, rather than hacking one up - but unfortunately I stopped having much free time before I got around to it…)
On Tuesday, October 11, 2016, Wladimir J. van der Laan < [email protected]> wrote:
I agree. There is a workaround for this, but having a built-in command to upgrade the base image (instead of the copy) would be nice. This costs a lot of time, especially with LXC which starts off with a un-upgraded distribution.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/devrandom/gitian-builder/issues/91#issuecomment-252933405, or mute the thread https://github.com/notifications/unsubscribe-auth/ACcNnm0TXXKdoDU6CV58hIzz0kuxQouIks5qy509gaJpZM4EfsUO .
Among repetitive time-consuming actions I saw the locales built 2 times at least (I don't know why all those languages are needed in the first place, but that's another issue).
