gitian-builder
gitian-builder copied to clipboard
devrandom
VirtualBox + precise == target-bin/grab-packages.sh very slow
I'm having a problem building Bitcoin version 0.9 (using a precise VM in a VirtualBox VM with my Mac as host) that I didn't have building previous versions of Bitcoin (using lucid, also VirtualBox).
The "getting manifest" step is extremely slow because target-bin/grab-packages.sh: dpkg-query -W -f '${Package}\n' | xargs -n 50 apt-get install --reinstall -y -d ... is re-downloading every package.
I can workaround the problem by just commenting that line out, but the puzzle is why it didn't cause problems for me before; I've never run apt-cacher on my Mac. But I don't know hardly nuthin about Linux packaging or how apt-get install --reinstall decides whether or not it needs to re-download things.
Ditto; just tried doing the same on my Mac, using a Vagrant Precise Ubuntu box, and it seems to be reinstalling every single package.
The man page for apt-get on Precise says that the --reinstall flag is to "Re-Install packages that are already installed and at the newest version" which has no conditional statements that the reinstall only happens under certain circumstances. The -d flag is setting it to only download not reinstall, but that's still a large chunk of time to download them all. The question is why on other boxes that wasn't happening?
Ah, I think I found out why; in my particular VM, the "Gitian-Clean" snapshot was taken at a time when the /var/cache/apt/archives folder was empty. I restored the VM to that snapshot, logged in, and ran sudo dpkg-query -W -f '${Package}\n' | sudo xargs -n 50 apt-get install --reinstall -y -d to download the slew of *.deb files, and then updated the snapshot to that state.
@gavinandresen sorry I missed this report. As @MidnightLightning says, the usual issue there is a cleaned /var/cache/apt/archives.
Incidentally, is there perhaps a different way to ensure what versions of software was used in the build? Requiring all the download packages bloats the size of the VM. Since all the VMs used to create Gitian builds are Ubuntu, can just the exact version name of the apt package be unique enough?