snapshot merge --no-remove drops some udebs

[bitte-ein-bit]

Im trying to freeze repos in time. For that I create a single merged snapshot. As I need multiple versions of the same package I try to merge with --latest=false --no-remove. The merge succeeds but the installer is broken an missing packages. The udebs are in the /pool/main/l/linux but are not referenced in /dists/wheezy/main/debian-installer/binary-amd64/Packages. A diff shows 39 missing packages. Aptly is 0.9.5

The repositories:

aptly@server:/srv/aptly/public/dists/wheezy$ aptly mirror show wheezy-main
Name: wheezy-main
Archive Root URL: http://ftp.de.debian.org/debian/
Distribution: wheezy
Components: main
Architectures: amd64
Download Sources: no
Download .udebs: yes
Filter: Priority (required) | Priority (important) | Priority (standard) | mysql-server | openssh-server | apache2 | open-vm-tools | perl | grub-pc | ipxe | python-argparse | python-yaml | python-argh | python-pathtools | python-pkg-resources | busybox-initramfs | debconf-utils | exim4-daemon-custom | fcron | libapache2-mod-passenger | libaugeas-ruby | libaugeas-ruby1.8 | libaugeas-ruby1.9.1 | libc6 (= 2.13-38+deb7u6) | libjson-ruby | libopenssl-ruby | libruby (>= 1:1.9.3.4) | libshadow-ruby1.8 | linux-headers-686-pae | linux-headers-generic | perl (<< 5.12.3-7) | perl (>= 5.15.6) | rdoc | ruby | ruby-interpreter | ruby-json | ruby-shadow | udev (<< 146-1) | ufsutils-udeb | virt-what | git | bundler | pkg-config | ruby-dev
Filter With Deps: yes
Last update: 2015-09-02 10:03:40 CEST
Number of packages: 784

Information from release file:
Architectures: amd64 armel armhf i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 s390x sparc
Codename: wheezy
Components: main contrib non-free
Date: Sat, 25 Apr 2015 10:59:07 UTC
Description:  Debian 7.8 Released 10 January 2015

Label: Debian
Origin: Debian
Suite: oldstable
Version: 7.8
aptly@server:/srv/aptly/public/dists/wheezy$ aptly mirror show wheezy-updates
Name: wheezy-updates
Archive Root URL: http://ftp.de.debian.org/debian/
Distribution: wheezy-updates
Components: main
Architectures: amd64
Download Sources: no
Download .udebs: yes
Filter: Priority (required) | Priority (important) | Priority (standard) | mysql-server | openssh-server | apache2 | open-vm-tools | perl | grub-pc | ipxe | python-argparse | python-yaml | python-argh | python-pathtools | python-pkg-resources | busybox-initramfs | debconf-utils | exim4-daemon-custom | fcron | libapache2-mod-passenger | libaugeas-ruby | libaugeas-ruby1.8 | libaugeas-ruby1.9.1 | libc6 (= 2.13-38+deb7u6) | libjson-ruby | libopenssl-ruby | libruby (>= 1:1.9.3.4) | libshadow-ruby1.8 | linux-headers-686-pae | linux-headers-generic | perl (<< 5.12.3-7) | perl (>= 5.15.6) | rdoc | ruby | ruby-interpreter | ruby-json | ruby-shadow | udev (<< 146-1) | ufsutils-udeb | virt-what | git | bundler | pkg-config | ruby-dev
Filter With Deps: yes
Last update: 2015-09-02 10:03:41 CEST
Number of packages: 3

Information from release file:
Architectures: amd64 armel armhf i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 s390x sparc
Codename: wheezy-updates
Components: main contrib non-free
Date: Wed, 02 Sep 2015 02:46:15 UTC
Description:  Updated packages for Debian 7.0

Label: Debian
Origin: Debian
Suite: oldstable-updates
Valid-Until: Wed, 09 Sep 2015 02:46:15 UTC
aptly@server:/srv/aptly/public/dists/wheezy$ aptly mirror show wheezy-security
Name: wheezy-security
Archive Root URL: http://security.debian.org/
Distribution: wheezy/updates
Components: main
Architectures: amd64
Download Sources: no
Download .udebs: yes
Filter: Priority (required) | Priority (important) | Priority (standard) | mysql-server | openssh-server | apache2 | open-vm-tools | perl | grub-pc | ipxe | python-argparse | python-yaml | python-argh | python-pathtools | python-pkg-resources | busybox-initramfs | debconf-utils | exim4-daemon-custom | fcron | libapache2-mod-passenger | libaugeas-ruby | libaugeas-ruby1.8 | libaugeas-ruby1.9.1 | libc6 (= 2.13-38+deb7u6) | libjson-ruby | libopenssl-ruby | libruby (>= 1:1.9.3.4) | libshadow-ruby1.8 | linux-headers-686-pae | linux-headers-generic | perl (<< 5.12.3-7) | perl (>= 5.15.6) | rdoc | ruby | ruby-interpreter | ruby-json | ruby-shadow | udev (<< 146-1) | ufsutils-udeb | virt-what | git | bundler | pkg-config | ruby-dev
Filter With Deps: yes
Last update: 2015-09-02 10:03:42 CEST
Number of packages: 124

Information from release file:
Architectures: amd64 armel armhf i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 s390x sparc
Codename: wheezy
Components: updates/main updates/contrib updates/non-free
Date: Mon, 31 Aug 2015 10:41:32 UTC
Description:  Debian 7.0 Security Updates

Label: Debian-Security
Origin: Debian
Suite: oldstable
Valid-Until: Thu, 10 Sep 2015 10:41:32 UTC
Version: 7.0

Snapshot is merged with aptly snapshot merge -latest=false -no-remove $RELEASE-final-$DATE $RELEASE-main-$DATE $RELEASE-updates-$DATE

if merged with aptly snapshot merge -latest=false -no-remove=false $RELEASE-final-$DATE $RELEASE-main-$DATE $RELEASE-updates-$DATE it works, for the installer part, but I'm missing required older versions.

[smira]

@bitte-ein-bit could you please provide minimal set of steps to reproduce it?

[bitte-ein-bit]

Unfortunately not really. Roughly

• Create mirrors main and updates with filters as displayed above.
• Take snapshot of mirror
• Merge to single snapshot

All I remember is that I started creating the mirrors with the filters as displayed above. Then I merged them together for use with a netboot pxe install and noticed the install failed.

I tried merging with the second command to no avail.

A I could not solve it by myself. I removed the filters and run a full mirror since.

[smira]

So it could be mirror filter, not snapshot merge?

[bitte-ein-bit]

I don't think so. Depending on the way snapshot merge was called the result was different. See samples above. When I just snapshoted and published the main repo all went well.

One wouldn't boot the installer via pxe because of missing udebs. The files were not listed in the "index" file.

The other would boot but fail later during install because of missing older versions. There was a dependency issue which was not there when three individual snapshots where used after the system was installed from a single main repro. One packet to be installed depended on a version from main, while security already had an update. The first one was not available in the merge. Unfortunately I did not not what packets had issues...

I'm sorry I can't provide more specific information. Unfortunately I do not remember more details.

Am Mo, 25. Jan 2016, um 22:55, schrieb Andrey Smirnov:

So it could be mirror filter, not snapshot merge?

---

Reply to this email directly or view it on GitHub: https://github.com/smira/aptly/issues/295#issuecomment-174683660

[geosone]

A example related to this problem is the debain installer errors out about btrfs-tools-udeb and wenn lokking into the debug log it was unable to find libc6-udeb so it looks like the deb was missing in the Release files for the udeb. so it could be related to this problem.

[gittygoo]

i can confirm this is also happening to me and hence limiting my hability to pxeboot machines

[smira]

@gittygoo does it involve aptly snapshot merge for you as well?

[gittygoo]

@smira Yes i cant pxeboot machines if i use --no-remove. Works fine ifi use just --latest but then i dont have necessary old packages

[smira]

I see where this bug comes from: aptly in some places considers two packages as being "equal" (or suitable to be overridden) when arch, name and version matches ignoring "udeb" flag. Udeb and non-udeb packages should never match each other

[smira]

I don't know if there's any easy way to fix that, as this touches very core package ID aptly is using to index packages internally. This ID can't be changed easily without causing many side-effects.

[gittygoo]

damn :( much needed feature though...

[smira]

So to explain it a bit more, it's a bug on my side. When support for udebs was introduced, package ID (internal primary for packages in the DB) wasn't updated to handle that. So internal ID looks like:

name_version_arch_hash, where hash is hash sum of package files

With that ID, if .udeb package has same name, version and arch, it still has different ID from .deb package, but prefix of the ID is same name_version_arch. Some aptly operations don't try to load package info, they work on package IDs (this includes merge), so when they see same name_version_arch, but different hash, they think these are duplicate packages and one of them gets removed.

Proper fix is to include udeb flag into package ID, but that means that already existing udeb packages (with old IDs) would become orphaned. We had this massive issue with upgrade from 0.9.x to 1.0.x (at that time files hash got changed).

So I think the only viable option is to make merge lookup packages (get package info) when such "match" is encountered.

I will think about it a bit more.

[gittygoo]

Thanks for the explanation, i'm sure you can get it sorted. Thanks for such a great tool too

[geosone]

great there will be a fix for this problem some time yey