freebsd
BACKUP_LIBRARIES improvements
Recording a list of possible improvements recently discussed:
- (noted by @markjdb) The solver could learn about BACKUP_LIBRARIES and understand that shlib dependencies will not go away if they're to be backed up; this would have been a reasonable workaround for #2496, which was caused by libutil.so.9 -> 10 trying to drop pkg when the shlib dependency went away before it rebuilt.
- Making it a per-repo option (or override) would be nice so that one could ship a base configuration where it's enabled for pkgbase, but disabled for the ports repo where it's not as essential
- Rethinking the package they get added to, or making it configurable at a minimum (to allow, e.g., a separation of base and ports backed-up-libs). @bapt suggested that it'd be better to split each out into backup-libname packages instead so that autoremove can later purge ones that don't need to be around anymore; this would solve the same problem as making it configurable, since we don't generally do name + SOVERSION collisions between base and ports.
09:29 <@bapt> also the current mecanism for backup libraries can corrupt the backed up library ;)
09:29 <@bapt> this is also a fix
09:29 <@bapt> iirc there is no writetmp, unlink, rename mecanism ;)
Another problem that I found empirically: the BACKUP_LIBRARIES mechanism doesn't work with split package upgrades, where we deinstall the old version and install the new version as separate steps. In that case, we don't back up the libraries at all.
Another problem that I found empirically: the BACKUP_LIBRARIES mechanism doesn't work with split package upgrades, where we deinstall the old version and install the new version as separate steps. In that case, we don't back up the libraries at all.
I have a patch for this now, but while testing it I found another bug: we back up libraries to the wrong path when pkg's rootdir isn't /. For instance, if I upgrade with pkg -r /mnt -o BACKUP_LIBRARIES=yes upgrade, pkg will put libraries in /mnt/mnt/usr/local/lib/compat/pkg. I will fix this too.
I have a patch for this now, but while testing it I found another bug: we back up libraries to the wrong path when pkg's rootdir isn't /. For instance, if I upgrade with
pkg -r /mnt -o BACKUP_LIBRARIES=yes upgrade, pkg will put libraries in /mnt/mnt/usr/local/lib/compat/pkg. I will fix this too.
This one is annoying to fix without breaking backward compatibility. If pkg is invoked with -r, the root is automatically prepended to the default BACKUP_LIBRARIES_PATH, so pkg -r /mnt upgrade uses BACKUP_LIBRARIES_PATH=/mnt/usr/local/lib/compat/pkg. The backup code makes the path relative to the rootfd, so in this example we get /mnt/mnt/usr/local/lib/compat/pkg, which is obviously wrong. But, if the user specified pkg -r /mnt -o BACKUP_LIBRARY_PATH=/backups, then we'll use /mnt/backups, which makes some sense.
We can stop automatically prepending the root dir to the default path, but this is unlike how all similar config options work, and it creates some problems elsewhere.
Yeah the chroot option is sometime inconsistent, we should stop prepending rootdir to default path and check what other inconsistency it creates and fix them imho
https://github.com/freebsd/pkg/issues/2492 is also somewhat relevant here (at least insofar as pkgbase is concerned).
I've been trying to create a minimal test case to demonstrate the solver problem. I was under the impression that if package foo-1 provides libfoo.so.1, and bar-1 contains libbar.so.1 which depends on libfoo.so.1, and an upgrade foo 1->2 bumps libfoo.so.1 to libfoo.so.2, then pkg would uninstall bar-1 as part of the upgrade.
However, when I actually try this, we upgrade foo-1 to foo-2 without uninstalling bar-1: https://github.com/markjdb/pkg/commit/009998b187a0050d3d5c3f191b69d0e561452339
From reading the solver code a bit, I'm a bit surprised by this behaviour.
I've been trying to create a minimal test case to demonstrate the solver problem. I was under the impression that if package foo-1 provides libfoo.so.1, and bar-1 contains libbar.so.1 which depends on libfoo.so.1, and an upgrade foo 1->2 bumps libfoo.so.1 to libfoo.so.2, then pkg would uninstall bar-1 as part of the upgrade.
However, when I actually try this, we upgrade foo-1 to foo-2 without uninstalling bar-1: markjdb@009998b
From reading the solver code a bit, I'm a bit surprised by this behaviour.
Hum, so if I amend the test such that it also upgrades bar-1 to bar-2 but bar-2 still depends on libfoo.so.1, then the upgrade will indeed remove bar. What exactly is the desired behaviour here?
excellent question, tbh I don't really know, for now, ytes I expect it to remove bar-2, some people might except the solver to just die. without interractivity, it is hard to make sane decision which please everyone.
Another problem that I found empirically: the BACKUP_LIBRARIES mechanism doesn't work with split package upgrades, where we deinstall the old version and install the new version as separate steps. In that case, we don't back up the libraries at all.
That is probably the explanation for some past examples where BACKUP_LIBRARIES did not cover everything that I'd expected.
Here's a problem with BACKUP_LIBRARIES+pkgbase. Suppose we bump libfoo.so.1 to libfoo.so.2, and keep libfoo.so.1 in /usr/lib/backup or whatever. Suppose FreeBSD then releases a security advisory which applies to both versions of the library. Only libfoo.so.2 will be updated and a vulnerable libfoo.so.1 will quietly be left on the system. We can't realistically patch libfoo.so.1, so the "obvious" thing is to remove it, but how?