Using yum in post-install script fails due to `releasever` is not set

[mhoeher]

I am not sure if this behavior is intended, however, I'd just like to describe what I observed today:

We have a script which - using mkosi creates images of CentOS 7 plus some additional internal packages and configs. Worked like a charm for years now!

However, it seems with recent version of the tool some internals changed. In our use case, we use a post-install script, which adds some custom configuration, registers an internal RPM repository and installs additional packages from that one. Unfortunately, the calls to yum install started to fail, with the following strange message:

Cannot find a valid baseurl for repo: base/$releasever/x86_64

So it seams that the releasever is not properly set anymore when the post-install script is run. I changed the script and explicitly added --releasever=7 to all yum calls, which fixes the issue for us (so, this is definitely not a critical thing). However, I think this still should work without explicitly specifying the version, right?

I prepared a minimal example that reproduces the issue:

mhoeher@localhost:~/mkosi-test> tree
.
├── build.sh
└── postinstall-script.sh

mhoeher@localhost:~/mkosi-test> cat build.sh 
sudo mkosi \
    -d centos \
    -r 7 \
    -t directory \
    --postinst-script postinstall-script.sh \
    -p yum \
    -p centos-release \
    --with-network \
    -o output-path \
    --force

mhoeher@localhost:~/mkosi-test> cat postinstall-script.sh 
#!/bin/bash

# Fails
yum install --nogpgcheck -y epel-release

# Works:
#yum install --nogpgcheck -y --releasever=7 epel-release

[behrmann]

Can you reconstruct the last mkosi version where this used to work as you expected? This would be helpful to bisect the issue.

[mhoeher]

Sorry, totally forgot to attach version information in the first place...

We are currently on v9 (using mkosi from the Fedora 33 repositories).

I just tried to pin-point the faulty version and went back until v5. I can remember that with this version (installed from the Fedora repos) it definitely used to work, because we had to patch the tool with the changes from ae485d2af3d9818a3bcecd01ef426227c6da2006 to get it working.

The interesting thing is... even with that version it is no longer working :disappointed: So for me it somehow like something else changed which causes the issue to pop up right now.

Please let me know if I can provide you with some additional information that might help to debug this further.

[DaanDeMeyer]

From a quick search, it seems the redhat-release package should be installed to make this work. The variable itself should come from the /etc/yum.conf file in the image. Can you check if the redhat-release package is installed or look at the contents of /etc/yum.conf? Maybe that gives us a hint as to why the releasever variable isn't being set.

[mhoeher]

I added the redhat-release package to the package list installed by mkosi in the first step - it didn't make any difference.

I also added a cat /etc/yum.conf to the post-install script. This is what gets printed:

[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release


#  This is the default, if you make this bigger yum won't see if the metadata
# is newer on the remote and so you'll "gain" the bandwidth of not having to
# download the new metadata and "pay" for it by yum not having correct
# information.
#  It is esp. important, to have correct metadata, for distributions like
# Fedora which don't keep old packages around. If you don't like this checking
# interupting your command line usage, it's much better to have something
# manually check the metadata once an hour (yum-updatesd will do this).
# metadata_expire=90m

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d

[DaanDeMeyer]

Try installing centos-release as well just to be sure

[mhoeher]

Tried this already (see the content of the build.sh script above) - it unfortunately does not make any difference.

[rezib]

I think I can reproduce the issue with Fedora 35 and mkosi 12. I think this is due to the empty RPM database at the end of the build, despite the release package being properly installed:

root@debian11:~# mkosi --distribution fedora --release 35 --format directory -p dnf --output test.img build
…
Fedora 35 - base                                                        42 MB/s |  66 MB     00:01    
Fedora 35 - updates                                                     23 MB/s |  25 MB     00:01    
Last metadata expiration check: 0:00:05 ago on Wed 02 Feb 2022 10:34:49 AM CET.
Dependencies resolved.
==================================================================================================
 Package                       Architecture       Version                    Repository       Size
==================================================================================================
Installing:
 dnf                           noarch             4.9.0-1.fc35               fedora          448 k
 fedora-release                noarch             35-36                      updates          12 k
 systemd                       x86_64             249.9-1.fc35               updates         4.0 M
Installing dependencies:
…
‣ Resulting image size is 178.1M.
root@debian11:~# mkosi --distribution fedora --release 35 --format directory -p dnf --output test.img shell
Spawning container test on /root/test.img.
Press ^] three times within 1s to kill container.
[root@test ~]# dnf update
Unable to detect release version (use '--releasever' to specify release version)
Fedora $releasever - x86_64                                            120 kB/s |  88 kB     00:00    
Errors during downloading metadata for repository 'fedora':
  - Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=x86_64 (IP: 185.141.165.254)
Error: Failed to download metadata for repo 'fedora': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=x86_64 (IP: 185.141.165.254)
[root@test ~]# rpm -qa
<nothing>

Actually dnf looks in the RPM db to detect the release:

https://github.com/rpm-software-management/dnf/blob/395541fbf8f87f81cdca7567f22be1182e55bea7/dnf/rpm/init.py#L29

In my case, forcing re-installation of fedora-release fixes it:

[root@test ~]# dnf install --releasever 35 fedora-release
…
[root@test ~]# rpm -qa
gpg-pubkey-9867c58f-601c49ca
fedora-release-identity-basic-35-36.noarch
fedora-gpg-keys-35-1.noarch
fedora-release-35-36.noarch
fedora-repos-35-1.noarch
fedora-release-common-35-36.noarch
[root@test ~]# dnf update
Last metadata expiration check: 0:00:24 ago on Wed Feb  2 10:38:24 2022.
Dependencies resolved.
Nothing to do.
Complete!

I tried using --clean-package-metadata=false but it doesn't change a thing.

[rezib]

I think it is fixed by #940 and c48cc8b.

[DaanDeMeyer]

Let's close this as we should have fixed all the RPM database related issues and install the -release packages by default now