fedora-copr
copr-sign fails with obs-sign 2.6.1
https://bodhi.fedoraproject.org/updates/FEDORA-2022-4779bd5643
we manually downgraded in production for now to obs-signd-2.5.10-4.20210907git5c32050.fc37.x86_64 (on keygen, server-side)
obs-signd-2.6.1-1.x86_64 works fine on copr-be (client side)
I think the issue was
[2022-11-30 21:23:08,316][WARNING][PID:3452181] Command '/bin/sign -4 -h sha256 -u tmz#[email protected] -r /var/lib/copr/public_html/results/tmz/git/srpm-builds/05076356/git-2.39.0-0.1.rc1.src.rpm' failed with: gpg-agent: 67108871 Bad secret key <GPG Agent>
gpg-agent: 67108871 Bad secret key <GPG Agent>
Somehow, the obs-signd package upgraded, so I downgraded it again and did
[root@copr-keygen ~][PROD]# dnf versionlock add obs-signd-2.5.10-4.20210907git5c32050.fc37.x86_64
$ dnf history info 26
Transaction ID : 26
Begin time : Wed 30 Nov 2022 08:30:12 PM UTC
Begin rpmdb : c98648fabfbf90727b5ef8f1704f878fff4fde5baf7fc5d16dee395612e39c01
End time : Wed 30 Nov 2022 08:30:59 PM UTC (47 seconds)
End rpmdb : c40d9be380733f05f3e950111a1d130d9da9c68d93257aadcf36ce3ae3dde4b7
User : fedora Cloud User <fedora>
Return-Code : Success
Releasever : 37
Command Line : -y update
Comment :
Packages Altered:
Install kernel-core-6.0.10-300.fc37.x86_64 @updates
Upgrade bluez-5.66-4.fc37.x86_64 @updates
Upgraded bluez-5.65-3.fc37.x86_64 @@System
Upgrade fontconfig-2.14.1-2.fc37.x86_64 @updates
Upgraded fontconfig-2.14.1-1.fc37.x86_64 @@System
Upgrade libxcrypt-4.4.33-3.fc37.x86_64 @updates
Upgraded libxcrypt-4.4.33-1.fc37.x86_64 @@System
Upgrade libxcrypt-compat-4.4.33-3.fc37.x86_64 @updates
Upgraded libxcrypt-compat-4.4.33-1.fc37.x86_64 @@System
Upgrade obs-signd-2.6.1-1.x86_64 @updates
Upgraded obs-signd-2.5.10-4.20210907git5c32050.fc37.x86_64 @@System
Upgrade perl-Getopt-Long-1:2.54-1.fc37.noarch @updates
Upgraded perl-Getopt-Long-1:2.52-489.fc37.noarch @@System
Upgrade vim-common-2:9.0.963-1.fc37.x86_64 @updates
Upgraded vim-common-2:9.0.915-1.fc37.x86_64 @@System
Upgrade vim-data-2:9.0.963-1.fc37.noarch @updates
Upgraded vim-data-2:9.0.915-1.fc37.noarch @@System
Upgrade vim-enhanced-2:9.0.963-1.fc37.x86_64 @updates
Upgraded vim-enhanced-2:9.0.915-1.fc37.x86_64 @@System
Upgrade vim-filesystem-2:9.0.963-1.fc37.noarch @updates
Upgraded vim-filesystem-2:9.0.915-1.fc37.noarch @@System
Upgrade vim-minimal-2:9.0.963-1.fc37.x86_64 @updates
Upgraded vim-minimal-2:9.0.915-1.fc37.x86_64 @@System
Might be related to: https://status.fedoraproject.org/
Update/reboots of servers
We will be upgrading and rebooting various servers. Services may be down during the outage window.
For more information and updates on the progress of this outage, see [ticket #11014](https://pagure.io/fedora-infrastructure/issue/11014)
after upgrade to 2.6.1, got in my k8s env
bash-5.2$ /bin/sign -u mywaaagh_admin#[email protected] -p
seteuid (for bindresvport): Operation not permitted
bash-5.2$ rpm -qf /usr/bin/sign
obs-signd-2.6.1-1.x86_64
after upgrade to 2.6.1, got in my k8s env
bash-5.2$ /bin/sign -u mywaaagh_admin#[email protected] -p seteuid (for bindresvport): Operation not permitted bash-5.2$ rpm -qf /usr/bin/sign obs-signd-2.6.1-1.x86_64
My bad, i miss the sign.conf item: allow-unprivileged-ports: true
After add a allow-unprivileged-ports: true, meet another error log
[2022-12-01 12:19:06,598][WARNING][PID:21] Command '/bin/sign -4 -h sha1 -u mywaaagh_admin#[email protected] -r /var/lib/copr/public_html/results/mywaaagh_admin/waaagh/srpm-builds/00000010/procenv-0.60-1.src.rpm' failed with: gpg-connect-agent: no running gpg-agent - starting '/usr/bin/gpg-agent'
gpg-connect-agent: failed to create temporary file '/var/lib/copr-keygen/.gnupg/.#lk0x00005581edda7820.copr-keygen-7dc84cb988-gmbhw.66': No such file or directory
gpg-connect-agent: can't connect to the gpg-agent: No such file or directory
gpg-connect-agent: error sending standard options: No agent running
gpg-connect-agent: no running gpg-agent - starting '/usr/bin/gpg-agent'
gpg-connect-agent: failed to create temporary file '/var/lib/copr-keygen/.gnupg/.#lk0x000055d58c1cc820.copr-keygen-7dc84cb988-gmbhw.69': No such file or directory
gpg-connect-agent: can't connect to the gpg-agent: No such file or directory
gpg-connect-agent: error sending standard options: No agent running
Seems /home/copr/.gnupg and /var/lib/copr-keygen/.gnupg/ dir was not created, which flow will create this directory?
The changelog says
- update to version 2.6.1
* support directly talking to the gpg agent
This feature was implemented in https://github.com/openSUSE/obs-sign/commit/8cb5bdfd0038cde00ffa2486a921a9eb40b96be7
I guess we started using the use_agent by default because we IMHO don't have gpg with the --files-are-digest parameter (one of our downstream patches). However, using use_agent: false doesn't fix our issue. But I tried setting
gnupghome: /var/lib/copr-keygen/gnupg/
and it seems to do the trick. I am now able to build and sign successfully on stage.
we IMHO don't have gpg with the --files-are-digest
But our gpg has --file-is-digest (downstream patch) and obs-signd is downstream patched too.
gnupghome: /var/lib/copr-keygen/gnupg/ and it seems to do the trick
That's awesome, but I fail to see how that helps actually. Can you elaborate?
Scratch everything ... a typo happened during the rebase https://src.fedoraproject.org/rpms/obs-signd/pull-request/5 With this, no config change is required.
.BR use_agent: " true|false" Make signd directly talk to the gpg-agent for signing instead of calling gpg. This is the default if the --files-are-digest option is not available in gpg.
Just for the record, even though it is not related to our issue. I read the code and the sentence isn't accurate. Default would mean, that if not specified explicitly in the config, it would fall back to the "does gpg have the --file-is-digest option?" check. In reality, if there is no --file-is-digest option available, whatever is set in the config is ignored.
@xsuchy built https://koji.fedoraproject.org/koji/buildinfo?buildID=2106812, I pushed it into infra-tags and installed it on Fedora Copr and Internal Copr production instances. Seems to work, closing.
In reality, if there is no --file-is-digest option available, whatever is set in the config is ignored.
Would you mind submitting a patch against upstream fixing the docs? I fear this will confuse us again in the future.