Apt can't authenticate package

[seocam]

• Puppet: 4.3.2
• Ruby: 2.1.5p273
• Distribution: Debian 8.3
• Module version: 0.4.0

How to reproduce (e.g Puppet code you use)

include nginx

What are you seeing

I'm receiving daily emails from my cron with the following content:

Cron <root@74f4a22b-web> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )

/etc/cron.daily/apt: WARNING: The following packages cannot be authenticated!

nginx

If I run apt-get update I get:

W: GPG error: http://nginx.org jessie Release: The following signatures were invalid: KEYEXPIRED 1471427554

What behaviour did you expect instead

Nginx packages should be correctly authenticated using GPG key provided by nginx.org

Any additional information you'd like to impart

If I download the key and add to the servers manually I stop to get the errors.

wget https://nginx.org/keys/nginx_signing.key
sudo apt-key add nginx_signing.key

[wyardley]

Seems similar to this issue (though of course different key): https://puppet.com/blog/updated-puppet-gpg-signing-key https://tickets.puppetlabs.com/browse/MODULES-3307 We also have a ticket for discussion of moving the signing key into the module itself, rather than pulling it directly from the nginx site (#651); this in and of itself might not fix this problem, but maybe we could add some code around it?

Not sure if anyone can think of a clean way to update the key without resorting to the hackery in the examples in the links above.

[emrtldrgn]

Is there a reason that you are just not using the ppa? apt::ppa { 'ppa:nginx/stable': ensure => present, }

[wyardley]

Not sure what that is; you're welcome to contribute a pull request if you'd like, and someone can review it. Keep in mind, though, that we're moving towards making the non-vendor repository non-default very shortly

I don't think that will address the issue when signing keys expire though, will it?

[kwisatz]

I'm seeing the same as OP, but the key doesn't seem to be outdated. Even when importing the nginx_signing.key manually, apt still refuses to authenticate them:

root@front1:~# apt-key list
/etc/apt/trusted.gpg
--------------------
pub   2048R/7BD9BF62 2011-08-19 [expires: 2024-06-14]
uid                  nginx signing key <[email protected]>

The key has clearly not expired, yet debian apt refuses to authenticate them:

puppet agent -t output:

The following extra packages will be installed:
  fontconfig-config fonts-dejavu-core libfontconfig1 libgd3 libjbig0
  libjpeg62-turbo libnginx-mod-http-auth-pam libnginx-mod-http-geoip
  libnginx-mod-http-image-filter libnginx-mod-http-xslt-filter
  libnginx-mod-mail libnginx-mod-stream libossp-uuid16 libtiff5 libvpx1
  libxpm4 libxslt1.1 nginx-common nginx-full
Suggested packages:
  libgd-tools uuid fcgiwrap nginx-doc ssl-cert
The following NEW packages will be installed:
  fontconfig-config fonts-dejavu-core libfontconfig1 libgd3 libjbig0
  libjpeg62-turbo libnginx-mod-http-auth-pam libnginx-mod-http-geoip
  libnginx-mod-http-image-filter libnginx-mod-http-xslt-filter
  libnginx-mod-mail libnginx-mod-stream libossp-uuid16 libtiff5 libvpx1
  libxpm4 libxslt1.1 nginx nginx-common nginx-full
0 upgraded, 20 newly installed, 0 to remove and 9 not upgraded.
Need to get 4295 kB of archives.
After this operation, 10.7 MB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  nginx-common libnginx-mod-http-auth-pam libnginx-mod-http-geoip
  libnginx-mod-http-image-filter libnginx-mod-http-xslt-filter
  libnginx-mod-mail libnginx-mod-stream nginx-full nginx
E: There are problems and -y was used without --force-yes

[kwisatz]

OK, here's what interesting: I'm using voxpupuli/puppet-php too, which sets up the dotdeb repository. This repository also has nginx. I manually deleted the dotdeb repo in sources.list.d, ran apt-get update, then installed nginx manually and since then running puppet agent -t works fine.

I suspect apt was trying to authenticate the dotdeb packages with the nginx key, or the other way around.

Here's the output of apt-cache policy for nginx:

root@front1:~# apt-cache policy nginx
nginx:
  Installed: 1.10.2-2~dotdeb+8.2
  Candidate: 1.10.2-2~dotdeb+8.2
  Version table:
 *** 1.10.2-2~dotdeb+8.2 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
        100 /var/lib/dpkg/status
     1.10.2-2~dotdeb+8.1 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.10.2-1~jessie 0
        500 http://nginx.org/packages/debian/ jessie/nginx amd64 Packages
     1.10.2-1~dotdeb+8.1 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.10.1-1~jessie 0
        500 http://nginx.org/packages/debian/ jessie/nginx amd64 Packages
     1.10.1-1~dotdeb+8.2 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.10.1-1~dotdeb+8.1 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.10.0-1~jessie 0
        500 http://nginx.org/packages/debian/ jessie/nginx amd64 Packages
     1.8.1-1~jessie 0
        500 http://nginx.org/packages/debian/ jessie/nginx amd64 Packages
     1.8.1-1~dotdeb+8.1 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.8.0-1~jessie 0
        500 http://nginx.org/packages/debian/ jessie/nginx amd64 Packages
     1.8.0-1~dotdeb+8.2 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.8.0-1~dotdeb+8.1 0
        500 http://packages.dotdeb.org/ jessie/all amd64 Packages
     1.6.2-5+deb8u4 0
        500 http://security.debian.org/ jessie/updates/main amd64 Packages
     1.6.2-5+deb8u2 0
        500 http://debian.mirror.root.lu/debian/ jessie/main amd64 Packages

Related to this: When nginx gets installed from dotdeb instead of nginx, the nginx user will not get created. Not sure how to handle this, by pinning the repos? Maybe the pinning process could become part of using { class ::nginx: } if manage_repo is set to true, so the following was done kind of automatically?

    ::apt::pin { 'nginx':
      priority        => 1001,
      release         => 'stable',
      originator      => 'nginx',
      component       => 'nginx',
      label           => 'nginx'
    }->
    class { '::nginx':
        manage_repo => true,
        package_source => 'nginx-stable'
    }

[wyardley]

@kwisatz:

I think that your analysis is right on. By default, the module will enable the repo (though we're considering changing this default in a future version), and ensure that nginx is present, but it won't explicitly ensure that it comes from a particular source. I think both yum and apt can support this with plugins, but the puppet providers themself cannot, other than ensuring a specific version.

In your version of Debian, looks like the official Nginx repo has a version that's older than the version from the dotdeb repo configured outside of the module, so that's why it's preferring that one.

You should be able to pin the version (and in that sense, also influence which repo is used) by using package_ensure and setting it to an explicit version instead of present or latest (see also comments in #962)

ps - I don't think this is the OP's problem though.

FWIW, it looks like Puppet's apt provider, at least in recent versions, will try to update an expired key.

[micah-uber]

Current version will not allow new installs... Submitted a PR #967 to move to using the PPA for this for debian.

[wyardley]

@micah-uber: can you post the error you're getting? As noted in #967, I don't think PPA will work for Debian.

[kali-brandwatch]

I had the same issue as the OP.

Inspecting the key list from apt I was seeing:

pub   2048R/7BD9BF62 2011-08-19 [expired: 2016-08-17]
uid                  nginx signing key <[email protected]>

The key had gone expired in Aug/2016.

Manually ran wget | apt-key add, now markes the updated key:

pub   2048R/7BD9BF62 2011-08-19 [expires: 2024-06-14]
uid                  nginx signing key <[email protected]>

Seems to me like the issue arises because the code on https://github.com/voxpupuli/puppet-nginx/blob/master/manifests/package/debian.pp#L39 is setting the GPG key, but since this key has been updated / expiry date extended while keeping the same hash (???? is that even possible?) therefore the module will not update / re-add the key (for it already exists) even if expired.

This leads to the scenario where the apt-get update will complain and upgrading to the latest version of the module (0.5.0) doesn't solve the issue.

[wyardley]

@kali-brandwatch From what I was able to glean, the apt provider is supposed to be smart about things like this, at least in recent versions (https://github.com/puppetlabs/puppetlabs-apt/blob/master/lib/puppet/provider/apt_key/apt_key.rb#L47-L49), though I don't think there's equivalent functionality in the Yum provider. But the general problem of a key getting updated when the user-id changes is not specific to this module or even to Puppet; see, for example, https://puppet.com/blog/updated-puppet-gpg-signing-key Even for puppet itself, the workaround was to add some extra stuff in your site code.

So far, the main problems that seem to exist are:

• signing key updated on systems where nginx is installed already
• users with additional repositories configured that also provide nginx

Neither of these are super easy to deal with directly in the module; for the first case, I don't think the module should try to handle it directly, however, updates to the apt / yum providers that it relies on might be appropriate.

In the second case, I don't know if there's an easy fix other than either pinning the package version using package_ensure, or using your package provider's (e.g., yum / apt) system for pinning a package to a particular repo. Because users won't reliably have this plugin installed, I don't think we can use it within the module, though it would make certain things easier.

[wyardley]

@kali-brandwatch ps - what Puppet version are you using? I took a quick look at at Puppet 3.8 and I don't think it has this version of the Apt provider. If you can reproduce this with Puppet 4.x, definitely seems worth opening a bug.

[kali-brandwatch]

@wyardley we are using puppet 4.3.1, and the apt module we are importing is puppetlabs/apt v2.1.1.

I indeed believe the issue should be probably handled on apt module side, and probably not on puppet-nginx side (or on our own wrappers, for what it's worth)

[kali-brandwatch]

For reference, I have checked the apt module and it seems that he code you referred to wasn't different in the tag 2.1.x which is the code we're using.

[wyardley]

@kali-brandwatch Thanks for the details. I spoke with someone, and sounds like puppetlabs-apt is a stand-alone module which is separate from the apt provider, so you might need to install this module as well to get the behavior I linked to [edit: though sounds like you might already have it anyway?].

I don't know whether Puppet will be interested in improving the behavior within the apt / yum providers directly, or what the exact category should be, but FWIW, I opened https://tickets.puppetlabs.com/browse/PUP-7005 -- feel free to put in more details or follow the ticket.