apt-offline icon indicating copy to clipboard operation
apt-offline copied to clipboard

apt-offline install does not accept custom public key path as in deb [signed-by=...] ...

Open McTrk opened this issue 1 year ago • 4 comments

me@z620:~/devel/work/apt-offline$ sudo apt-offline set --update ud.sig
Gathering details needed for 'update' operation
me@z620:~/devel/work/apt-offline$ sudo apt-offline get ud.sig --bundle ud.zip

Fetching APT Data

Downloading http://packages.microsoft.com/repos/code/dists/stable/Release.gpg                                                             
http://packages.microsoft.com/repos/code/dists/stable/Release.gpg done                                                             
...
Downloading http://archive.ubuntu.com/ubuntu/dists/noble-backports/multiverse/cnf/Commands-all.xz                                                             
...
Downloading https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.gpg                                                             
https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.gpg done                                                             
...
Downloading https://pkgs.k8s.io/core:/stable:/v1.28/deb/Contents-all.xz                                                             
Downloading http://downloads.linux.hpe.com/SDR/repo/mcp/dists/noble/current/Release.gpg                                                             
...
Downloading https://download.docker.com/linux/ubuntu/dists/jammy/stable/cnf/Commands-all.xz                                                             
1061 / 1061 items: [##############################] 100.0% of 70 MiB
Downloaded data to /home/me/devel/work/apt-offline/ud.zip
me@z620:~/devel/work/apt-offline$ ls -l
total 71436
-rw-r--r-- 1 root root    57214 Jun  3 19:46 ud.sig
-rw-r--r-- 1 root root 73088713 Jun  3 19:49 ud.zip
me@z620:~/devel/work/apt-offline$ sudo apt-offline install ud.zip 
Proceeding with installation
gpgv: Signature made Fri 31 May 2024 07:38:25 AM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Fri 31 May 2024 07:38:25 AM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 06:51:30 PM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 06:51:30 PM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Thu 25 Apr 2024 11:11:21 AM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Thu 25 Apr 2024 11:11:21 AM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Tue 28 May 2024 10:07:10 AM EDT
gpgv:                using RSA key 7EA0A9C3F273FCD8
gpgv: Can't check signature: No public key
ERROR: /tmp/tmpbplh675u/download.docker.com_linux_ubuntu_dists_jammy_InRelease bad signature.  Not syncing because in strict mode.
gpgv: Signature made Tue 28 May 2024 10:07:10 AM EDT
gpgv:                using RSA key 7EA0A9C3F273FCD8
gpgv: Can't check signature: No public key
ERROR: /tmp/tmpbplh675u/download.docker.com_linux_ubuntu_dists_jammy_Release.gpg bad signature. Not syncing because in strict mode.
gpgv: Signature made Thu 09 May 2024 04:15:37 AM EDT
gpgv:                using RSA key 57446EFDE098E5C934B69C7DC208ADDE26C2B797
gpgv: Can't check signature: No public key
ERROR: /tmp/tmpbplh675u/downloads.linux.hpe.com_SDR_repo_mcp_dists_noble_current_Release.gpg bad signature. Not syncing because in strict mode.
gpgv: Signature made Tue 18 Jul 2023 03:04:24 PM EDT
gpgv:                using RSA key C95B321B61E88C1809C4F759DDCAE044F796ECB0
gpgv: Can't check signature: No public key
ERROR: /tmp/tmpbplh675u/nvidia.github.io_libnvidia-container_stable_ubuntu18.04_amd64_InRelease bad signature.  Not syncing because in strict mode.
gpgv: Signature made Thu 15 Apr 2021 11:01:52 PM EDT
gpgv:                using RSA key F9FDA6BED73CDC22
gpgv: Good signature from "Canonical Archive Automatic Signing Key <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 11:42:39 AM EDT
gpgv:                using RSA key EB3E94ADBE1229CF
gpgv: Good signature from "Microsoft (Release signing) <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 11:43:09 AM EDT
gpgv:                using RSA key EB3E94ADBE1229CF
gpgv: Good signature from "Microsoft (Release signing) <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 04:55:25 AM EDT
gpgv:                using RSA key EB3E94ADBE1229CF
gpgv: Good signature from "Microsoft (Release signing) <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 04:55:38 AM EDT
gpgv:                using RSA key EB3E94ADBE1229CF
gpgv: Good signature from "Microsoft (Release signing) <[email protected]>"
gpgv: Signature made Tue 14 May 2024 06:01:40 PM EDT
gpgv:                using RSA key 234654DA9A296436
gpgv: Can't check signature: No public key
ERROR: /tmp/tmpbplh675u/pkgs.k8s.io_core:_stable:_v1.28_deb_InRelease bad signature.  Not syncing because in strict mode.
gpgv: Signature made Tue 14 May 2024 06:01:40 PM EDT
gpgv:                using RSA key 234654DA9A296436
gpgv: Can't check signature: No public key
ERROR: /tmp/tmpbplh675u/pkgs.k8s.io_core:_stable:_v1.28_deb_Release.gpg bad signature. Not syncing because in strict mode.
gpgv: Signature made Mon 03 Jun 2024 04:18:18 PM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
gpgv: Signature made Mon 03 Jun 2024 04:18:18 PM EDT
gpgv:                using RSA key F6ECB3762474EDA9D21B7022871920D1991BC93C
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2018) <[email protected]>"
archive.ubuntu.com_ubuntu_dists_noble-backports_InRelease synced.
archive.ubuntu.com_ubuntu_dists_noble-backports_Release.gpg synced.
...
security.ubuntu.com_ubuntu_dists_noble-security_universe_source_Sources.xz synced.
security.ubuntu.com_ubuntu_dists_noble-security_universe_source_Sources.xz synced.
me@z620:~/devel/work/apt-offline$ 

Problem: Files downloaded from sources with the so-called "bad signature" (namely, those from download.docker.com, downloads.linux.hpe.com, nvidia.github.io, pkgs.k8s.io) have not been synced. These are sources with a custom signed-by field, such as deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /. Such custom-located signatures are recognized by apt-get:

me@z620:~/devel/work/apt-offline$ sudo apt-get update
Hit:2 https://packages.microsoft.com/ubuntu/22.04/prod jammy InRelease                                                                                                                       
Get:3 https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/amd64  InRelease [1,484 B]                                                                                             
Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease                                                                                                                             
Hit:5 https://download.docker.com/linux/ubuntu jammy InRelease                                                                                                                               
Get:1 https://packages.microsoft.com/repos/code stable InRelease [3,590 B]                                                                                                                   
Hit:7 http://archive.ubuntu.com/ubuntu noble InRelease                                                                                                                                       
Hit:8 http://oem.archive.canonical.com/updates focal-qemu InRelease                                   
Hit:9 http://archive.ubuntu.com/ubuntu noble-updates InRelease                                        
Ign:10 http://downloads.linux.hpe.com/SDR/repo/mcp noble/current InRelease             
Hit:11 http://archive.ubuntu.com/ubuntu noble-backports InRelease                      
Hit:6 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.28/deb  InRelease
Hit:12 http://downloads.linux.hpe.com/SDR/repo/mcp noble/current Release         
Fetched 5,074 B in 1s (3,680 B/s)
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://packages.microsoft.com/ubuntu/22.04/prod jammy InRelease' doesn't support architecture 'i386'
N: Skipping acquire of configured file 'stable/binary-i386/Packages' as repository 'https://download.docker.com/linux/ubuntu jammy InRelease' doesn't support architecture 'i386'
N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'http://packages.microsoft.com/repos/code stable InRelease' doesn't support architecture 'i386'
N: Missing Signed-By in the sources.list(5) entry for 'http://oem.archive.canonical.com/updates'
N: Missing Signed-By in the sources.list(5) entry for 'http://packages.microsoft.com/repos/code'
me@z620:~/devel/work/apt-offline$ 

Desired behavior should match the output from apt-get update, above. Version detail:

me@z620:~/devel/work/apt-offline$ apt-offline -v
1.8.5
me@z620:~/devel/work/apt-offline$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 24.04 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
me@z620:~/devel/work/apt-offline$ 

McTrk avatar Jun 04 '24 00:06 McTrk

This has just been fixed in master with 8cd98befe3860fe09a8d4badf97a25ecb26203b4 Could you please test and report ?

rickysarraf avatar Jun 04 '24 09:06 rickysarraf

Sorry for jumping on an old thread but I'm not sure that the fix in the master branch will address the issue fully. My interpretation of the changes is that it will use APT's config to find the default locations (which makes sense) and now adds a program parameter to add a custom location. I believe there's value in the new option but I don't believe it addresses a security issue that the APT developers are trying to resolve.

Going forward, third-party keys are now supposed to be placed in /usr/share/keyrings/ and each source/repo requiring a non-default/system key should be using the 'Signed-By' tag which points to the repo's corresponding key. This is necessary to prevent a key from being used to maliciously authorize any/other APT sources. I'm currently overhauling my company's systems to avoid this same issue. I also believe this is why apt-key is being deprecated.

Ideally, apt-offline would use the 'Signed-By' prefixes in the sources files (when found) to verify that source rather than relying on a global list of keys. If you're interested, I came across this page which has a decent summary of the problem and how to workaround it: https://www.linuxuprising.com/2021/01/apt-key-is-deprecated-how-to-add.html

Finally, I just want to say that I have found your efforts to be very help in maintaining offline (and seldom online) systems so, thank you.

ForeverACE avatar Jan 23 '25 20:01 ForeverACE

Bumping this @rickysarraf - seeing the same issue as @ForeverACE, where system-installed keyrings are not recognized by apt-offline.

I'm using apt-offline to make reproducible builds & upgrade bundles of airgapped VMs, so "as close to apt-get's behavior as possible" is desirable. I might have some budget to support work on this issue if it would help.

wearpants avatar Mar 14 '25 19:03 wearpants

I'm using apt-offlune on many of my systems to ensure that no system breakage occur.

I'll check on the new developments in apt land but thus far, while preparing for the Trixie release, I've not encountered any bug yet.

My time has gotten very limited and as such I struggle for exploratory topics. I'll check the mentioned links when I can.

Meanwhile, if you people are running into issues, please preferably provide a PR if possible. Otherwise, clear and concise steps to reproduce on a Debian system.

s3nt fr0m a $martph0ne, excuse typ0s

On Sat, 15 Mar 2025, 01:17 Pete Fein, @.***> wrote:

Bumping this @rickysarraf https://github.com/rickysarraf - seeing the same issue as @ForeverACE https://github.com/ForeverACE, where system-installed keyrings are not recognized by apt-offline.

I'm using apt-offline to make reproducible builds & upgrade bundles of airgapped VMs, so "as close to apt-get's behavior as possible" is desirable. I might have some budget to support work on this issue if it would help.

— Reply to this email directly, view it on GitHub https://github.com/rickysarraf/apt-offline/issues/232#issuecomment-2725608761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAP7SH2N7RCIAWNZZB22OL2UMW4HAVCNFSM6AAAAABVYF3RXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRVGYYDQNZWGE . You are receiving this because you were mentioned.Message ID: @.***> [image: wearpants]wearpants left a comment (rickysarraf/apt-offline#232) https://github.com/rickysarraf/apt-offline/issues/232#issuecomment-2725608761

Bumping this @rickysarraf https://github.com/rickysarraf - seeing the same issue as @ForeverACE https://github.com/ForeverACE, where system-installed keyrings are not recognized by apt-offline.

I'm using apt-offline to make reproducible builds & upgrade bundles of airgapped VMs, so "as close to apt-get's behavior as possible" is desirable. I might have some budget to support work on this issue if it would help.

— Reply to this email directly, view it on GitHub https://github.com/rickysarraf/apt-offline/issues/232#issuecomment-2725608761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAP7SH2N7RCIAWNZZB22OL2UMW4HAVCNFSM6AAAAABVYF3RXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRVGYYDQNZWGE . You are receiving this because you were mentioned.Message ID: @.***>

rickysarraf avatar Mar 16 '25 15:03 rickysarraf