canonical
package-update-upgrade-install does not update when run with --frequency=always
This bug was originally filed in Launchpad as LP: #1785225
Launchpad details
affected_projects = [] assignee = None assignee_name = None date_closed = None date_created = 2018-08-03T11:12:56.923746+00:00 date_fix_committed = None date_fix_released = None id = 1785225 importance = medium is_complete = False lp_url = https://bugs.launchpad.net/cloud-init/+bug/1785225 milestone = None owner = siriobalmelli owner_name = Sirio Balmelli private = False status = triaged submitter = siriobalmelli submitter_name = Sirio Balmelli tags = [] duplicates = []
Launchpad user Sirio Balmelli(siriobalmelli) wrote on 2018-08-03T11:12:56.923746+00:00
- The following stanzas in /etc/cloud/cloud.cfg:
cloud_final_modules:
# always update+upgrade on boot
- [package-update-upgrade-install, always]
packages: # will run apt-update
- git
package_update: true
package_upgrade: true
package_reboot_if_required: true
- Result in the following log entry at boot-time:
helpers.py[DEBUG]: update-sources already ran (freq=once-per-instance)
- Running the module manually with:
sudo cloud-init single --name package-update-upgrade-install --frequency always
... also fails to call apt-get update
- Google gives no results.
- No similar bugs visible here.
- No additional configuration keys shown in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#package-update-upgrade-install.
- No relevant keys given in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#apt-configure.
- Nowhere is "update-sources" mentioned.
- Attempted guess at "update-sources" fails:
sudo cloud-init single --name update-sources --frequency always
- Expected behavior is that
apt-get updateis executed beforeapt-get installevery time the machine is booted. On my system the install looks like:
['eatmydata', 'apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'install', 'git']
- Alternate behavior would be a way to explicitly request that an update be performed. This was attempted/guessed-at with:
cloud_config_modules:
- [apt-configure, always]
But did not obtain an update. There seem to be no other applicable modules listed in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#
- Environment:
- Ubuntu 16.04.5 LTS
- /usr/bin/cloud-init 18.2
- /etc/cloud/cloud.cfg
cloud_init_modules: - seed_random - bootcmd - set_hostname - update_hostname - update_etc_hosts - ca-certs - rsyslog - users-groups - ssh
cloud_config_modules: - emit_upstart - ssh-import-id - locale - set-passwords - grub-dpkg - timezone - [apt-configure, always] - [runcmd, always] # doesn't actually EXECUTE the modules - that's left for scripts-user :P
cloud_final_modules: - landscape - lxd - ssh-authkey-fingerprints - final-message - power-state-change - [package-update-upgrade-install, always] - [scripts-user, always]
system_info: distro: ubuntu paths: cloud_dir: /var/lib/cloud/ templates_dir: /etc/cloud/templates/ upstart_dir: /etc/init/ ssh_svcname: ssh
disable_root: true
datasource_list: [NoCloud, None]
preserve_hostname: false hostname: {hostname} manage_etc_hosts: true
apt: preserve_sources_list: false primary: - arches: - amd64 - i386 - default uri: "http://archive.ubuntu.com/ubuntu/" search: - "http://ch.archive.ubuntu.com/ubuntu" - "http://us.archive.ubuntu.com/ubuntu" search_dns: true security: - arches: - amd64 - i386 - default uri: "http://security.ubuntu.com/ubuntu/" search_dns: true # (custom PPAs elided) sources_list: | deb http://archive.ubuntu.com/ubuntu/ xenial main restricted deb-src http://archive.ubuntu.com/ubuntu/ xenial main restricted deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted deb http://archive.ubuntu.com/ubuntu/ xenial universe deb-src http://archive.ubuntu.com/ubuntu/ xenial universe deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates universe deb http://archive.ubuntu.com/ubuntu/ xenial multiverse deb-src http://archive.ubuntu.com/ubuntu/ xenial multiverse deb http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse deb http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse deb http://security.ubuntu.com/ubuntu/ xenial-security main restricted deb-src http://security.ubuntu.com/ubuntu/ xenial-security main restricted deb http://security.ubuntu.com/ubuntu/ xenial-security universe deb-src http://security.ubuntu.com/ubuntu/ xenial-security universe deb http://security.ubuntu.com/ubuntu/ xenial-security multiverse deb-src http://security.ubuntu.com/ubuntu/ xenial-security multiverse runcmd: - [echo, startup run $(date)]
packages: # SHOULD run apt-update; does not - git package_update: true package_upgrade: true package_reboot_if_required: true
Launchpad user Ryan Harper(raharper) wrote on 2019-07-19T17:46:56.002900+00:00
Hi,
Thanks for filing the bug.
It appears that the distro.update_package_sources() runs with hardcoded frequency=PER_INSTANCE.
def update_package_sources(self):
self._runner.run("update-sources", self.package_command,
["update"], freq=PER_INSTANCE)
Cloud-init could supply the package-update-upgrade frequency when calling this distro method if configured.
Hi,
I was trying to add a PPA and install packages from it but it seems such scenarios are made impossible by this issue because it's not possible to "apt update" afterwards.
My config is at https://gitlab.com/crypto-config/crypto-configuration/-/blob/main/lxd-profile if you wish to look at it.
I'm not familiar with the cloud-init code and I don't know if freq can handle that but I would expect there should be a variable that is set after apt update, and unset whenever there is an operation that touches sources.
Thanks @adrien-n for the update and reproducer steps here. I think something like the following is what we want in this case. we want to force apt update to run if previously something like custom apt sources were provided (which triggered that first apt update call and sets theupdate-sources semaphore). Here's a very basic diff that only fnuctionally handles debian.ubuntu, but we'd need to reflect update_package_sources call signatures to all distributions which subclass this method to ensure we don't traceback when freq param is provided.
diff --git a/cloudinit/config/cc_package_update_upgrade_install.py b/cloudinit/config/cc_package_update_upgrade_install.py
index a26e001d3..f62819e9d 100644
--- a/cloudinit/config/cc_package_update_upgrade_install.py
+++ b/cloudinit/config/cc_package_update_upgrade_install.py
@@ -17,7 +17,7 @@ from cloudinit.config import Config
from cloudinit.config.schema import MetaSchema, get_meta_doc
from cloudinit.distros import ALL_DISTROS
from cloudinit.log import flush_loggers
-from cloudinit.settings import PER_INSTANCE
+from cloudinit.settings import PER_ALWAYS, PER_INSTANCE
REBOOT_FILE = "/var/run/reboot-required"
REBOOT_CMD = ["/sbin/reboot"]
@@ -104,7 +104,13 @@ def handle(name: str, cfg: Config, cloud: Cloud, args: list) -> None:
errors = []
if update or upgrade:
try:
- cloud.distro.update_package_sources()
+ # We may want to only provide PER_ALWAYS if we also know
+ # update_packages was already performed due to user-data or system
+ # configuration in /etc/cloud/cloud.cfg.*/
+ kwargs = {}
+ if cloud.distro.runner.has_run("update-pacakges", PER_INSTANCE):
+ kwargs["freq"] = PER_ALWAYS
+ cloud.distro.update_package_sources(**kwargs)
except Exception as e:
util.logexc(LOG, "Package update failed")
errors.append(e)
diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
index 87390f634..85988db75 100644
--- a/cloudinit/distros/__init__.py
+++ b/cloudinit/distros/__init__.py
@@ -37,6 +37,7 @@ from cloudinit import (
importer,
net,
persistence,
+ settings,
ssh_util,
subp,
temp_utils,
@@ -322,10 +323,12 @@ class Distro(persistence.CloudInitPickleMixin, metaclass=abc.ABCMeta):
# managers.
raise NotImplementedError()
- def update_package_sources(self):
+ def update_package_sources(
+ self, freq: str = settings.PER_INSTANCE
+ ):
for manager in self.package_managers:
try:
- manager.update_package_sources()
+ manager.update_package_sources(freq=freq)
except Exception as e:
LOG.error(
"Failed to update package using %s: %s", manager.name, e
diff --git a/cloudinit/distros/package_management/apt.py b/cloudinit/distros/package_management/apt.py
index 3c4164cd2..378fb0888 100644
--- a/cloudinit/distros/package_management/apt.py
+++ b/cloudinit/distros/package_management/apt.py
@@ -105,12 +105,12 @@ class Apt(PackageManager):
apt_get_upgrade_subcommand=cfg.get("apt_get_upgrade_subcommand"),
)
- def update_package_sources(self):
+ def update_package_sources(self, freq: str = PER_INSTANCE):
self.runner.run(
"update-sources",
self.run_package_command,
["update"],
- freq=PER_INSTANCE,
+ freq=freq,
)
@functools.lru_cache(maxsize=1)
diff --git a/cloudinit/distros/package_management/package_manager.py b/cloudinit/distros/package_management/package_manager.py
index 864555f6a..381e9673b 100644
--- a/cloudinit/distros/package_management/package_manager.py
+++ b/cloudinit/distros/package_management/package_manager.py
@@ -18,7 +18,7 @@ class PackageManager(ABC):
return cls(runner)
@abstractmethod
- def update_package_sources(self):
+ def update_package_sources(self, freq: str):
...
@abstractmethod
diff --git a/cloudinit/distros/package_management/snap.py b/cloudinit/distros/package_management/snap.py
index 92eb1af8f..582cf29ab 100644
--- a/cloudinit/distros/package_management/snap.py
+++ b/cloudinit/distros/package_management/snap.py
@@ -1,6 +1,6 @@
# This file is part of cloud-init. See LICENSE file for license information.
import logging
-from typing import Iterable, List
+from typing import Iterable, List, Optional
from cloudinit import subp, util
from cloudinit.distros.package_management.package_manager import (
@@ -14,7 +14,7 @@ LOG = logging.getLogger(__name__)
class Snap(PackageManager):
name = "snap"
- def update_package_sources(self):
+ def update_package_sources(self, freq: Optional[str]):
pass
def install_packages(self, pkglist: Iterable) -> UninstalledPackages:
Add hacktoberfest label on this issue as I think it is reasonable for anyone to work this issue.
The 23.04 cloud images have now been pulled and we can't deploy our stuff with 23.10 until this has been fixed.
The Azure Ubuntu images suffer from having a predefined list of mirrors in /etc/cloud/cloud.cfg.d/90-azure.cfg which means that you can't provision these images with your own images and install packages properly. As this is only present in 23.10 currently I suspect this issue is going to be important with the release of the next Ubuntu LTS 24.04 as people will need to actively work around this for all their Azure Ubuntu installs.
To work around this issue you need to replace the package directive with a runcdm that contains apt-get -y update and then use apt-get install -y for all the packages you need.
Alternatively you need to run apt-get -y update in a runcmd before the packages directive.