UKI: kairos-agent upgrade

[mudler]

Follow-up of #1151 #872

[Itxaka]

upgrade is now in main with normal upgrade command. We should just need to confirm that this works.

[mudler]

we also need documentation on how to build images that can be used for upgrades in the docs. Also manual steps works fine (e.g. build efi file, and use luet pack to pack a container image)

[Itxaka]

Imho this should be on os builder, as the workflow should piggyback into the iso creation and then you can output an image from the same workflow, including signing the artifacts inside. Should not be that much work

[jimmykarily]

Depends on this: https://github.com/kairos-io/kairos/issues/2171 (the intermediate artifact is needed). The upgrade process also needs documentation. I'm keeping this issue open until both are done.

[jimmykarily]

This is done but we need documentation + we need to try upgrading through Kubernetes (using a Plan).

[jimmykarily]

Before we test it with Kubernetes, let's wait until #2226 is done, because the upgrade process will change.

[mudler]

@jimmykarily I'm not sure if this card is entirely done yet - or we are covering the active.efi passive.efi and recovery.efi file changes in #2226 already?

[jimmykarily]

#2226 is supposed to handle the "rotation" of the various images/confs so that they change roles. This one, we kept open just for the "upgrade through Kubernetes" part.

[jimmykarily]

To clarify, this should work: https://kairos.io/docs/upgrade/kubernetes/

[jimmykarily]

Currently blocked by: https://github.com/kairos-io/kairos/issues/2275 (no Kubernetes, no upgrades through Kubernetes)

[jimmykarily]

No longer blocked. A fedora standard image built from kairos master can be used to debug this ticket.

[jimmykarily]

Applying the following yaml make uki upgrade work:

---
apiVersion: v1
kind: Secret
metadata:
  name: upgrade
  namespace: system-upgrade
type: Opaque
stringData:
  upgrade.sh: |
    #!/bin/sh
    mount --rbind host/dev /dev
    mount --rbind host/run /run
    kairos-agent upgrade --source oci:ttl.sh/fedora-standard-update-aksjhdgf:48h
---
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: os-upgrade
  namespace: system-upgrade
  labels:
    k3s-upgrade: server
spec:
  concurrency: 1
  # This is the version (tag) of the image to upgrade to.
  version: "38-core-amd64-generic-master"
  nodeSelector:
    matchExpressions:
      - {key: kubernetes.io/hostname, operator: Exists}
  serviceAccountName: system-upgrade
  secrets:
    - name: upgrade
      path: /host/run/system-upgrade/secrets/upgrade
  cordon: false
  drain:
    force: false
    disableEviction: true
  upgrade:
    image: quay.io/kairos/fedora
    command: ["chroot", "/host"]
    args: ["sh", "/run/system-upgrade/secrets/upgrade/upgrade.sh"]

So the idea here is that we use a compatible image (e.g. the image that was used to produce the uki upgrade image) and we manually upgrade pointing to the uki upgrade image using a script.

The example above is combining these 2:

• https://github.com/rancher/system-upgrade-controller/blob/master/examples/ubuntu/bionic.yaml
• https://github.com/kairos-io/packages/blob/821de2dded0c2f590b539261002c5d257fb8ea07/packages/system/suc-upgrade/suc-upgrade.sh#L13-L15