OEM: Provide Proxmox images

[pothos]

This uses https://github.com/coreos/afterburn/pull/1023 and https://github.com/coreos/ignition/pull/1790 to provide Proxmox images.

This pulls in https://github.com/flatcar/bootengine/pull/91 and https://github.com/flatcar/init/pull/115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.

How to use

Follow-up: fix coreos-cloudinit as mentioned above, and better don't make it work through the mount unit trigger but explicitly

For now it's limited to testing until the upstream PRs get merged.

Testing done

Jenkins

• [x] Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
• [ ] Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

[github-actions[bot]]

Build action triggered: https://github.com/flatcar/scripts/actions/runs/10281015073

[arcln]

Hello, there is an existing Igntion PR that should work at coreos/ignition#1790. Could you include the Ignition PR into this one so we can do further testing?

[pothos]

Hello, there is an existing Igntion PR that should work at coreos/ignition#1790. Could you include the Ignition PR into this one so we can do further testing?

This needs to be reworked first and we have to drop the afterburn integration in Flatcar as the afterburn PR relies on cloud-init userdata and because Flatcar will run coreos-cloudinit anyway if it doesn't see Ignition user-data we would have it processed twice. For the network config that can be quite confusing because the afterburn network unit would only support a subset.

[pothos]

Triggered a new build with the Ignition PR included

[arcln]

@pothos hey, it looks like the build never started, can you take a look please ? (cc @plgueugnon)

[tormath1]

@pothos hey, it looks like the build never started, can you take a look please ? (cc @plgueugnon)

@arcln hey, pothos is AFK for now. I'll take over this PR. A CI build has been triggered :)

[tormath1]

@arcln here's the proxmox image if you want to try it: http://bincache.flatcar-linux.net/images/amd64/9999.0.101+kai-proxmox-support/flatcar_production_proxmoxve_image.img.bz2

[arcln]

Thank you, I tried the image and it didn't work because of a problem in the ignition PR (see https://github.com/coreos/ignition/pull/1790#issuecomment-2137855552). Can you please run a new build when the PR is fixed ? Thanks

[tormath1]

Thank you, I tried the image and it didn't work because of a problem in the ignition PR (see coreos/ignition#1790 (comment)). Can you please run a new build when the PR is fixed ? Thanks

@arcln ah I see, I already got this issue once. I started a new build with an updated patch: https://github.com/flatcar/scripts/pull/1783/files#diff-c65e2140094c80c4b4ce5c7c8494b6e4e72b78ef0692d236ffee1fb3932e11c5R157

[arcln]

great. it seems that the build needs approval once again @tormath1

[tormath1]

@arcln it builds directly on our Jenkins (github actions only build for qemu targets) - I'll let you know once it's done.

[tormath1]

@arcln image is available here: http://bincache.flatcar-linux.net/images/amd64/9999.0.102+kai-proxmox-support/flatcar_production_proxmoxve_image.img.bz2

[arcln]

after doing some testing it seems to work fine! hostname is set, ssh keys are written properly and the network config is applied. we will now focus on finishing the afterburn PR and we'll be waiting for the ignition PR to be upstreamed.

[tormath1]

after doing some testing it seems to work fine! hostname is set, ssh keys are written properly and the network config is applied. we will now focus on finishing the afterburn PR and we'll be waiting for the ignition PR to be upstreamed.

Awesome, thanks a lot for testing! Any chance to draft what you've done in the flatcar-website repo? To act as a beginning of documentation for users interested to try (in the same spirit as: https://github.com/flatcar/flatcar-website/pull/333)

[abuisine]

Awesome, thanks a lot for testing! Any chance to draft what you've done in the flatcar-website repo? To act as a beginning of documentation for users interested to try (in the same spirit as: flatcar/flatcar-website#333)

Here is a proposition : https://github.com/flatcar/flatcar-website/pull/337

[abuisine]

@arcln ah I see, I already got this issue once. I started a new build with an updated patch: https://github.com/flatcar/scripts/pull/1783/files#diff-c65e2140094c80c4b4ce5c7c8494b6e4e72b78ef0692d236ffee1fb3932e11c5R157

For the record the issue has been fixed 2 hours ago : https://github.com/coreos/ignition/pull/1790 To be tested :)

[tormath1]

Hello @arcln following this: https://github.com/flatcar/flatcar-website/pull/337#discussion_r1629115962, could you give a try to this image: http://bincache.flatcar-linux.net/images/amd64/9999.0.104+kai-proxmox-support/flatcar_production_proxmoxve_image.iso ?

[arcln]

So I tried the image, it is much easier to get running because I can just import it from proxmox gui instead of using command line etc.

The image boots fine, however afterburn does not seem to be invoked. Hostname and ssh keys are not set. By looking at the boot log, I can see Flatcar metadata agent was skipped because no trigger condition check were met

Is the OEM ID properly set on this image?

[tormath1]

So I tried the image, it is much easier to get running because I can just import it from proxmox gui instead of using command line etc.

The image boots fine, however afterburn does not seem to be invoked. Hostname and ssh keys are not set. By looking at the boot log, I can see Flatcar metadata agent was skipped because no trigger condition check were met

Is the OEM ID properly set on this image?

@arcln yes, I checked the OEM ID is not passed when booting ISO images. Before going further, I'm wondering if it makes sense to provide ISO images for Proxmox? From what I read in your documentation, you create a VM using the image but using an ISO it means that one would need to install Flatcar to a disk provided via the UI.

Which gives two options:

1. Boot with current Flatcar ISO and install Flatcar on disk with flatcar-install script (like we're doing for PXE booting)
2. Ship the qcow image and import it via proxmox CLI

In both cases we need to provide a qcow2 image.

[abuisine]

@arcln yes, I checked the OEM ID is not passed when booting ISO images. Before going further, I'm wondering if it makes sense to provide ISO images for Proxmox? From what I read in your documentation, you create a VM using the image but using an ISO it means that one would need to install Flatcar to a disk provided via the UI.

Which gives two options:

1. Boot with current Flatcar ISO and install Flatcar on disk with flatcar-install script (like we're doing for PXE booting)
2. Ship the qcow image and import it via proxmox CLI

In both cases we need to provide a qcow2 image.

qcow makes far more sense from our point of view as it is compatible with either vm templates in ProxmoxVE or terraform on the fly upload.

[tormath1]

@arcln yes, I checked the OEM ID is not passed when booting ISO images. Before going further, I'm wondering if it makes sense to provide ISO images for Proxmox? From what I read in your documentation, you create a VM using the image but using an ISO it means that one would need to install Flatcar to a disk provided via the UI. Which gives two options:

1. Boot with current Flatcar ISO and install Flatcar on disk with flatcar-install script (like we're doing for PXE booting)
2. Ship the qcow image and import it via proxmox CLI

In both cases we need to provide a qcow2 image.

qcow makes far more sense from our point of view as it is compatible with either vm templates in ProxmoxVE or terraform on the fly upload.

Then we're good on this side, we now just need for upstream PRs to be reviewed / merged. In the documentation we will mention the various option to install Flatcar on Proxmox:

1. As you mentioned, via Terraform
2. Via the graphical UI using the Flatcar ISO image + flatcar-install
3. Via the proxmox CLI to create the vm and attach an existing disk

[abuisine]

Hi here :) Just a quick notice to let everyone know that PRs on afterburn and ignition in order to natively support ProxmoxVE have both been merged 🚀 : https://github.com/coreos/afterburn/pull/1023 https://github.com/coreos/ignition/pull/1910

[tormath1]

Hi here :) Just a quick notice to let everyone know that PRs on afterburn and ignition in order to natively support ProxmoxVE have both been merged 🚀 : coreos/afterburn#1023 coreos/ignition#1910

@abuisine yeah, that's great - I'm following this carefully. Now waiting for Afterburn and Ignition to be released (https://github.com/coreos/afterburn/issues/1095)