Seed not supported by DataSourceNoCloud when using a http-URL

[thannaske]

Bug report

While cloud-init properly finds a NoCloud datasource with instructions to pull the configuration from a http-source but fails to recognise it as compatible datasource seed. This leads to the cloud-init configuration to be not pulled and executed. The main error message is:

DataSourceNoCloud.py[DEBUG]: Seed from http://<domain>/api/service/v1/cloud-init/ not supported by DataSourceNoCloud [seed=None][dsmode=net]

So cloud-init properly detects it's running in a NoCloud environment, properly extracts the seed URL from the SMBIOS serial but then somehow thinks that's not a valid seed which is really confusing.

Steps to reproduce the problem

We create and run an ordinary libvirt KVM where Oracle Linux 9 is being installed on. The libvirt XML contains the following SMBIOS entry:

<sysinfo type='smbios'>
    <system>
        <entry name='serial'>ds=nocloud;s=http://<domain>/api/service/v1/cloud-init/</entry>
    </system>
</sysinfo>

I also tried the following serials to make sure I don't do anything majorly wrong:

ds=nocloud-net;s=http://<domain>/api/service/v1/cloud-init/
ds=nocloud;s=https://<domain>/api/service/v1/cloud-init/
ds=nocloud-net;s=https://<domain>/api/service/v1/cloud-init/
ds=nocloud;s=http://<domain>/api/service/v1/cloud-init/;h=test
ds=nocloud;s=https://<domain>/api/service/v1/cloud-init/;h=test
ds=nocloud-net;s=http://<domain>/api/service/v1/cloud-init/;h=test
ds=nocloud-net;s=https://<domain>/api/service/v1/cloud-init/;h=test

Then start the machine for the first time after the installation. The expectation would be that cloud-init properly detects that it's running in a NoCloud environment, pulls the seed URL from the SMBIOS and then proceeds to request the contents of user-data and vendor-data from that URL. However, the process immediately fails and there's no request at the web server you could observe at all.

Environment details

• Cloud-init version: 23.4-7.0.1.el9_4
• Operating System Distribution: Oracle Linux 9
• Cloud provider, platform or installer type: KVM/Libvirt

cloud-init logs

2024-06-07 09:29:08,560 - util.py[DEBUG]: Cloud-init v. 23.4-7.0.1.el9_4 running 'init-local' at Fri, 07 Jun 2024 09:29:08 +0000. Up 8.70 seconds.
2024-06-07 09:29:08,560 - main.py[INFO]: PID [1] started cloud-init.
2024-06-07 09:29:08,560 - main.py[DEBUG]: No kernel command line url found.
2024-06-07 09:29:08,560 - main.py[DEBUG]: Closing stdin.
2024-06-07 09:29:08,568 - util.py[DEBUG]: Writing to /var/log/cloud-init.log - ab: [640] 0 bytes
2024-06-07 09:29:08,569 - util.py[DEBUG]: Writing to /var/lib/cloud/data/python-version - wb: [644] 3 bytes
2024-06-07 09:29:08,569 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance/boot-finished
2024-06-07 09:29:08,570 - handlers.py[DEBUG]: start: init-local/check-cache: attempting to read from cache [check]
2024-06-07 09:29:08,570 - util.py[DEBUG]: Reading from /var/lib/cloud/instance/obj.pkl (quiet=False)
2024-06-07 09:29:08,570 - stages.py[DEBUG]: no cache found
2024-06-07 09:29:08,570 - handlers.py[DEBUG]: finish: init-local/check-cache: SUCCESS: no cache found
2024-06-07 09:29:08,570 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance
2024-06-07 09:29:08,587 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.rhel.Distro'>
2024-06-07 09:29:08,588 - __init__.py[DEBUG]: Looking for data source in: ['NoCloud', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM']
2024-06-07 09:29:08,607 - __init__.py[DEBUG]: Searching for local data source in: ['DataSourceNoCloud']
2024-06-07 09:29:08,607 - handlers.py[DEBUG]: start: init-local/search-NoCloud: searching for local data from DataSourceNoCloud
2024-06-07 09:29:08,607 - __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'>
2024-06-07 09:29:08,608 - __init__.py[DEBUG]: Update datasource metadata and network config due to events: boot-new-instance
2024-06-07 09:29:08,609 - __init__.py[DEBUG]: Detected platform: DataSourceNoCloud [seed=None][dsmode=net]. Checking for active instance data
2024-06-07 09:29:08,610 - dmi.py[DEBUG]: querying dmi data /sys/class/dmi/id/product_serial
2024-06-07 09:29:08,611 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/user-data (quiet=False)
2024-06-07 09:29:08,611 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/meta-data (quiet=False)
2024-06-07 09:29:08,611 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/vendor-data (quiet=False)
2024-06-07 09:29:08,611 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/network-config (quiet=False)
2024-06-07 09:29:08,612 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/user-data (quiet=False)
2024-06-07 09:29:08,612 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/meta-data (quiet=False)
2024-06-07 09:29:08,612 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/vendor-data (quiet=False)
2024-06-07 09:29:08,612 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/network-config (quiet=False)
2024-06-07 09:29:08,613 - subp.py[DEBUG]: Running command ['blkid', '-tTYPE=vfat', '-odevice'] with allowed return codes [0, 2] (shell=False, capture=True)
2024-06-07 09:29:08,661 - subp.py[DEBUG]: Running command ['blkid', '-tTYPE=iso9660', '-odevice'] with allowed return codes [0, 2] (shell=False, capture=True)
2024-06-07 09:29:08,677 - subp.py[DEBUG]: Running command ['blkid', '-tLABEL=CIDATA', '-odevice'] with allowed return codes [0, 2] (shell=False, capture=True)
2024-06-07 09:29:08,693 - subp.py[DEBUG]: Running command ['blkid', '-tLABEL=cidata', '-odevice'] with allowed return codes [0, 2] (shell=False, capture=True)
2024-06-07 09:29:08,708 - subp.py[DEBUG]: Running command ['blkid', '-tLABEL_FATBOOT=cidata', '-odevice'] with allowed return codes [0, 2] (shell=False, capture=True)
2024-06-07 09:29:08,723 - DataSourceNoCloud.py[DEBUG]: Seed from http://<domain>/api/service/v1/cloud-init/ not supported by DataSourceNoCloud [seed=None][dsmode=net]
2024-06-07 09:29:08,723 - __init__.py[DEBUG]: Datasource DataSourceNoCloud [seed=None][dsmode=net] not updated for events: boot-new-instance
2024-06-07 09:29:08,724 - handlers.py[DEBUG]: finish: init-local/search-NoCloud: SUCCESS: no local data found from DataSourceNoCloud
2024-06-07 09:29:08,724 - main.py[DEBUG]: No local datasource found

2024-06-07 09:29:10,303 - util.py[DEBUG]: Cloud-init v. 23.4-7.0.1.el9_4 running 'init' at Fri, 07 Jun 2024 09:29:10 +0000. Up 10.45 seconds.
2024-06-07 09:29:10,304 - main.py[INFO]: PID [1] started cloud-init.
2024-06-07 09:29:10,304 - main.py[DEBUG]: No kernel command line url found.
2024-06-07 09:29:10,304 - main.py[DEBUG]: Closing stdin.
2024-06-07 09:29:10,305 - util.py[DEBUG]: Writing to /var/log/cloud-init.log - ab: [640] 0 bytes
2024-06-07 09:29:10,306 - subp.py[DEBUG]: Running command ['ip', '--json', 'addr'] with allowed return codes [0] (shell=False, capture=True)
2024-06-07 09:29:10,314 - subp.py[DEBUG]: Running command ['ip', '-o', 'route', 'list'] with allowed return codes [0] (shell=False, capture=True)
2024-06-07 09:29:10,318 - subp.py[DEBUG]: Running command ['ip', '--oneline', '-6', 'route', 'list', 'table', 'all'] with allowed return codes [0, 1] (shell=False, capture=True)
2024-06-07 09:29:10,331 - handlers.py[DEBUG]: start: init-network/check-cache: attempting to read from cache [trust]
2024-06-07 09:29:10,331 - util.py[DEBUG]: Reading from /var/lib/cloud/instance/obj.pkl (quiet=False)
2024-06-07 09:29:10,334 - stages.py[DEBUG]: no cache found
2024-06-07 09:29:10,335 - handlers.py[DEBUG]: finish: init-network/check-cache: SUCCESS: no cache found
2024-06-07 09:29:10,335 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance
2024-06-07 09:29:10,348 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.rhel.Distro'>
2024-06-07 09:29:10,349 - __init__.py[DEBUG]: Looking for data source in: ['NoCloud', 'None'], via packages ['', 'cloudinit.sources'] that matches dependencies ['FILESYSTEM', 'NETWORK']
2024-06-07 09:29:10,361 - __init__.py[DEBUG]: Searching for network data source in: ['DataSourceNoCloudNet', 'DataSourceNone']
2024-06-07 09:29:10,362 - handlers.py[DEBUG]: start: init-network/search-NoCloudNet: searching for network data from DataSourceNoCloudNet
2024-06-07 09:29:10,362 - __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloudNet'>
2024-06-07 09:29:10,362 - __init__.py[DEBUG]: Update datasource metadata and network config due to events: boot-new-instance
2024-06-07 09:29:10,368 - __init__.py[DEBUG]: Datasource type DataSourceNoCloudNet [seed=None][dsmode=net] is not detected.
2024-06-07 09:29:10,368 - __init__.py[DEBUG]: Datasource DataSourceNoCloudNet [seed=None][dsmode=net] not updated for events: boot-new-instance
2024-06-07 09:29:10,368 - handlers.py[DEBUG]: finish: init-network/search-NoCloudNet: SUCCESS: no network data found from DataSourceNoCloudNet
2024-06-07 09:29:10,368 - handlers.py[DEBUG]: start: init-network/search-None: searching for network data from DataSourceNone
2024-06-07 09:29:10,368 - __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceNone.DataSourceNone'>
2024-06-07 09:29:10,369 - __init__.py[DEBUG]: Update datasource metadata and network config due to events: boot-new-instance
2024-06-07 09:29:10,369 - __init__.py[DEBUG]: Detected platform: DataSourceNone. Checking for active instance data