Fully implement CloudInit (Ubuntu)

[nyanhp]

Description

Fixes #1498

• [x] - I have tested my changes.
• [x] - I have updated CHANGELOG.md and added my change to the Unreleased section
• [x] - The PR has a meaningful title.
• [x] - I updated my fork/branch and have integrated all changes from AutomatedLab/develop before creating the PR.

Type of change

• [ ] Bug fix
• [x] New functionality
• [ ] Breaking change
• [ ] Documentation

How was the change tested?

$labName = 'ALLovesLinux'
New-LabDefinition -Name $labName -DefaultVirtualizationEngine HyperV
Add-LabVirtualNetworkDefinition -Name $labName -AddressSpace 192.168.130.0/24
Add-LabVirtualNetworkDefinition -Name 'Default Switch' -HyperVProperties @{ SwitchType = 'External'; AdapterName = 'Ethernet' }
$PSDefaultParameterValues = @{
    'Add-LabMachineDefinition:Network' = $labName
    'Add-LabMachineDefinition:ToolsPath'= "$labSources\Tools"
    'Add-LabMachineDefinition:DnsServer1' = '192.168.130.10'
    'Add-LabMachineDefinition:Memory' = 1.5GB
    'Add-LabMachineDefinition:OperatingSystem' = 'Windows Server 2022 Datacenter'
}
$PSDefaultParameterValues.Add('Add-LabMachineDefinition:Gateway', '192.168.130.10')

Add-LabDomainDefinition -Name contoso.com -AdminUser install -AdminPassword Somepass1
Set-LabInstallationCredential -Username Install -Password Somepass1

$netAdapter = @()
$netAdapter += New-LabNetworkAdapterDefinition -VirtualSwitch $labName -Ipv4Address 192.168.130.10
$netAdapter += New-LabNetworkAdapterDefinition -VirtualSwitch 'Default Switch' -UseDhcp
$postInstallActivity = @()
$postInstallActivity += Get-LabPostInstallationActivity -ScriptFileName PrepareRootDomain.ps1 -DependencyFolder $labSources\PostInstallationActivities\PrepareRootDomain
Add-LabMachineDefinition -Name LINDC1 -Roles RootDC, Routing -Memory 1GB -PostInstallationActivity $postInstallActivity -NetworkAdapter $netAdapter -DomainName contoso.com
Add-LabMachineDefinition -Name UB001 -Memory 8GB -DomainName contoso.com -OperatingSystem 'Ubuntu-Server 22.04.2 LTS "Jammy Jellyfish"' -SshPublicKeyPath $home\.ssh\AzurePubKey.pub -SshPrivateKeyPath $home\.ssh\AzurePubKey

Install-Lab

[nyanhp]

That was a painful development. I'm not adding any more distributions unless there is at least some standardization 😖

[stale[bot]]

Labeling this pull request (PR) as abandoned since it has gone 14 days or more since the last update. An abandoned PR can be continued by another contributor. The abandoned label will be removed if work on this PR is taken up again.

[nyanhp]

@raandree ? :)

[raandree]

Sorry for the delay, @nyanhp. I have just tried the lab you have provided. The deployment wasn't able to reach the Linux VM. Where can I start troubleshooting?

Note: I have used the image Ubuntu 23.04 "Lunar Lobster".

10:58:09|00:02:30|00:00:00.000| - Installing Root Domain Controllers
10:58:09|00:02:30|00:00:00.025|   - Machines with RootDC role to be installed: 'LINDC1'
10:58:09|00:02:30|00:00:00.047|   - Waiting for machines to start up..
10:58:22|00:02:43|00:00:12.776|   - Waiting for Root Domain Controllers to complete installation of Active Directory and restart.........done
11:03:52|00:08:13|00:05:42.800|   - Root Domain Controllers have now restarted. Waiting for Active Directory to start up.........
11:05:32|00:09:53|00:07:23.356|   - Done
11:05:32|00:09:53|00:00:00.000| - Configuring routing
11:05:33|00:09:53|00:00:00.059|   - Waiting for machines with Routing Role to startup..
11:05:44|00:10:05|00:00:11.878|   - Configuring Routing role..................
11:08:49|00:13:10|00:03:16.660|   - Waiting for configuration of routing to complete.......
11:10:11|00:14:31|00:04:38.091|   - Done
11:10:11|00:14:32|00:00:00.000| - Starting remaining machines
11:10:11|00:14:32|00:00:00.010|   - There are 1 Linux hosts in the lab.
                On Windows, those are installed from scratch and do not use differencing disks.

                If you did not connect them to an external switch or deploy a router in your lab,
                AutomatedLab will not be able to reach your VMs, as PowerShell will not be installed.

                The timeout to wait for VMs to be accessible via PowerShell was reduced from 60 to 15
            minutes.
11:10:11|00:14:32|00:00:00.015|   - Waiting for machines to start up.................................Wait-LWLabJob: Timeout while waiting for job 206
Write-LogFunctionExitWithError: Wait-LabVM...leaving: The following machines are not ready: UB001

The Linux VM has trouble reaching the internet. I guess I need a network adapter connected to the internet. I am wondering how your lab worked successfully.

[nyanhp]

My lab has a router, in case you didn't try it

[nyanhp]

@raandree powershell is not yet available for 23.04. Please use an LTS version.

[raandree]

After using Ubuntu 22.04.3 LTS "Jammy Jellyfish" the same happened:

22:55:13|00:07:02|00:04:38.640|   - Root Domain Controllers have now restarted. Waiting for Active Directory to start up.........
22:56:51|00:08:40|00:06:16.249|   - Done
22:56:51|00:08:40|00:00:00.000| - Configuring routing
22:56:51|00:08:40|00:00:00.022|   - Waiting for machines with Routing Role to startup..
22:57:02|00:08:52|00:00:11.631|   - Configuring Routing role...............
22:59:26|00:11:15|00:02:34.986|   - Waiting for configuration of routing to complete......
23:00:49|00:12:38|00:03:58.077|   - Done
23:00:49|00:12:38|00:00:00.000| - Starting remaining machines
23:00:49|00:12:38|00:00:00.004|   - There are 1 Linux hosts in the lab.
                On Windows, those are installed from scratch and do not use differencing disks.

                If you did not connect them to an external switch or deploy a router in your lab,
                AutomatedLab will not be able to reach your VMs, as PowerShell will not be installed.

                The timeout to wait for VMs to be accessible via PowerShell was reduced from 60 to 15
            minutes.
23:00:49|00:12:38|00:00:00.011|   - Waiting for machines to start up.................................Wait-LWLabJob: Timeout while waiting for job 466
Write-LogFunctionExitWithError: Wait-LabVM...leaving: The following machines are not ready: UB001
..............................Wait-LWLabJob: Timeout while waiting for job 468
Write-LogFunctionExitWithError: Wait-LabVM...leaving: The following machines are not ready: UB001
.
23:32:11|00:44:00|00:31:21.960|   - Done
23:32:11|00:44:00|00:00:00.000| - Performing pre/post-installation tasks defined for each machine
23:32:11|00:44:01|00:00:00.365|   - Pre/Post-installations done
23:32:18|00:44:07|00:00:00.000| - Installing RDS certificates of lab machines
23:32:18|00:44:07|00:00:00.181|   - ..
23:32:19|00:44:09|00:00:01.559|   - Done

[nyanhp]

Looks like your VM still boots the installation medium.

[dkettman]

@nyanhp

If you need/want someone else to test, I am trying to use exactly what you are using (Ubuntu 22.04.03) and run into a similar issue. I downloaded the server installer, and it sits and waits for a cloud-init response, then continues on with the regular install process.

I may be able to explain my issue due to the type of networking I'm doing. I'm using a software NAT (New-NetNat) to separate everything in the lab from everything else on my network. My assumption is, my AutomatedLab setup is not listening on the right interface to know someone is offering up cloud-init data. But I dont know enough about all of it to know for sure.

If you need help testing something I'll do what I can to help. I think I may end up using this with some of my colleagues for some training instances so helping to further the cause is in my best interest. :)

[stale[bot]]

Labeling this pull request (PR) as abandoned since it has gone 14 days or more since the last update. An abandoned PR can be continued by another contributor. The abandoned label will be removed if work on this PR is taken up again.

[nyanhp]

🎉

Sorry for the long hiatus @raandree , being a VP is surprisingly time-consuming.

I've rethought my approach and now think it works better. The machine's Install VHD is detached and removed after it has shut down during the automated setup. This ensure the VM boots properly, without random issues.

I've tested with this beauty as suggested by @dkettman. Remember to use your own SSH keys when testing, @raandree

# Lab Variables

$labName = 'MYLAB'
$labSubnet = '172.0.99' #Just the first three octets
$sql_iso_url = 'https://go.microsoft.com/fwlink/p/?linkid=2215158&clcid=0x409&culture=en-us&country=us'

Import-Module AutomatedLab

#checking if our NAT is created. We want internet access!
$nats = Get-NetNat

if ($nats.Count -eq 0) {
    Write-ScreenInfo -Message "No NATs defined! Defining one for us!"
    New-NetNat -Name $labName -InternalIPInterfaceAddressPrefix $labSubnet'.0/24'
}
else {
    Write-ScreenInfo -Message "There is already a NAT defined. Windows can only handle one NAT at a time. (Weak!)"
    Write-ScreenInfo -Message "Checking if we can use it..." -TaskStart
    if ($nats.InternalIPInterfaceAddressPrefix -like "$labSubnet*") {
        Write-ScreenInfo -Message "As far as I can tell, it looks like this should work. If not, remove the NAT using"
        Write-ScreenInfo -Message "the 'Remove-NetNAT' cmdlet and restart this script." -TaskEnd
    } 
    else {
        Write-ScreenInfo -Message "Looks like this NAT may not work."
        Write-ScreenInfo -Message "Please double-check that the existing NAT will and adjust accordingly."
        Write-ScreenInfo -Message "Without a NAT configured to handle the $labSubnet.0/24 subnet, you will not have Internets Access" -TaskEnd
        break
    }
}

#create an empty lab template and define where the lab XML files and the VMs will be stored
New-LabDefinition -Name $labName -DefaultVirtualizationEngine HyperV

#make the network definition
Add-LabVirtualNetworkDefinition -Name $labName -AddressSpace $labSubnet'.0/24'

# Define the Domain and the Domain Administration acocunt
Add-LabDomainDefinition -Name MYLAB.lab -AdminUser cs_admin -AdminPassword Somepass1

# Create a local user on each box as an administrator of 'cs_admin'
Set-LabInstallationCredential -Username cs_admin -Password Somepass1

# Setting up defaults for all machines. These can be overridden when the machine is defined
$PSDefaultParameterValues = @{
    'Add-LabMachineDefinition:DomainName'      = 'MYLAB.lab'
    'Add-LabMachineDefinition:Network'         = $labName
    'Add-LabMachineDefinition:Gateway'         = $labSubnet + '.1'
    'Add-LabMachineDefinition:DnsServer1'      = $labSubnet + '.10'
    'Add-LabMachineDefinition:ToolsPath'       = "$labSources\Tools"
    'Add-LabMachineDefinition:OperatingSystem' = 'Windows Server 2022 Standard (Desktop Experience)' # GUI by default
}


# Machine Definitions

# DC01
## Roles: RootDC
Add-LabMachineDefinition -Name dc01 `
    -Memory 2GB `
    -OperatingSystem 'Windows Server 2022 Standard' `
    -IpAddress $labSubnet'.10' `
    -DomainName MYLAB.lab `
    -Roles RootDC, CaRoot

Add-LabMachineDefinition -Name ubuntu01 `
    -Memory 8GB `
    -Processors 1 `
    -IpAddress $labSubnet'.90' `
    -SshPublicKeyPath "$home\.ssh\AzurePubKey.pub" `
    -SshPrivateKeyPath "$home\.ssh\AzurePubKey" `
    -OperatingSystem 'Ubuntu-Server 22.04.3 LTS "Jammy Jellyfish"'


# Get to work!
Install-Lab

[nyanhp]

Little side note, of course you already know this: PowerShell on Ubuntu 23 is not yet supported and the package cannot be found in the package repository. I've skipped generating errors due to this, but it means that 23 is not yet remotely reachable.

[raandree]

@nyanhp, I have tried it again today. The domain controller was installed fine of course, the Ubuntu machine ran into a timeout:

11:38:51|00:06:12|00:00:00.000| Installing Certificate Servers
11:38:51|00:06:12|00:00:00.000| - Machines with Root CA role to be installed: 'dc01'
11:38:51|00:06:12|00:00:00.001|   - Waiting for machines to start up..
11:39:00|00:06:21|00:00:08.727|   - ..Waiting for Root CA(s) to complete installation.....
11:39:59|00:07:20|00:01:07.548|   - Publishing certificates from CA servers to all online machines....
11:40:17|00:07:38|00:01:25.347|   - Finished installation of Root CAs
11:40:17|00:07:38|00:01:25.350| - Done
11:40:17|00:07:38|00:00:00.000| Starting remaining machines
11:40:17|00:07:38|00:00:00.012| - There are ubuntu01 Linux hosts in the lab.
                On Windows, those are installed from scratch and do not use differencing disks.

                If you did not connect them to an external switch or deploy a router in your lab,
                AutomatedLab will not be able to reach your VMs, as PowerShell will not be installed.

                The timeout to wait for VMs to be accessible via PowerShell was reduced from 60 to 15
            minutes.
11:40:17|00:07:38|00:00:00.163| - Waiting for machines to start up.................................Wait-LWLabJob: Timeout while waiting for job 484
Write-LogFunctionExitWithError: Wait-LabVM...leaving: The following machines are not ready: ubuntu01
[email protected]'s password:

This is the screen of the ubuntu machine. Unfortunately I cannot scroll up to see the rest:

[nyanhp]

So, which version of ubuntu did you use? From what I can see it failed to partition the disk which is odd.

[nyanhp]

@raandree there is still a slight timing issue that I cannot quite pinpoint, but in general it works: The thing is that the host keys need to be updated after the installation. Maybe you have a good idea when to do that. We can have a chat via Teams on that ;)

[raandree]

Just tried it and did run into this:

The AL installation session hangs here:

21:46:58|00:07:51|00:00:00.000| Starting remaining machines
21:46:58|00:07:51|00:00:00.000| - There are ubuntu01 Linux hosts in the lab.
                On Windows, those are installed from scratch and do not use differencing disks.
        
                If you did not connect them to an external switch or deploy a router in your lab,
                AutomatedLab will not be able to reach your VMs, as PowerShell will not be installed.

                The timeout to wait for VMs to be accessible via PowerShell was reduced from 60 to 15
            minutes.
21:46:58|00:07:51|00:00:00.078| - Waiting for machines to start up...SSH Transport is not available from within Windows PowerShell.
21:47:13|00:08:05|00:00:14.351| - ...................

The Ubuntu machine shows this (unfortunately I cannot copy the text):

I am using that Ubuntu ISO image:

Ubuntu-Server 22.04.3 LTS "Jammy Jellyfish"         0   x64  22.4            08.12.2023 00:00:00 D:\LabSources\ISOs\jammy-live-server-amd64.iso

[nyanhp]

Well, in that case I give up. I have no idea why your VM fails partitioning the disk. I'll give it one last try myself with a new VM to install the lab on. After that, I don't care if we keep Ubuntu in as "don't bother us with issues, if it works, count yourself lucky" or if we close this PR. This ridiculous subiquity gave me so much headache already.

[nyanhp]

@raandree further testing:

• 20.04 ??? Failed me once for no apparent reason
• 22.04 Works fine, on two separate physical hosts as well as in a VM acting as Hypervisor
• ~24.04 Too new, subiqutiy format changed~ Testing 24.04 again with slightly updated cloudinit conf