Enhancement Ideas to support Automation Use Cases

[kimfaint]

My use case is for Automated testing on Windows. I'm using my fork to add the things I need. But I'd like to contribute as much back upstream as I can in case others find it useful.

The changes I've made (but yet to commit) are:

1. Enable SSHD on Windows (so from my Linux host I can debug logs or copy files to the Windows guest without needing a graphical console or any file sharing)
2. Disable Windows Automatic Updates (they are not useful for my use case)
3. Disable the Lock Screen
4. Enable Automatic Login for the default user (so I can start a new VM, the default user logs in and the tests run)

Things I plan to do soon:

5. Provide an option to set the default user account to something other than admin

6. Provide an option to "warmup" the default user account, before we write out the VM image:

   • Perform an initial login for the default user so the "We're getting things ready for you" part happens before we store the VM image
   • Run an additional optional script provided by the user (in my case to install the automatic testing framework and put a bat file in the users Startup folder, which will run the tests)

I realize many of things will not be useful to everyone, but perhaps some of them could be options in the build script and others like sshd could be defaults. @antifob what do you think?

[antifob]

I've been wanting to add flexibility to the imaging process since I started the project. The way I see it, there are 2 main components to the process:

1. the Autounattend.xml file
2. the commands to run (i.e. what's in the local/ directory)

For point 2, it would be easy to replace the list in the Autounattend.xml files with one that finds and runs the scripts in local/. I believe it covers your points 1, 2, 3, 4 and 5. @kimfaint Does adding a user-defined directory to the ISO and having a hook call a script in it automatically help you in any way? This script could copy files to the VM and configure the environment to your liking (sshd, auto-login, accounts, etc.).

Regarding "warming up", it's a matter of re/booting the VM and waiting for Windows' setup to end. I have not found a way to do this reliably programmatically. I'm open to ideas.

[kimfaint]

I thought SSHD would be a default feature as most Linux users would find this useful.

For everything else I was thinking of adding an option to build.sh (which would be passed to pack.sh), for example --extra <scriptfile> where scriptfile is a ps1 file contains all of the extra user defined setup. It could be added to the ISO and run by sysprepf.bat just prior to the sysprep.exe line.

The "warming up" I'm also not sure about how to do it.

I was also wondering about why the VM is published to an image buildxxxx and this image is exported to output/..., then the image is deleted. Before you can use it, you need to run import.sh. Would it be more efficient to publish the image as alias win$ver and then export the image to output/... and not delete the image?

[antifob]

I was also wondering about why the VM is published to an image buildxxxx and this image is exported to output/..., then the image is deleted. Before you can use it, you need to run import.sh. Would it be more efficient to publish the image as alias win$ver and then export the image to output/... and not delete the image?

It would be more efficient. However, there's no way to know if the user wants to replace their alias or even if the win$ver format is desirable (I use windows/$ver). Replacing an alias is a destructive operation, so the safe way is to not touch them.

From that perspective, the build image is just lying around so deleting it makes sense.

[kimfaint]

Those are good points. Perhaps I have changed my thinking. If I use this toolset to create a generic windows image, export the archive and import an an image, e.g. alias windows/10e.

I then need to launch an instance of windows/10e and perform some additional setup for automation, before publishing as another image, e.g. alias windows/10e/robot.

There is some benefit to this approach as when I am tinkering with the automation setup, I don't need to start from scratch each time and repeat the windows installation. So it's faster to try changes in my robot setup, I can live with having a few extra gigabytes of images.

However I can't find a way to remotely run the additional automation setup script. I have not been able to connect via winrm (I tried from the linux host using pywinrm). So currently I'm using my fork branch which adds sshd https://github.com/kimfaint/incus-windows/tree/sshd and having some success doing it this way.

To take care of the warmup of the 'admin' user, my setup configures automatic logon for the admin user. So I figure if I just restart the VM again and wait for the admin user to auto login, I can wait and somehow tell that this OOBE is complete and then shutdown and publish the windows/10e/robot image.

[antifob]

If I use this toolset to create a generic windows image [...]

Yes. That's the intended use case for this project; essentially some kind of baseline that one can add to to their liking (and re-image, possibly).

I have not been able to connect via winrm [...]

What you're experiencing might be the delayed start of the winrm service. It's the default configuration, but you might want to change that to Automatic.

Here's an adapted script from https://pypi.org/project/pywinrm/

#!/usr/bin/env python3
import sys, winrm.protocol

p = winrm.protocol.Protocol(
    endpoint=f'https://{sys.argv[1]}:5986/wsman',
    transport='ntlm',
    username='admin',
    password='changeme',
    server_cert_validation='ignore')
shell_id = p.open_shell()
command_id = p.run_command(shell_id, 'ipconfig', ['/all'])
std_out, std_err, status_code = p.get_command_output(shell_id, command_id)
p.cleanup_command(shell_id, command_id)
p.close_shell(shell_id)

print(std_out.decode())

[kimfaint]

Thanks for that, I have it working now. I was using the winrm.Session object and set target=hostname. Setting target to the full URL seems to fix it. I also needed to add the server_cert_validation='ignore' and wait for the delayed start.

I'm pushing files to and also running scripts using winrm now. As part of my robot setup, and still enable sshd just in case it's useful for debugging later.

The python script I use to run commands, runs an initial test command in a try/except block in a while loop. This allows it to wait for WinRM to be started and available, before the real command is run. I use this after the second reboot and have assumed that WinRM starts after the OOBE has concluded.

[antifob]

I finally got around to making a long-planned change in https://github.com/antifob/incus-windows/commit/9ec75b32a2fa55c866a4a3366bfb9b4e4c69fae1

This should allow you to more easily add customizations to the image, on the side, and better being able to integrate changes in the main repo.

Please, let me know if this is enough for your use case.

[kimfaint]

I have not reviewed your change in detail, but it seems like it will enable user customizations, such as OpenSSH.Server #11 and File Sharing with VirtioFS/WinFSP #6.

My use case project is using the main branch of my fork. The only thing that your change above would not allow me to do is this --noexport option for build.sh, so the image is able to be published to the local server without needing to export to files and re-import https://github.com/antifob/incus-windows/compare/main...kimfaint:incus-windows:noexport.

But as I would prefer to be on the main branch of your repo, I can probably live without the small efficiency that the --noexport option provides.

Later this week I will put #11 and #6 into my use case project as local/ customizations. Once I have done this and tested it all works, I will close the associated PRs.