dangerzone icon indicating copy to clipboard operation
dangerzone copied to clipboard

0.10.0-rc1 Fails to initialize on windows (wsl --update)

Open deeplow opened this issue 2 months ago • 6 comments

What happened?

Failed to initialize.

operating system version

Windows 10 (newly installed) and I had never messed (or installed / updated) wsl.exe

Dangerzone version

v0.10.0-rc1

Podman info

Starting Podman machine: dz-internal-0.10.0
Error: dz-internal-0.10.0: VM does not exist

(I didn't bother copying and pasting the command, but it failed with error 125)

Document conversion logs


Additional info

It also showed the following in the logs and it appears to be a wsl.exe not getting called with the right parameters:

Image

But for some reason I could not copy and paste this text.

deeplow avatar Nov 03 '25 15:11 deeplow

Great, thanks a lot for this! I vaguely remember now that wsl --update is a command that may not exist on Windows 10, but it does exist on Windows 11. I'll see if there's a wsl --update equivalent there.

Other than that though, I have two questions:

  1. Did I get it right that "Copy to clipboard" does not work?
  2. The failure of wsl --update is kind of expected, which his why we continue with podman machine init and podman machine start. Is there a way to grab the logs of these commands from the log window?

apyrgio avatar Nov 03 '25 15:11 apyrgio

I'll try that. But if it's a windows 10-specific issue, then one may consider it not worth fixing given the Windows 10 EOL date. But I'll let you know.

deeplow avatar Nov 03 '25 17:11 deeplow

Thanks a lot esteemed test user (😛 ).

So, we debugged this live, and here's what we found:

  1. The wsl --update command somehow messes up the clipboard, and its contents cannot be copy-pasted. Users have to manually copy the text before and after the output of this command (yikes!). It seems we need to strip the offending characters before logging them.
  2. Our opportunistic wsl --update is not enough, and fails with this wsl.exe help message that's shown in the first comment. Then, we run podman machine init, which also fails when Podman attempts internally to do C:\Windows\system32\wsl.exe -l --quiet. We have tested a workaround by doing wsl --install but I don't really like this approach. It requires install Ubuntu (why?) and then rebooting, which is something that the user will probably never see, because it's printed as a log message.

We have to do some tests in a similar Windows 10 installation, and see if a clean Windows 11 installation is affected as well.

Windows version: 22H2 19045.6332

apyrgio avatar Nov 04 '25 10:11 apyrgio

Following up on 2. specifically, after the restart the installation does work but a black terminal window pops up for quite a few minutes:

Image

And then after a while it opens a WSL welcome screen in the background:

Image

I installed Docker desktop a very long time ago, but I remember it being more seamless and I recall them using the WSL backend. But that since that one is proprietary, we won't get many clues on how they solved it. Maybe you could look into installing Podman Desktop on a clean windows system to see if the experience is any better and how it differs from podman? Anyways, I'm not too deep in the context do proper suggestions, but I thought I'd mention this.

Let me know in case you need any extra logs / debugging / help.

deeplow avatar Nov 04 '25 10:11 deeplow

Hm, one thing we definitely need to do is to add the --quiet flag in our wsl --update invocation. I'll look into the Podman Desktop source code as well, to see how they handle this, if at all.

Let us know if conversions and upgrades work as well. Thaaanks!

apyrgio avatar Nov 04 '25 11:11 apyrgio

Let us know if conversions and upgrades work as well. Thaaanks!

They have worked out great (I'll share the logs below), although I was a bit surprised by the background download without prompting, but maybe I didn't read the "accept" prompt too well. It's nothing critical, but I started a discussion on that here.

I've saved the logs for what I think was the update process: icu-logs.txt, just in case I missed something.

deeplow avatar Nov 04 '25 12:11 deeplow

I have done a bit of research on how other container-based solutions handle this bootstrap issue on Windows. That is, what's the user journey to install WSL2 before running the actual application.

Testing environment

The environment in which I did my tests is the following:

  1. Download Windows 11 Home edition locally from here.
    • I chose the Home edition, because it may have less options related to WSL2 enabled by default, such as virtualization.
  2. Install Windows 11 in a VM.
    • Here I chose Gnone Boxes, since it has some sane options by default (Host CPU passthrough, easy snapshoting, etc.)
    • Windows 11 cannot be installed natively in a VM, so I chose to go down the BIOS route, by setting some registry keys.
  3. Snapshot the machine after everything is configured and is freshly started.

For each of the applications below, when I'm done with testing it, I restore the VM back to the snapshot I had taken, so that the next attempt comes from a fresh state.

Docker Desktop

Docker Desktop is closed-source, so we don't really know what they are doing internally during installation time. What the user experiences though is:

  1. Download Docker Desktop and start the installer.

  2. After the installation completes, the user is asked to reboot.

  3. After the reboot, once they start Docker Desktop, the user is prompted to run wsl --update in their terminal with admin privileges:

    Image
  4. After this is successful, the user clicks on "Restart", and Docker Desktop is working.

Podman Desktop

Podman Desktop has two phases. First you install Podman Desktop, and then you can install the Podman tool. The WSL2 installation is only triggered in the second phase. Here's what the user experiences:

  1. Download Podman Desktop and start the installer.

  2. After the installation completes, start Podman Desktop. It will prompt the user to install Podman.

  3. During the Podman installation, it will make some checks, and will detect that a Windows feature (VirtualMachinePlatform) is not enabled:

    Image
  4. The user is prompted to run dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart in their terminal with admin privileges.

  5. After this is successful, they resume the install operation of Podman.

  6. Podman notifies the user that it will enable WSL2, and this may require a reboot.

  7. After WSL2 is installed, the user is asked to reboot.

  8. After the reboot, the user can open Podman Desktop.

Rancher Desktop

Rancher Desktop fails the installation from the get go, if WSL2 is not configured, and provides instructions to the user:

Image

apyrgio avatar Nov 06 '25 12:11 apyrgio

What's interesting from the above comparison is that is wasn't always like this. More specifically, both Podman Desktop and Rancher Desktop were installing WSL2 right from the installer.

Podman Desktop:

		<ComponentGroup Id="WSLFeature" Directory="INSTALLDIR">
			<Component Id="WSLFeatureComponent" Guid="F6A693BC-186C-4E64-8015-C3073013B3A8" Condition="(NOT Installed) AND WSL_INSTALL = 1">
				<CreateFolder />
				<PanelSW:Dism EnableFeature="VirtualMachinePlatform" ErrorHandling="prompt" />
				<PanelSW:Dism EnableFeature="Microsoft-Windows-Subsystem-Linux" ErrorHandling="prompt" />
			</Component>
		</ComponentGroup>

(from https://github.com/containers/podman/blob/4f02e65557f5aa516a57a57e606cf7306e105511/contrib/win-installer/podman.wxs#L74-L80)

Rancher Desktop:

function Install-Features {
  # Install Windows features as needed.
  $features = @('Microsoft-Windows-Subsystem-Linux', 'VirtualMachinePlatform')
  foreach ($feature in $features) {
    Write-Output "Checking Windows feature $feature..."
    if ((Get-WindowsOptionalFeature -FeatureName $feature -Online).State -ne "Enabled") {
      Write-Output "Enabling Windows feature $feature..."
      Enable-WindowsOptionalFeature -All -FeatureName:$feature -Online -NoRestart
    }
  }
  Write-Output "Windows features installed."
}

function Install-Kernel {
  # Install WSL kernel on reboot.
  Write-Output "Will install Linux kernel after reboot."

  Set-ItemProperty `
    -Path 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce' `
    -Name '!Rancher Desktop WSL Kernel Install' `
    -Value "`"$Env:SystemRoot\system32\wsl.exe`" --update"
}

(from https://github.com/rancher-sandbox/rancher-desktop/blob/ebd68d933426f5950a5b4527ebcb90c8240e52d4/build/wix-install-wsl.ps1)

This changed since the start of 2025, where both projects removed this functionality, because they experienced some regressions:

  • https://github.com/containers/podman/pull/25237
  • https://github.com/rancher-sandbox/rancher-desktop/pull/8110

There's also an interesting point that I've seen in the comments somewhere, which is that it makes sense to let the user install WSL on their own, because an application installer should in theory be able to fully uninstall itself, and get back to the original state. With WSL though, you can't just disable it during uninstall, because you don't know if another app requires it in the meantime.

apyrgio avatar Nov 06 '25 12:11 apyrgio

From the above comparisons, we see that Docker Desktop somehow does not ask the user to enable the VirtualMachinePlatform windows feature (probably does it at install time), although they do ask them to perform wsl --update. On the other hand, we can see that Podman does wsl --install under the hood, but it doesn't do dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart (not sure why).

These projects can ask their users to enter some commands in the terminal, because at the end of the day they are used by technical people. With the switch to embedded Podman, we want to make Dangerzone more accessible to non-technical people, which are people who have not interacted with a terminal before, even less spawn one with admin rights.

The ideal UX journey that we want for Dangerzone would be the following:

  1. If a user has not WSL installed: Install Dangerzone, prompt for a reboot, start Dangerzone, install WSL without a reboot, convert a document
  2. If a user has WSL installed: Install Dangerzone, start Dangerzone, convert document.

I see two ways we can achieve this:

Option 1 - Enable VirtualMachinePlatofrm during the install

  1. Detect in the Dangerzone installer if the VirtualMachinePlatform feature is enabled.
  2. If not, enable it, since we're in a privileged context
  3. Schedule a reboot, since this feature requires a reboot.
  4. Once rebooted, let Podman install WSL under the hood. It should hopefully not require a reboot.

Option 2 - Bundle the WSL .msi

This is basically what a Rancher dev suggested, which is to bundle the Dangerzone .msi and the WSL one into a single .exe, and install them both at the same time. This can help in case of airgapped installations, although it's quite simple for a user to install this MSI alongside our application.


I'm currently trying out option 1 and will report here soon. If anyone has more extensive WiX/Windows knowledge and can assist with this, that would be badass. Thanks!

apyrgio avatar Nov 06 '25 13:11 apyrgio

I have managed to enable the VirtualMachinePlatform and Microsoft-Windows-Subsystem-Linux Windows features while the installation runs (see 592e4ab5). I'm also scheduling a reboot after they are done, and I can verify that Dangerzone works properly after the reboot.

(Note: I still haven't decided if wsl --update or wsl --install or wsl --install --no-distribution should be used before invoking Podman, but that's the easy part)

My current problem is detecting how to prompt for the above actions (mainly the reboot) conditionally. Basically, we want to set a "session property" (Windows Installer term) if these Windows features are not enabled, and perform our actions only in that case. In practice though, setting a property requires building a .dll and invoking some esoteric Windows API. I haven't found a way to do it via command output or something else.

Rancher Desktop, for instance, uses a Go module (see how it sets session properties). We can potentially re-use this for our needs.

Another alternative I tried is to run scripts, either VBScript or JScript, which can set properties. In practice, we cannot use them, because VMScript is officially deprecated. Read also this: https://documentation.help/WiX-Help/robmen_20040520.htm

Yet another alternative I tried is to run a command, and create a file as a flag, so that we can create a property out of it using FileSearch. In practice, this can't work because FileSearch runs very early, before AppSearch, and I haven't managed to run a Powershell command (or any command for that matter) before AppSearch.

Another way that I have not explored yet is to create a script that exits with 3010. This is a special code that signal to the Windows installer that reboot is required.

apyrgio avatar Nov 07 '25 12:11 apyrgio

I took a step back from the reboot problem, because I was stuck at this point, and focused on the last problem that we have, which is what should Dangerzone do after the reboot. Should it run wsl --update, as it does now?

We know that the windows-2022 GitHub runner is in a state where the necessary Windows features are enabled, but WSL is not properly installed, so we must do something. For a while, our solution was to opportunistically run wsl --update, but this proved insufficient.

Scenarios

First of all, in order to find out which are the proper WSL commands to use, we need to understand how the WSL CLI behaves in different Windows versions and states. Unfortunately, it's behavior varies wildly, and that makes it hard to depend on. I have grouped some WSL invocations into scenarios, and for every invocation, I add its exit code at the end of the command output. I haven't copied the exact command output unfortunately, so what I'm reporting here is either a portion of it, or a summary.

The main question in these scenarios is "will podman machine init run successfully in them"?

Scenario 1 - Windows features are disabled (fresh installation)

Windows 10 22H2

> wsl --status
[shows short help message]
# 1

> wsl -l --quiet
[shows short help message]
# 1

> wsl --update
[shows short help message]
# 1

> wsl --install --no-distribution
[enables Windows features, installs WSL locally, prints in the terminal that it will need a reboot]
# 0

# After `wsl --install` succeeds, but without a reboot
> wsl -l --quiet
This application requires the Windows Subsystem for Linux Optional Comonent.
Install it by running: wsl.exe --install -no-distribution
The system may need to be restarted ...
Error code: Wsl/WSL_E_WSL_OPTIONAL_COMPONENT_REQUIRED 
# -1

On a fresh installation, podman machine init fails. If we run wsl --install beforehand and don't reboot, podman machine init fails too.

Windows 11

> wsl --status
WSL is not installed ... you can install it with wsl --install ...
# 50

> wsl -l --quiet
... WSL is not installed, you can install it with --install ...
[block for 60 seconds, waits for user prompt!?]
# 1 (if no prompt for 60 seconds)

> wsl --update
[enables Windows features, installs WSL locally, prints in the terminal that it will need a reboot]
# 0

> wsl --install --no-distribution
[enables Windows features, installs WSL locally, prints in the terminal that it will need a reboot]
# 0

# After `wsl --install` succeeds, but without a reboot
> wsl -l --quiet
# 0

On a fresh installation, podman machine init fails. If we run wsl --install beforehand and don't reboot, podman machine init will work though, and prompt the user to reboot.

Scenario 2 - Windows features are enabled (no reboot)

Windows 10

> wsl -l --quiet
[shows help message]
# 1

> wsl --update
[shows help message]
# 1

> wsl --install --no-distribution
[enables Windows features, installs WSL locally, prints in the terminal that it will need a reboot]
# 0

# After `wsl --install` succeeds, but without a reboot
> wsl -l --quiet
This application requires the Windows Subsystem for Linux Optional Comonent.
Install it by running: wsl.exe --install -no-distribution
The system may need to be restarted ...
Error code: Wsl/WSL_E_WSL_OPTIONAL_COMPONENT_REQUIRED 
# -1

In an installation where the necessary Windows features are enabled, podman machine init can still fail. If we run wsl --install beforehand and don't reboot, podman machine init fails too.

Windows 11

> wsl -l --quiet
... WSL is not installed, you can install it with --install ...
[block for 60 seconds, waits for user prompt!?]
# 1 (if no prompt for 60 seconds)

> wsl --update
[installs WSL locally, prints in the terminal that it will need a reboot]
# 0

> wsl --install --no-distribution
[installs WSL locally, prints in the terminal that it will need a reboot]
# 0

# After `wsl --install` succeeds, but without a reboot
> wsl -l --quiet
# 0

The behavior of podman machine init in this scenario is the same as in Windows 10.

Scenario 3 - Windows features are enabled, after reboot

From this point on, podman machine init will work and prompt for a reboot, both on Windows 10 and Windows 11.

Scenario 3 - In Dangerzone installer

Within the Dangerzone installer, if we attempt to run wsl --update or wsl --install, they will fail, with the error message that contains the UTF-16 characters. If we enable the Windows features beforehand, and attempt to re-run them, they will fail again, this time with a different error.

Conclusion

If the wsl -l --quiet command works, Podman can take it from here, and instruct the user to reboot, if necessary. Unfortunately, this does not happen in Windows 10, if the user has not rebooted, so we can't rely on just Podman here.

The most robust way I can think of, to run podman machine init on a fresh installation, that works on Windows 10/11, is to update the task that initializes the Podman machine (MachineInitTask) and replace our wsl --update invocation with the following steps:

  • Check if wsl --quiet fails. If it doesn't fail, proceed with podman machine init.
  • If yes, run wsl --install -no-distribution
    • If it fails, attempt to run wsl --install, in case the --no-distribution flag is not present.
      • If that fails again, show a dialog with troubleshooting instructions, that point to this guide: https://podman-desktop.io/docs/troubleshooting/troubleshooting-podman-on-windows
      • This command has the drawback that it will install an unnecessary distribution in the user's system, but it's our best bet at this point.
  • If the installation succeeds, prompt the user to restart, and block the rest of the tasks from starting.

The above should work even in the absence of special commands in the Windows installer, since the wsl command installs the necessary Windows features. It should also work in our GitHub CI, and when developing Dangerzone on a Windows machine.

apyrgio avatar Nov 10 '25 12:11 apyrgio

0.10.0 RC2 was just released with this fix, thanks!

almet avatar Nov 25 '25 18:11 almet