Windows and macOS testing capabilities should not be at Randy's house

[rfay]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your feature request related to a problem?

We've recognized long-term that the current test runners being located at Randy's house is not optimal or in-line with our goal of having the project decrease dependence on Randy.

We are fine with Linux testing, using GitHub's graciously provided GitHub Actions, but we haven't been successful with either macOS or Windows/WSL testing elsewhere.

Exceptions:

• MacStadium has provided a mac M1 that works great and is properly managed. It's possible that in the future we could have more there (we probably need 3 more macOS ARM64 machines). (At current costs, about $100/month/machine, this would be cost-prohibitive with our current budget)
• @seeben has provided a Windows machine that is currently deployed for WSL2 testing. Those machines cost about $42/month, and in some configurations can support more than one test at the same time.

Describe your solution

Options include:

• macOS: Pay the money to MacStadium or similar vendor. This would be at least $300/month, a lot given our budget
• GitHub is supposed to start providing macOS ARM64 machines this year to open-source projects, and with greater memory capacity than in the past. It's possible that our macOS testing, or at least testing for some permutations, could land in GitHub again. (In the past, we had Colima testing there, but it was abysmally slow and unreliable.)
• Moving Windows testing to hetzner or another similar provider might work (Hetzner is where @seeben's provided Windows test runner is). In most cases we've tried, the lack of support (or good support) for nested virtualization has kept us from success. (It's been a few years since we tried Azure windows instances, it's worth looking at that again.
• appveyor has Windows and macOS instances that just might work. It's been a long time since we checked in with htem.
• Microsoft's Azure might work for Windows

Describe alternatives

Suggestions are welcome!

Additional context

No response

[GuySartorelli]

According to https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories windows and macos are both available for free for public repos in GitHub Actions.

[rfay]

We used Github Actions for macOS for a year or two for Colima. It was incomprehensibly slow and unreliable. (They are promising bigger machines and arm64 later this year though, right now they have only amd64, which is essentially obsolete as far as users go.)

We have attempted Windows on several services like Github Actions, and in general because they don't have a UI and don't support nested virtualization (WSL2 and Docker) they have failed.

[rfay]

I experimented with the current macos-15 (arm) and windows-2025 runners today using https://github.com/rfay/trywin and learned a few things.

• On windows-2022, I was in fact able to run WSL v2. (But it wasn't set as default). I was able to install docker and ddev inside an Ubuntu distro. (I did have to upgrade WSL wsl --update) . Performance was adequate.
• windows-2025 came with a current version of WSL, so just wsl --install Ubuntu worked.
  • I was able to use the v1.24.7 Windows Installer in silent mode for DockerCE and it worked, with the exception of setting mkcert.exe (and I couldn't do the mkcert.exe manually, it hung, apparently for the UAC). snap works fine, so sudo snap install --classic go is fine.
  • ./ddev_windows_amd64_installer.exe /S hung, apparently it was doing mkcert.exe or something. I wasn't able to <ctrl-C>
  • Both windows-2022 and windows-2025 come provisioned with an ancient docker version, 27.5.1, apparently an old Docker Desktop. But... It's configured for Windows containers.
  • I thought maybe Traditional Windows would work (with just providing ddev.exe) but got # ddev.exe start Failed to ensure Docker network ddev_default: Error response from daemon: could not find plugin bridge in v1 plugin registry: plugin not found (Probably a result of Docker being configured for Windows containers)
• macos-15:
  • Docker Desktop now has a way to be installed from command line! sudo /Volumes/Docker/Docker.app/Contents/MacOS/install --accept-license --user=runner (but it didn't work for me, and docker desktop start hung)
  • Colima start failed with Virtualization is not available on this hardware
  • Lima fails with Invalid virtual machine configuration. Virtualization is not available on this hardware.

Summary:

• Windows/WSL2-DockerCE: We might be able to do a decent job testing DockerCE-based WSL2 testing.
• Windows ARM64 runners don't seem to be available, but they would be nice for when we need a WSL2 experiment environment.
• WIndows/WSL2-Docker Desktop: Not likely, as we have no control of integration with distros.
• Windows/Traditional: Can't work by default with the built-in Docker, which is old and set to windows containers.
• macos: Nothing seems to work for the same reason parallels and all other approaches don't seem to work, no nested virtualization.

[rfay]

Next steps:

• [ ] Run regular DockerCE WSL tests on Windows-2025. I'm pretty sure that the job of tb-wsl-11, tb-wsl-14, and tb-wsl-mirrored-16 could be replaced with GitHub workflow. So https://buildkite.com/ddev/wsl2-docker-inside (and also https://buildkite.com/ddev/wsl2-mirrored).

Unfortunately,

• macos is probably off the table for now for GitHub Actions.
• Traditional Windows testing may not be possible at this point because the windows-2025 has a built-in Docker that isn't configured in a way that we can use. It might be that there's.a way to override that and get it to use Linux containers and we might make progress then.