actions
Latest windows-server 2022/2025 pre-release image breaks Rust compilation
Description
Building a Rust program fails with:
linking with `link.exe` failed: exit code: 0xc0000409
This consistently reproduces for me when our GUI application is being built on the new Windows runners.
Platforms affected
- [ ] Azure DevOps
- [x] GitHub Actions - Standard Runners
- [ ] GitHub Actions - Larger Runners
Runner images affected
- [ ] Ubuntu 22.04
- [ ] Ubuntu 24.04
- [ ] macOS 13
- [ ] macOS 13 Arm64
- [ ] macOS 14
- [ ] macOS 14 Arm64
- [ ] macOS 15
- [ ] macOS 15 Arm64
- [ ] Windows Server 2019
- [x] Windows Server 2022
- [x] Windows Server 2025
Image version and build link
Windows Server 2022 (20250609) Image Update (https://github.com/actions/runner-images/releases/tag/win22%2F20250617.1)
https://github.com/firezone/firezone/actions/runs/15777748095/job/44475944088?pr=9590
Is it regression?
https://github.com/firezone/firezone/actions/runs/15765079094/job/44440022609
Expected behavior
To be able to compile and link Rust projects on the Windows runner.
Actual behavior
The linking step fails.
Repro steps
Compile and link a Rust project on the pre-release runner image.
Hi @thomaseizinger, Thank you for bringing this issue to our attention. We will look into this issue and will update you after investigating.
This is affecting windows-2025 now as well: https://github.com/firezone/firezone/actions/runs/15789902409/job/44513762369
Interestingly, this only appears to affect one particular workflow of ours. Other Windows workflows where we build Rust code work fine.
@thomaseizinger We have been following your repository and noticed that the job which previously failed on Windows Server 2022 is now passing, as seen in the GitHub Actions workflow below. Can you please confirm us if the issue is resolved. Thanks.
https://github.com/firezone/firezone/actions/runs/15821315684/job/44591093421
This is not the job that is failing. The job that is failing is: https://github.com/firezone/firezone/actions/runs/15816630435/job/44576867194
It appears to be a regression in the Microsoft compiler shipped with the new Visual Studio version that has been installed: https://github.com/rust-lang/rust/issues/141626
We seem to be affected by this too, over at https://github.com/NLnetLabs/domain/actions/runs/15872079664/job/44750833965?pr=491.
As a workaround, we are now using lld as suggested in https://github.com/rust-lang/rust/issues/141626#issuecomment-2919988483.
No issues so far but we still need to do further testing on the produced binaries.
@thomaseizinger, I can confirm that adding the linked .cargo/config.toml file fixes things for us. It might be a bit faster too, not sure.
Could this flag be set in the runner, so not every project has to experience a failure, search for the solution, then set the flag, then unset it later when MSVC is fixed?
Could this flag be set in the runner, so not every project has to experience a failure, search for the solution, then set the flag, then unset it later when MSVC is fixed?
I don't think this would be a good solution. But it may become the default from Rust's perspective in the future: https://github.com/rust-lang/rust/issues/71520
@thomaseizinger I hope your issue got resolved after adding the linker = "rust-lld". Could you please confirm on this.Thank you.
@thomaseizinger I hope your issue got resolved after adding the
linker = "rust-lld". Could you please confirm on this.Thank you.
Well, it is a workaround. The default linker shipped on the Windows images is stilly buggy and I wouldn't be surprised if C/C++ code is also affected by this. So strictly speaking, the issue is not resolved
Have you informed Microsoft that the linker in this VS version is buggy?
@thomaseizinger The new version of Visual Studio has been released. Please sync the main branch and try to run the build and confirm. Thanks.
I am not sure what you mean by "syncing the main branch". I tried running a build of our software with the standard linker that comes with the latest Windows runner image but it still fails: https://github.com/firezone/firezone/actions/runs/16317127782/job/46085637870
@thomaseizinger Thanks for the confirmation. Could you please try with the runner windows-2025 or windows-2022 Instead of xlarge and confirm.
The above run uses the latest windows-2025 image: https://github.com/actions/runner-images/releases/tag/win25%2F20250708.1
@thomaseizinger Could you please provide a smaller, simpler workflow that reproduces the issue.Thanks.
@thomaseizinger Could you please provide a smaller, simpler workflow that reproduces the issue.Thanks.
I don't unfortunately know exactly, what causes the problem. There are several other issues linked to this one already though so perhaps some of those applications are simpler to build than ours.
@thomaseizinger We have tested with one of the customer's repositories, and they are also experiencing the same link.exe issue. Please find the attached reference logs for both cases — with linker = "rust-lld and without linker = "rust-lld Both builds completed successfully.
And also go through the discussion on the Microsoft developer community for this issue. Thanks.
For us, it doesn't really matter which linker we use. rust-lld works fine for us so we are likely going to stick with it.
Reading the issue on the Microsoft developer community forum, it seems there is a solution to also change the symbol mangling format.
I am not sure what the path forward for you guys is then here. It seems MS is unable to provide a proper fix right now that does not involve changes in the source repo (either using rust-lld or changing compiler flags). So I guess this could be closed as unable to fix?
@thomaseizinger The root cause appears to be upstream, and it needs to be addressed there.Please refer to the following issues for further updates and tracking:Rust issue #141626 and Visual Studio Developer Community issue As this is an upstream issue, we are closing this issue. Thank you.