runner-images icon indicating copy to clipboard operation
runner-images copied to clipboard

The Windows 2019 Actions runner image will begin deprecation on 2025-06-01 and will be fully unsupported by 2025-06-30

Open subir0071 opened this issue 8 months ago • 24 comments

Breaking changes

We will soon start the deprecation process for Windows 2019. While the image is being deprecated, you may experience longer queue times during peak usage hours. Deprecation will begin on 2025-06-01 and the image will be fully unsupported by 2025-06-30.

To raise awareness of the upcoming removal, we will temporarily fail jobs using windows 2019. Builds that are scheduled to run during the brownout periods will fail. The brownouts are scheduled for the following dates and times:

  • June 3 13:00 - 21:00 UTC
  • June 10 13:00-21:00 UTC
  • June 17 13:00-21:00 UTC
  • June 24 13:00-21:00 UTC

For Azure DevOps users, more information can be found in this blog post: https://aka.ms/azdo-windows

Target date

Final retirement date: 2025-06-30

The motivation for the changes

We maintain the latest two stable versions of any given OS version. Windows 2025 is moved to GA on 2025-04-07 so we start deprecating the oldest image at that time.

Possible impact

Workflows using the windows-2019 image label should be updated to windows-2022 or windows-2025.

Platforms affected

  • [x] Azure DevOps
  • [x] GitHub Actions

Runner images affected

  • [ ] Ubuntu 20.04
  • [ ] Ubuntu 22.04
  • [ ] Ubuntu 24.04
  • [ ] macOS 13
  • [ ] macOS 13 Arm64
  • [ ] macOS 14
  • [ ] macOS 14 Arm64
  • [ ] macOS 15
  • [ ] macOS 15 Arm64
  • [x] Windows Server 2019
  • [ ] Windows Server 2022
  • [ ] Windows Server 2025

Mitigation ways

N/A

subir0071 avatar Apr 15 '25 17:04 subir0071

Hey looks like you got a spam issue.

Anyway, I wanted to ask: Migrating to windows-2022 I noticed that some headers are now missing. Looking at https://github.com/actions/runner-images/issues/9701 and https://github.com/actions/runner-images/issues/9873 there's a solution offered (which I managed to shrink down to: https://github.com/actions/runner-images/issues/9873#issuecomment-2833115951 )

But there's still a few open questions:

  1. Why Microsoft.VisualStudio.Component.VC.14.29.16.11 exactly, and how did @mikhailkoliada found that's the exact version number that was needed?
  2. Can projects be retargeted so that manually installing a VS component isn't needed? 2.1. If so, what should be the targets?
  3. Why does vs_installer need to be run twice? Each run takes ~4 minutes. So that makes every run job take an extra ~8 minutes. 3.1. Can this be cached between runs ?

Avasam avatar May 02 '25 15:05 Avasam

I've got ADO pipelines that are set to windows-latest that are now failing because of this brownout.

dombarnes avatar Jun 03 '25 14:06 dombarnes

I've got ADO pipelines that are set to windows-latest that are now failing because of this brownout.

This is a terrible idea btw. Shutting down key services on purpose for 8 hours at a time to raise awareness of an upcoming deprecation. I have a pipeline that uses windows-latest for all stages except on a simple deploy step that for whatever tries to use windows-2019 and now won't deploy

jobs:
  - deployment: DeployWebServicePortal
    displayName: "Deploy Web Service: Portal"
    environment: ${{ parameters.environment}}
    pool:
      vmImage: '${{ parameters.vmImage }}' // this is windows-latest
    variables:
    - template: variables.yml
    strategy:
      runOnce:
        deploy:
          steps:
          - download: current
            displayName: 'Download the pipeline artifacts'
            artifact: 'Portal'
          - task: AzureRmWebAppDeployment@4

dombarnes avatar Jun 03 '25 15:06 dombarnes

I think something went wrong, I'm using windows-2022 and still the the brown-out error message when trying to release to azure.

sato-code-admin avatar Jun 03 '25 15:06 sato-code-admin

And how to build XP in windows-2022 ? (How to install Visual Studio from windows-2019)

(v141_xp toolset for vs2022)

UnrealKaraulov avatar Jun 03 '25 15:06 UnrealKaraulov

I am using windows-latest. Still I am getting below error.

##[Error 1] This is a scheduled Windows Server 2019 brownout. The Windows Server 2019 image will be removed on 2025-06-30. For more details, see https://github.com/actions/runner-images/issues/12045

##[Error 2] The remote provider was unable to process the request.

febinDonz avatar Jun 03 '25 16:06 febinDonz

I am using windows-latest. Still I am getting below error.

##[Error 1] This is a scheduled Windows Server 2019 brownout. The Windows Server 2019 image will be removed on 2025-06-30. For more details, see #12045

##[Error 2] The remote provider was unable to process the request.

We are having the same error, as we were releasing to production... now we're stuck

maxlacroix-elv8 avatar Jun 03 '25 20:06 maxlacroix-elv8

We are having the same error, as we were releasing to production... now we're stuck

For anyone running into this problem, setting your image to windows-2022 should fix the issue. We also ran into the same problem, where windows-latest was grabbing a windows-2019 agent for our builds... sometimes.

Not ideal, but it at least gets you shipping for now, hopefully we see a fix for that issue soon?

twchapman avatar Jun 03 '25 20:06 twchapman

So from looking at some of our code and the example above, I think it might be related to using deployment jobs. We have regular jobs that work fine in the same build as deployment jobs that fail. Both using the same variable.

Salim-Khan-Techsmith avatar Jun 03 '25 20:06 Salim-Khan-Techsmith

So from looking at some of our code and the example above, I think it might be related to using deployment jobs. We have regular jobs that work fine in the same build as deployment jobs that fail. Both using the same variable.

That seems to be my summation too. I use one variable all the way through (usually passed as parameters to templates) and when I download the logs, the expanded yaml file has $(vmImage) where in there places its substituted 'windows-latest' instead.

dombarnes avatar Jun 03 '25 22:06 dombarnes

This is very sad news. I need to build for cuda 11.4 which is no longer possible in windows-2022. Does anyone else have ideas?

LostRuins avatar Jun 04 '25 01:06 LostRuins

Will this impact custom large runners using Windows Server 2019? Or is it quite literally the windows-2019 standard runner image only?

Example: Image

mattjcly avatar Jun 05 '25 14:06 mattjcly

How to use v141_xp build tools now without windows-2019 ?

UnrealKaraulov avatar Jun 05 '25 16:06 UnrealKaraulov

Will it effect self-hosted windows 2019 GitHub runner also or Github hosted runners? Thanks

rej007 avatar Jun 05 '25 17:06 rej007

Will this window 2019 image depreciation cause the already deployed server to go down, or it will be causing the deployment to fail next time when we do the deployment?

updesh-998 avatar Jun 07 '25 11:06 updesh-998

I have a pipeline that defines windows-latest as vmImage


variables:
  vmImage: 'windows-latest'

stages:
  - stage: Build
    displayName: Build and publish stage
    jobs:
      - job: Build
        displayName: Build and publish
        pool:
          vmImage: $(vmImage)

  - stage: Production
    displayName: Deploy to Production environment
    dependsOn: Build
    condition: and(or(succeeded(), and(succeeded('Build'), eq(variables.isHotfix, true))), ne(variables['Build.Reason'], 'PullRequest'))
    variables:
      webAppName: 'my-app-prod'
    jobs:
      - deployment: Deploy
        displayName: Deploy to production
        pool:
          vmImage: '$(vmImage)'
        environment: Production

but the pipeline is failing with this log.

##[error]This is a scheduled Windows Server 2019 brownout. The Windows Server 2019 image will be removed on 2025-06-30. For more details, see https://github.com/actions/runner-images/issues/12045
,##[error]This is a scheduled Windows Server 2019 brownout. The Windows Server 2019 image will be removed on 2025-06-30. For more details, see https://github.com/actions/runner-images/issues/12045
,##[error]The remote provider was unable to process the request.
Pool: [Azure Pipelines](https://dev.azure.com/mytenant/...)
Image: windows-2019-vs2019

Edit: it seems that https://github.com/actions/runner-images/issues/12045#issuecomment-2937157730 still relevant.

webartoli avatar Jun 10 '25 15:06 webartoli

Without VS2019 we can no longer target drivers for windows 7 Is there a workaround planned? e.g. offering an appropriate version of EWDK in the 2022 or 2025 runner?

DavidXanatos avatar Jun 12 '25 08:06 DavidXanatos

This feels really premature for Windows specifically, considering extended support for Windows 2019 is good until Jan 9 2029, and there is STILL no way to even run a Windows container on a Windows runner. At least for Linux runner deprecations, we can easily run a container if we need to do a bit of work for a while longer on an older platform.

lorengordon avatar Jun 17 '25 14:06 lorengordon

I've got ADO pipelines that are set to windows-latest that are now failing because of this brownout.

happening again today!

steve-nickerson avatar Jun 17 '25 16:06 steve-nickerson

We were having issues with our workflow running windows-2022 getting the brownout error. Turns out we had a comment lower in the YAML file that contained the string windows-2019 and that was causing it to error out. Hope this helps some people.

zbrown001 avatar Jun 17 '25 16:06 zbrown001

Hello! Does anybody know, are there any plans for adding following components to windows-2022 or windows-2025 images?:

  • Microsoft.VisualStudio.Component.VC.Tools.x86.x64
  • Microsoft.VisualStudio.Component.VC.ATL
  • Microsoft.VisualStudio.Component.VC.ATLMFC

I am struggling for the third day in a row trying to manage my pipeline to work and Im somehow desperate. Windows-2019 contained next components that were required for building our C++ ATL COM objects: https://github.com/actions/runner-images/blob/win19/20250609.1/images/windows/Windows2019-Readme.md Image

I am trying to install these components with visual studio installer as a pipeline step, but it seems not installed properly. But, if there are plans to upgrade some of supported VM images by the end of deprecation of windows-2019, maybe, I will just save my time.

dmitriyrudyka avatar Jun 18 '25 15:06 dmitriyrudyka

I don't mind the method of using brownouts to raise awareness, but as mentioned in https://github.com/UltraStar-Deluxe/USDX/issues/1022, it would be useful if the brownouts actually covered all the different timezones before simply unsupporting it.

As it is now, our European early-morning CI never caught any of them, which then gets compounded because the CI is also unlikely to be triggered by a push on a (European) Tuesday afternoon.

If you want to stick to 8-hour periods, consider for example the following:

  • June 3 00:00 - 08:00 UTC
  • June 10 08:00 - 16:00 UTC
  • June 17 16:00 - 00:00 UTC
  • June 24 (do whatever, why not the full 24 hours?)

That at least gives everyone around the world a worst case 2-week notice that something is going to break permanently, regardless of when their daily CI runs. As opposed to the situation caused by the solution used for the 2019 image, which is that July 1 is the first time the Windows part of our CI has failed at all.

barbeque-squared avatar Jul 01 '25 13:07 barbeque-squared

Closing this, as Windows-2019 deprecation is completed successfully.

subir0071 avatar Jul 02 '25 14:07 subir0071

Will there be any support for legacy apps that use dotnet 4? windows 2022 does not have dotnet 4 included and the dotnet-setup action also does not support dotnet framework

patrycja-bachleda avatar Jul 16 '25 15:07 patrycja-bachleda

Great, i have some windows containers that are fixed on a window-2019 server build using legacy sdks that cannot be updated, and now there is no way i can build projects in github, even as a paying github actions customer.

My life would be so much happier if i never had to deal with a single service microsoft provides

mlcruz avatar Jul 24 '25 20:07 mlcruz

Well, darnit - I am trying to setup CI for an application supporting windows xp to windows 11, and the later versions of the windows image no longer provides dotnet 4.0 support, and later versions of dotnet no longer supports windows XP.

Danielv123 avatar Oct 26 '25 16:10 Danielv123