runner-images
runner-images copied to clipboard
actions
The Ubuntu 18.04 Actions runner image will begin deprecation on 2022/08/08 and will be fully unsupported by 2023/04/01
Breaking changes
We have started the deprecation process for Ubuntu 18.04. While the image is being deprecated, You may experience longer queue times during peak usage hours. Deprecation will begin on 2022/08/08 and the image will be fully unsupported by 2023/04/01
To raise awareness of the upcoming removal, we will temporarily fail jobs using Ubuntu 18.04. Builds that are scheduled to run during the brownout periods will fail. The brownouts are scheduled for the following dates and times:
GitHub Actions\Azure DevOps::
- October 3, 12:00 UTC – October 3, 14:00 UTC
- October 18, 14:00 UTC – October 18, 16:00 UTC
- November 15, 18:00 UTC – November 15, 20:00 UTC
- November 30, 20:00 UTC – November 30, 22:00 UTC
- December 15, 20:00 UTC - December 16 00:00 UTC
- January 5, 10.00 UTC - January 5, 14.00 UTC
- January 13, 12.00 UTC - January 13, 16.00 UTC
- January 18, 14.00 UTC - January 18, 18.00 UTC
- January 24, 16.00 UTC - January 24, 20.00 UTC
- February 1, 18.00 UTC - February 1, 22.00 UTC
- February 7, 16.00 UTC - February 7, 22.00 UTC
- February 13, 14.00 UTC - February 13, 22.00 UTC
- February 21, 10.00 UTC - February 21, 22.00 UTC
- February 28, 10.00 UTC - February 28, 22.00 UTC
- March 6, 00.00 UTC - March 7, 00.00 UTC
- March 13, 00.00 UTC - March 14, 00.00 UTC
- March 21, 00.00 UTC - March 22, 00.00 UTC
Target date
April 1st, 2023
The motivation for the changes
We maintain the latest two stable versions of any given OS version. Ubuntu 22.04 is going GA on 8/8/22 thus we start deprecating the oldest image.
Possible impact
Workflows using the ubuntu-18.04 image label should be updated to ubuntu-latest, ubuntu-20.04, or ubuntu-22.04
Platforms affected
- [X] Azure DevOps
- [X] GitHub Actions
Virtual environments affected
- [X] Ubuntu 18.04
- [ ] Ubuntu 20.04
- [ ] Ubuntu 22.04
- [ ] macOS 10.15
- [ ] macOS 11
- [ ] macOS 12
- [ ] Windows Server 2019
- [ ] Windows Server 2022
Mitigation ways
N\A
Don't get me wrong, I fully understand that you're deprecating 18.04 - but incorporating brownout periods to raise awareness?
If my team needs to get a build out for a critical bugfix on the 22nd of August... I'm just going to have to wait for 4 hours? If my understanding of that is true, I find that very odd behavior in an enterprise environment. Some businesses actually have SLAs with their clients. Would kindly request the team to find another way to "raise awareness". I came here because of the red banner when using the image, that worked fine as well and is a lot less invasive.
You're giving us two weeks to migrate away before brownouts start. Migrating is an action that requires testing and validation.
Does this impact hosted agents only, or does this also mean the agent tooling/infra (for self hosted pools, but also for 1ES pools for internal folks) will become EOL for ubuntu18? My team still needs to keep shipping ubuntu18 versions of our product for years.
I am sad to see this removal of ubuntu-18.04 and hope you will reconsider. I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases. Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.
I am sad to see this removal of ubuntu-18.04 and hope you will reconsider. I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases. Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.
@smistad as per the policy we only support the two latest LTS
I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases. Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.
You can use container: option to build with ubuntu:18.04 image or statically link binaries (or install Ubuntu 18.04 sysroot on new runners and build with that).
I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases. Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.
You can use
container:option to build withubuntu:18.04image or statically link binaries.
Thanks for the tip @panekj
Deprecation will begin on 8/8/2022 and the image will be fully unsupported by 12/1/2022
Should be 12/1/2023 or is it a US format date?
@smistad as per the policy we only support the two latest LTS
@mikhailkoliada Is there any way we can convince GitHub to change this policy to the latest three LTS releases? We actually switched from 20.04 to 18.04 a few months ago because we found it worked better for our use cases.
What about projects which still need python 3.3 support for a reasonable time?
see: https://github.com/wbond/package_control/runs/7884696802?check_suite_focus=true
You must not simply break existing workflows!!!!
Ubuntu 18.04 does not reach the end of its standard support phase until April 2023. If the policy here is to support the last two LTS releases, I would suggest it be amended to align dropping support when Ubuntu does.
And another vote for 2 weeks before the brownouts start being very short.
Ubuntu 18.04 does not reach the end of its standard support phase until April 2023. The AppImage project is relying on it to be available on GitHub Actions for the whole timeframe doring which it is supported.
There seems to be quite a discovery issue with the brown out. I hadn't seen this issue originally which isn't the biggest deal, but seeing CI jobs fail with no context whatsoever is very confusing.
All I saw was a "This check was cancelled" with no context whatsoever about why it was cancelled. No message that there's a brown out, no link to this issue etc.
Somehow I was able to navigate to a different page of a status check that finally showed that information. Is it possible to make this more visible and prominent so it can be found easier?
It seems to not be visible if you click "Details" from a pull request. Only when manually navigating from the "Actions" tab can this be found.
So yeah, we've just found out that 18.04 is deprecated. Considering I just upgraded to Enterprise at the cost of the order of thousands of dollars a month, I'm not massively pleased to see you just breaking our whole company's jobs for four hours.
Couldn't this be done in a different way? A banner on jobs that use the image? An email to org admins? It's pretty bad behaviour.
A banner on jobs that use the image?
What can I say? A whole company of 20 people didn't notice it.
There is also https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22
There is also https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22
I'm sure there is. But we pay for this. You (as in GH, not you personally) need to be proactive about reaching out to customers when you're planning to break something. The rest of Microsoft is pretty good at this.
This brownout scheme is the worst way to "raise awareness for deprecation" and has put a dark tint over our otherwise happy relationship with GitHub. Who figured that this would be a good solution? We sure disagree.
@miketimofeev Brownout is pretty extreme, you are getting in the way of people shipping and being able to work. And stopping for service we are paying for.
This is huge loss of trust, and pushing us to consider alternatives.
👋 Thank you all for expressing your concerns with the deprecation schedule and feedback on preferred availability. We will revise the deprecation schedule and are will immediately stop the brownouts.
We are working on a complex migration with a pretty tight deadline (tomorrow). We have noticed the deprecation and planned for it accordingly. Running into a random brownout is the worst thing any tool could do to us at the moment. Yes, we are aware of the deprecation. Yes, we have planned to make the necessary changes when we are able to. You are just adding insult to an injury - not raising awareness. Much disappointment. I thought microsoft would have learned from its passed mistakes - but apparently we are still in the Windows SE area. That's the kind of awareness you are raising. And now I am pretty sure I'll keep my other projects over at GitLab.
This is impacting AWS SDK For Ruby CI - I agree with others that this is a poor way to raise awareness, as this is the first time I've heard of this campaign. We can make steps to migrate.
Please come up with a more reasonable way to handle deprecation of the runners.
Consider how AWS handles lambda runtime deprecations as an example. They distribute emails when old runtimes are being used. You have access to all the workflows so you know whos using them.
With a three week notice period, introducing these brownouts is like pulling the deprecation date forward. Because you can’t rely on a production system that breaks deployments every week for 6 hours. Not being able to perform an action during this time is Ill-advised and a serious risk for anyone relying on GitHub actions for production deployments.
It’s like putting a house on fire to raise awareness that playing with matches is dangerous.
See https://www.linkedin.com/posts/jansch_github-cicd-activity-6967561507354120192-084T?utm_source=share&utm_medium=member_ios for how this lead to a serious production incident at one of our customers when we needed to roll back a release because of a bug.
At least make this a setting so we can only do this ‘Forced awareness’ on test/acceptance builds. A feature like this on production is just killing reliability or GitHub actions for serious production work.
We got hit by this brownout yesterday but nothing told us our jobs were getting cancelled because of the brownout, I found this issue purely by chance. If you're going to do something like this you really need to give an obvious error message and explain why jobs are being cancelled, just seeing the job being cancelled made me think there was an outage. Even our github account manager wasn't aware of this brownout that was happening.
Also, last night, I also experienced this brownout , but there is no explanation, I feel a little baffling.
Too sloppy.
Don't get me wrong, I fully understand that you're deprecating 18.04 - but incorporating brownout periods to raise awareness?
If my team needs to get a build out for a critical bugfix on the 22nd of August... I'm just going to have to wait for 4 hours? If my understanding of that is true, I find that very odd behavior in an enterprise environment. Some businesses actually have SLAs with their clients. Would kindly request the team to find another way to "raise awareness". I came here because of the red banner when using the image, that worked fine as well and is a lot less invasive.
You're giving us two weeks to migrate away before brownouts start. Migrating is an action that requires testing and validation.
We had the exact same problem when windows-2017 was deprecated, and it was extremely painful to have this transition forced unto us.
IMHO this one is particularly bad since there are legitimate reasons to build on older versions of Ubuntu; namely, so that the linker won't pull in symbols from newer versions of glibc which the customer's systems don't have.
The deprecation date has shifted to March 1st, 2023 to give more time to adapt the workflows to Ubuntu20 or Ubuntu22. The brownout schedule is also adjusted and now has shortened periods.