runner
runner copied to clipboard
actions
Support YAML Anchors
Note to incoming readers (2024)
~~This issue has been open for several years, and despite commitment from staff that it would be done there has been no progress.~~ I will update this if anything changes, you don't need to read this entire thread.
UPDATE Aug 2024: The latest comment from staff says they will be looking at this next year
Thanks for engaging on this all, this is on our list to get to in the early part of next year. Right now we are re-working the services in Actions which do our YAML expansion (focusing on how we improve their availability). Once we are live with the new services we will start looking at adding features and this is on our list <3
If you want to help, upvote this comment by reacting to it with 👍. This will help it sort higher in the issues list.
Adding "me too" or "+1" comments does not help. Issues are not sorted by the number of comments, they are sorted by the number of reactions to the top issue comment (this comment). The only thing you accomplish by adding "me too", or "+1" is to send an email to issue subscribers (the other people who have commented, not staff). This will not help the issue get done faster, it does not get seen by GitHub staff.
Original Issue
Describe the enhancement Support YAML anchors in in workflow files
Code Snippet
- env: &docker_cred
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
# ...
- env:
<<: *docker_cred
OR
anchors:
docker_hub_credentials: &docker_hub_credentials
credentials:
username: my-dockerhub-username
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
postgres_service: &postgres_service
postgres:
image: mdillon/postgis:9.6
<<: *docker_hub_credentials
Additional information This has been asked for before, in other places
Note to implementors
This is generally handled by YAML parsers automatically. Perhaps it has been disabled, or a parser that does not support it is being used. This might be as simple as switching YAML parsers.
3+ years of people complaining and u don't take any action? Either, say u won't do or do it already. not a big deal to implement.
This is truly absurd. I think that, given the existence of so many YAML parsers, it is harder to not support anchors and merge keys than doing it. Just let a normal YAML parser run.
@DanySK (and to a lesser degree @freitasmurillo), while I appreciate the support on issues, since upvotes help show priority, I don't think hurling abuse at the maintainers or anyone else is going to get this done any faster. If either of you are professional software developers I'm sure you can relate to working on project's whose feature requests exceed your team's bandwidth. There are 300 open issues on this repository and some of them describe actual bugs. Surely this is not the highest priority ticket.
Please keep in mind there is a human on the other side reading these messages. Consider how you might react to someone who said as much to you.
Hi @kyeotic, I have no intention to be rude to people, much less to those whose work benefits the FOSS community so much. I expressed a technical concern that is puzzling me from a software engineering perspective, and that you also highlighted in the first post. I did not think that could hurt others -- usually, there is either some deep issue behind it, or it could just be a matter of applying small correctives.
That said, yes I know that it's impossible to keep up with the pace of feature requests in many cases, and especially so for FOSS. Given that, if someone can help me figure out where the YAML parsing happens, I'm willing to help.
The error message appears in YamlObjectReader.cs which references YamlDotNet.Core (and the csproj references version 5.3.0), so I expect that's the parser.
They're a few major versions behind, but it looks like there are bugs even on the latest version, e.g. this one from August. I'm not sure if that can be ignored, because they're not using the serialization from YamlDotNet, only the parser, and they've written their own custom code on top of the library.
So in short, I think it's a custom parser, written on top of another parser. The parser looks very local in how it parses things, so supporting anchors might require a bit of a rearchitect, which would explain why it's been open for so long.
@kyeotic sorry for being late to get your response. I definitely wasn't abusing anyone, sorry if that is what it sounded. Not having English as my primary language probably helped a lot. 😞
What I was trying to convey is that 3 years without replying seemed to me a lot, u know? This feature would be amazing and I know that they have a lot but I think once in a while they could provide updates and let people know what is on their plate or not for the moment.
If they don't have a minimal clue on if they will do or not post it on the thread. 💥
As much as we love GitHub Actions, the lack of this really makes us think about to switch back to GitLab. Re-using steps shouldn't be that complicated to work with (composite actions, etc. aren't really comparable to in-file templates).
Right now, GitHub doesn't seem to focus on their enterprise and power users, which need to work with complex workflows. It's understandable, as GitHub is very much more driven by the open-source community, but it nevertheless doesn't change the fact, that a lack of something like this, is very frustrating for enterprise users.
It simply makes it hideous to build complex workflows without a feature like that, even though GitHub has the best fundaments for building complex workflows in comparison to any other CI/CD platform.
When we evaluated both, GitHub and GitLab, we always expected in-file templating support to be released very soon for GitHub and that's why we never really considered the lack of this, as a decision defining point in the outcome of our evaluation. But looking back to it from today, it would have definitely changed the outcome of our evaluation, without any doubt.
GitLab CI / CD supports this since more than like three years already, and I haven't seen anyone, who didn't use it for their pipeline definition files. In recent time, they supported this without any anchors, which made it even easier to work with.
I would really love to see this being supported here as well.
Right now, GitHub doesn't seem to focus on their enterprise and power users, which need to work with complex workflows. It's understandable, as GitHub is very much more driven by the open-source community, but it nevertheless doesn't change the fact, that a lack of something like this, is very frustrating for enterprise users.
@aramfe, your wording seems to imply that enterprise and power users have more complex requirements than the open-source community. I'd say that most enterprise and power users in GitHub are producing open-source software. Thus, that's a false dicotomy.
I'd say that most enterprise and power users in GitHub are producing open-source software
I don't know if you have any data to back this up, but I would be very, very surprised to learn that this was true. I expect most enterprises, even those using GitHub, to be developed closed-source, proprietary software.
@kyeotic it's an opinion, based on the enterprises I know which use GitHub. Nevertheless, the point holds regardless of quantitative data: licensing of a software project is unrelated to the complexity. Moreover, the difference between enterprise users and others is that enterprise clients pay GitHub, not the licenses they use or the complexity of their projects.
FTR:
mithro/actions-includes allows to work around the lack of YAML anchor support, by preprocessing Workflow files.
Alternatively, in hdl/containers, a Reusable Workflow is combined with a Python script in order to dynamically generate the matrix by processing a YAML file with anchors.
- The Reusable Workflow: common.yml.
- The YAML file: config.yml (see section
jobs) - Calling the Reusable Workflow in a regular Workflow, and passing the
keyto decide which job matrix to generate: boolector.yml. - Wrapping the Reusable Workflow in a dispatchable Workflow: dispatch.yml. See pyTooling/Actions: Callable vs dispatchable workflows.
A similar approach is used in the Params Reusable Workflow of pyTooling/Actions: Params.yml.
I agree, we need to do this. In the meantime, we've made a number of improvements in simplifying complex workflows, like adding reusable workflows. There are some tricky bits about this implementation - as we have multiple components that deal with workflows (it's not just the runner in this repo) - but this is on the backlog. I don't have an ETA, but yes, we will do this.
Thanks everybody for the feedback.
This is a desperately needed feature.
Without it, basic things like abstracting out a single static value requires the use of the env key, which is not supported across all of the GitHub Actions format.
Coming from other CI/CD platforms such as Bitbucket Cloud pipelines, this is a must-have feature. Especially, since it's been defined in the YAML specification for more than 10 years (anchors and aliases)
Hope this gets implemented soon!
Thanks
To be honest, the only way to persuade the team to implement this is to tweet about it. That is one of the most basic parts of YAML specification which they decided not to bother to implement when they went DIY instead of using an existing open-source implementation for the loader.
In fact the claim of being YAML is really shaky. It should be named partial-yaml, yaml-like, subset-of-yaml or crippled-yaml but not yaml.
@ssbarnea Please do not do that, that's horrible. Everyone, please consider that the "me too" comments are not useful. Issues are not sorted by the number of comments, and they provide noise for those following this thread in hopes of a response from GitHub. Upvote the original comment and leave it at that.
Do not harass GitHub employees. Do not harass anyone. Be kind.
Nobody was named here but the reality that this was part of the specification for more than 15 years, long before even the first version of GH actions was made available. From day one it was known that yaml supports anchors for reducing duplication, and that feature is not an unsafe one.
What it would really be nice is to have a sincere official reply that would state if there is planned for work to address this is like next year or not. So far I did not see anything in that area even it lots of people were quite surprised that they cannot use what is standard yaml syntax.
Again, I do understand that there are no guarantees but some transparency would likely be just what others are looking for.
shame on twitter or similar social medium
@ssbarnea dude, you've literally said people should go and hunt down GitHub staff and harass virtually. How you are not seeing what is wrong with that is beyond wild.
Nobody was named here but the reality that this was part of the specification for more than 15 years, long before even the first version of GH actions was made available. From day one it was known that yaml supports anchors for reducing duplication, and that feature is not an unsafe one.
What it would really be nice is to have a sincere official reply that would state if there is planned for work to address this is like next year or not. So far I did not see anything in that area even it lots of people were quite surprised that they cannot use what is standard yaml syntax.
Again, I do understand that there are no guarantees but some transparency would likely be just what others are looking for.
You're even trying to justifying it now. It's a CI/CD system, no one needs to be harassed because it doesn't have a feature you'd like.
shame on twitter or similar social medium
@ssbarnea dude, you've literally said people should go and hunt down GitHub staff and harass virtually. How you are not seeing what is wrong with that is beyond wild.
Nobody was named here but the reality that this was part of the specification for more than 15 years, long before even the first version of GH actions was made available. From day one it was known that yaml supports anchors for reducing duplication, and that feature is not an unsafe one. What it would really be nice is to have a sincere official reply that would state if there is planned for work to address this is like next year or not. So far I did not see anything in that area even it lots of people were quite surprised that they cannot use what is standard yaml syntax. Again, I do understand that there are no guarantees but some transparency would likely be just what others are looking for.
You're even trying to justifying it now. It's a CI/CD system, no one needs to be harassed because it doesn't have a feature you'd like.
I redacted the initial comment. I should have read it twice before posting as I seen much later that it could be interpreted wrongly. If anyone was offended, please accept my apology. I never intended to suggest that tagging individuals on twitter might be a good idea to achieve something, most likely only to annoy them.
It appears that this vital feature is not on the public roadmap yet: https://github.com/orgs/github/projects/4247/views/1?filterQuery=label%3Aactions+
Any update on this? It's a pretty fundamental YAML concept that's available on other CI systems, for example this works great in GitLab: https://docs.gitlab.com/ee/ci/yaml/yaml_optimization.html
@sammcj there was no significant update in almost three years. This feature was requested as soon as GitHub Actions were released: https://github.com/actions/starter-workflows/issues/245. Meanwhile, workarounds such as composite actions or reusable workflows have been implemented, which are double betting on doing their own stuff rather than reusing standard solutions.