opentelemetry-collector-releases
opentelemetry-collector-releases copied to clipboard
open-telemetry
Allow independent distribution releases
Our current release problem has a couple problems:
- We cannot release distributions independently. If we want to release a patch for the Contrib Distribution we have to also release a patch for the Core distribution.
- Related, if the job fails after 1 distribution is published but not the other, we can't rerun the job, we have to push new tags.
- We have run into diskspace issues in the past where the job fills up the allotted amount. The more we add to contrib and the more distributions we create the more this will be an issue.
As we look to support more distributions the connectedness of our CI will be a roadblock.
I propose we split the release of each distribution into it's own workflow with their own tags, such as v0.90.0-contrib or v0.90.0-core. The logic of each workflow can reuse a shared action, but I believe we'd need a goreleaser.yaml per distribution.
If each distribution has its own workflow we can:
- Release each distribution independently via their own tags. If necessary we can add a make target to simplify the Release Manager's steps.
- Each distribution will have more disk space (or any resource, including time) to work with.
Based on https://github.com/open-telemetry/opentelemetry-collector-releases/issues/357#issuecomment-1593537385.
I agree with that. I've been experimenting with splitting the distributions on their own workflows as well on a personal repository, and I prefer that approach than what I originally built in this repo here.
While I think it's great to have the ability to release different distributions separately, I don't think it's a good idea to do so: I think it might be confusing for users to have, say, otelcol-contrib-v1.0.1 and not an otelcol-core-v1.0.1 (and vice-versa).
Related, if the job fails after 1 distribution is published but not the other, we can't rerun the job, we have to push new tags.
Care to expand on this part? Is this only about the step where tags are pushed, or is it something else? I think the release should already be idempotent for most steps.
When there's an issue with a release, we delete the tags and push them again. There's a note about tags in the README:
⚠️ Important note: Git tags in this repository may change at any time to fix any issues found during a release. They are only meant to trigger Github releases and should not be relied upon.
I agree that there needs to be separate workflows for the distros, otherwise we're limited in what we can release in the future. We've already seen this in the past.
Re. different versions for different distros, I'm not sure I agree that we should always keep the different versions in sync. I'm thinking of the world of the future where most of the components in core are stable and no longer need to release on a frequent cadence. This would mean that we could end up in a scenario where core and contrib are at different versions and that should be perfectly acceptable.
I understand that's not where we are today, but I also don't see this as a dealbreaker when it comes to how releases should be made.
Care to expand on this part? Is this only about the step where tags are pushed, or is it something else? I think the release should already be idempotent for most steps.
I've not looked into it in depth, but if the action goreleaser/goreleaser-action@v5 we use for publishing has an error it may be possible to push 1 image but not others. This is currently a theoretic issue.
It sounds like we are in agreement to split up the release jobs. I could get started on that as we continue to discuss tagging.
Spent some time on this today and ran into this problem: https://github.com/orgs/goreleaser/discussions/4469. A workaround would be to use multiple goreleaser configuration files, but I'd like to avoid that if possible.
@jpkrohling looking at your repo I do like how there are individual gorelease configs. This will be really useful for certain distributions, like k8s or sidecar, that don't need to build specific images/binaries.
If we went that route I think we'd no longer auto-generate the file, are you ok with that?
I'm fine with that, but I'd do it as part of a next step, so that we have smaller changes applied at each iteration.
If we went that route I think we'd no longer auto-generate the file, are you ok with that?
Can you clarify this part? I see benefit in still generating this, but perhaps you found limitations around that?
I was wrong, we can still generate the files like you do in your repo.
@jpkrohling here is a PR for what the generated files can look like: https://github.com/open-telemetry/opentelemetry-collector-releases/pull/486.
I think the auto-generated files will break down if we ever make any specific distributions, like a Windows distribution, but it is possible that we can adjust the code for those cases. For now, generating the files ensures the distributions are the same.