pipeline
pipeline copied to clipboard
tektoncd
Pipeline release.notags.yaml would not install on OpenShift
Expected Behavior
Be able to install release.notags.yaml on OpenShift
Actual Behavior
LAST SEEN TYPE REASON OBJECT MESSAGE
20s Warning FailedCreate replicaset/tekton-pipelines-controller-6df78f8d67 Error creating: pods "tekton-pipelines-controller-6df78f8d67-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 65532: must be in the ranges: [1000600000, 1000609999]]
61s Normal ScalingReplicaSet deployment/tekton-pipelines-controller Scaled up replica set tekton-pipelines-controller-6df78f8d67 to 1
19s Warning FailedCreate replicaset/tekton-pipelines-webhook-59668cf699 Error creating: pods "tekton-pipelines-webhook-59668cf699-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 65532: must be in the ranges: [1000600000, 1000609999]]
61s Normal ScalingReplicaSet deployment/tekton-pipelines-webhook Scaled up replica set tekton-pipelines-webhook-59668cf699 to 1
Steps to Reproduce the Problem
Trying to install the .release.notags.yaml on OpenShift (which is the release.yaml that needs to be used for openshift) would fail due of the runAsUser "forcing".
% kubectl create -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.notags.yaml
Additional Info
- Kubernetes version:
% oc version
Client Version: 4.5.16
Server Version: 4.5.7
Kubernetes Version: v1.18.3+2cf11e2
- Tekton Pipeline version:
Latest as of today. 0.17.1
Both release.yaml or release.notags.yaml share the same problem.
We need to either:
- do a
release.openshift.yamlthat removes therunAsUser - document how to install on OpenShift (either a
sedoranyuid)
The red hat openshift-tekton team (us :-)) provides a mid-stream release which let's pipeline release.yaml install on openshift to this url :
https://github.com/openshift/tektoncd-pipeline/blob/release-v${version}/openshift/release/tektoncd-pipeline-v${version}.yaml
i.e:
https://github.com/openshift/tektoncd-pipeline/blob/release-v0.17.1/openshift/release/tektoncd-pipeline-v0.17.1.yaml
it does a sed to remove the runAsUser but that's not the only thing (altough the other things are not an absolute requirement to do compared to the removal of runAsUser) :
https://github.com/openshift/tektoncd-pipeline/blob/master/openshift/resolve-yamls.sh#L29
I think it would be great if we can have the generation of the release.openshift.yaml with that script as part of the release, we could point openshift user to use the "mid-stream" release too but we would need a tight sync between the upstream and mid stream release.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle stale
Send feedback to tektoncd/plumbing.
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle rotten
Send feedback to tektoncd/plumbing.
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/close
Send feedback to tektoncd/plumbing.
@tekton-robot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity. Reopen the issue with
/reopenwith a justification. Mark the issue as fresh with/remove-lifecycle rottenwith a justification. If this issue should be exempted, mark the issue as frozen with/lifecycle frozenwith a justification./close
Send feedback to tektoncd/plumbing.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Documentation here https://tekton.dev/docs/getting-started/ let the user thinks he can install it on OCP. I tried with OpenShift 4.8.11 and it didn't work. Should we update the documentation somehow?
Current good command is: kubectl apply --filename https://raw.githubusercontent.com/openshift/tektoncd-pipeline/release-v0.24.3/openshift/release/tektoncd-pipeline-v0.24.3.yaml
As of today the documentation is still not updated and the instructions here https://tekton.dev/docs/installation/pipelines/#installing-tekton-pipelines-on-openshift still result failures which is quite confusing given they are pretty short and straightforward.
/assign
We need to update the documentation on installing it on OpenShift. Basically, because of the securityContext.runAsUser, without any change, none of the release.yaml are installable "as is" on OpenShift.
https://github.com/tektoncd/website/pull/511 for fixing the documentation part.
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/close
Send feedback to tektoncd/plumbing.
@tekton-robot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity. Reopen the issue with
/reopenwith a justification. Mark the issue as fresh with/remove-lifecycle rottenwith a justification. If this issue should be exempted, mark the issue as frozen with/lifecycle frozenwith a justification./close
Send feedback to tektoncd/plumbing.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@vdemeester: Reopened this issue.
In response to this:
/reopen
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
This is still relevant, the documentation is not updated . The error is partly:
.. is forbidden: unable to
validate against any security context constraint:
[pod.metadata.annotations[container.seccomp.security.alpha.kubernetes.io/tekton-pipelines-controller]:
Forbidden: seccomp may not be set
Comment from Guillaume seems to work around it. This issue is so common, it should be clearly documented : https://tekton.dev/docs/installation/pipelines/
@vdemeester I'm working on https://github.com/tektoncd/pipeline/pull/6515, which adds this security context to taskrun pod init containers. @dibyom suggested gating this behind feature flag for the same reason as reported in this issue (openshift problems). I think it would be nice to give users just 1 flag that controls setting securityContext for both the init containers and the pipelines controller, but it seems like this issue needs to be addressed at the release yaml level. Maybe an openshift-specific release yaml could turn off the feature flag that controls the behavior in https://github.com/tektoncd/pipeline/pull/6515? Or maybe the feature flag should just be off by default regardless of how pipelines is installed?
@vdemeester I'm working on #6515, which adds this security context to taskrun pod init containers. @dibyom suggested gating this behind feature flag for the same reason as reported in this issue (openshift problems).
I would suggest doing the same, not necessarily due to openshift, but in general (because it "might" affect ppl), so having a time so users can have a bit of time to adjust might be good.
I think it would be nice to give users just 1 flag that controls setting securityContext for both the init containers and the pipelines controller, but it seems like this issue needs to be addressed at the release yaml level.
I am not sure I follow too much. Having securityContext on the controller(s) is very different to having init container for users workloads (TaskRun's Pod), so I don't really understand having those both with the same "configuration". We already have securityContext on controller's as well, so there is nothing to change there is it ?
Maybe an openshift-specific release yaml could turn off the feature flag that controls the behavior in #6515? Or maybe the feature flag should just be off by default regardless of how pipelines is installed?
Having it configured would, more or less, be all we need to be able to force it to false (or to something that works with OpenShift) I think. Otherwise we would need to patch it. But we don't really need an openshift specific release of pipeline, this is where the operator and openshift-pipelines are for. Today, the upstream release.yaml cannot run/be installed on OpenShift already, at least without any changes (see https://github.com/tektoncd/website/pull/511).
Having the same type of options as tektoncd/triggers had makes sense to me.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle stale
Send feedback to tektoncd/plumbing.
Given that https://github.com/tektoncd/pipeline/pull/6515 is merged, I will close this.