pipeline
pipeline copied to clipboard
tektoncd
Add support for TaskRef in Tekton bundles
Feature request
In tekton bundles (which is still in alpha), one cannot use taskRef in a bundled pipeline. This is the case if the referenced task is in the same file as the pipeline yaml (see use case 1), and also if the task is in a separate file within the same bundle (see use case 2).
When trying to create a pipelineRun that references the bundle, this error is encountered:
Pipeline /bundle-pipeline can't be Run; it contains Tasks that don't exist: Couldn't retrieve Task "bundle-task1": tasks.tekton.dev "bundle-task1" not found`
Use case 1: Task in same file as pipeline
pipeline & task yaml published as a bundle:
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: bundle-pipeline
spec:
tasks:
- name: my-task1
taskRef:
name: bundle-task1
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: bundle-task1
spec:
steps:
- name: task-one-step-one
image: centos
command: ["bash", "-c"]
args:
- echo "=======================";
echo "Bundle Task1";
echo "=======================";
echo "IBM Cloud - Status - https://cloud.ibm.com";
echo "=======================";
sleep 10;
alive=$(curl -s -o /dev/null -L -w %{http_code} https://cloud.ibm.com);
if [ "$alive" == "200" ];
then echo "TEST PASS - 200"; exit 0;
else echo "TEST FAIL - $alive"; exit 1;
fi
PipelineRun that uses the bundle:
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: sample-pipelinerun-
spec:
pipelineRef:
name: bundle-pipeline
bundle: index.docker.io/beglee/refbundle@sha256:2f8086aadeb35c4e685b0e729369a355139491447c1ed2e42f222ce93b2325ec
params: []
Above example published to docker hub: https://hub.docker.com/layers/175205644/beglee/refbundle/0.1/images/sha256-2f8086aadeb35c4e685b0e729369a355139491447c1ed2e42f222ce93b2325ec?context=repo
Use case 2: Task and pipeline in separate files within the same bundle
pipeline.yaml:
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: bundle-pipeline
spec:
tasks:
- name: my-task1
taskRef:
name: bundle-task1
task.yaml:
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: bundle-task1
spec:
steps:
- name: task-one-step-one
image: centos
command: ["bash", "-c"]
args:
- echo "=======================";
echo "Bundle Task1";
echo "=======================";
echo "IBM Cloud - Status - https://cloud.ibm.com";
echo "=======================";
sleep 10;
alive=$(curl -s -o /dev/null -L -w %{http_code} https://cloud.ibm.com);
if [ "$alive" == "200" ];
then echo "TEST PASS - 200"; exit 0;
else echo "TEST FAIL - $alive"; exit 1;
fi
Thanks for this report @briangleeson and sorry about the late follow-up. I think having this feature is critical to the usability of bundles.
I also wonder if we should decouple the concept of bundle from the OCI backend, and make it a general feature that can be implemented by various remote backends when they become available.
/cc @sbwsg @vdemeester
@briangleeson Thanks for raising this issue. Indeed, I tend to agree with @afrittoli, this is a key feature to support to make bundle more usable. On decoupling I wonder if that can be tackled independently as, in the end, we are just "shipping" a Task/Pipeline spec in the bundle. As long as we lookup for oci bundle and/or later one the different remote backends, it should just work, 🤔 ?
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle stale
Send feedback to tektoncd/plumbing.
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle rotten
Send feedback to tektoncd/plumbing.
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/close
Send feedback to tektoncd/plumbing.
@tekton-robot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity. Reopen the issue with
/reopenwith a justification. Mark the issue as fresh with/remove-lifecycle rottenwith a justification. If this issue should be exempted, mark the issue as frozen with/lifecycle frozenwith a justification./close
Send feedback to tektoncd/plumbing.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
/reopen This is something we should still make sure is supported by TEP-0060 remote resource resolution if that's going to replace the experimental bundle support.
@AlanGreene: Reopened this issue.
In response to this:
/reopen This is something we should still make sure is supported by TEP-0060 remote resource resolution if that's going to replace the experimental bundle support.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle stale
Send feedback to tektoncd/plumbing.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.
/lifecycle stale
Send feedback to tektoncd/plumbing.
I think the same feature would be desirable when using git resolvers. Should that be a separate issue, or is the solution for tekton bundles expected to cover both cases?
@lcarva given it's a different resolvers, most likely another feature request/issue but maybe the solution would be similar to both.
Hello! I think the same issue is reproducible in Cluster Resolver. I am creating a Pipeline and Task in one namespace called hello-world
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: echo
namespace: hello-world
spec:
description: "Just hello world task"
params: []
results: []
steps:
- name: echo
image: registry.redhat.io/ubi8/ubi-minimal@sha256:574f201d7ed185a9932c91cef5d397f5298dff9df08bc2ebb266c6d1e6284cd1
script: |
#!/usr/bin/env bash
set -euo pipefail
sleep 25
echo "Sleep completed"
echo "Hello World"
---
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: echo
namespace: hello-world
spec:
params: []
tasks:
- name: echo
taskRef:
name: echo
kind: Task
params: []
Then I'm referencing the above Pipeline using Cluster Resolver by running this PR in different namespace:
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: test
spec:
pipelineRef:
resolver: cluster
params:
- name: kind
value: pipeline
- name: name
value: echo
- name: namespace
value: hello-world
The Cluster resolver seems to be able to find the PipelineRun definition but not about its tasks. Error captured:
status:
completionTime: "2024-06-28T12:10:42Z"
conditions:
- lastTransitionTime: "2024-06-28T12:10:42Z"
message: 'Pipeline hello-world/echo can''t be Run; it contains Tasks that don''t
exist: Couldn''t retrieve Task "echo": tasks.tekton.dev "echo" not found'
reason: CouldntGetTask
status: "False"
type: Succeeded
But if I manually create the Task echo inside the PR namespace, then PR is successfully able to run the pipeline. So Cluster Resolver is resolving the Pipeline from the right namespace but not of its tasks.
After tinkering around, I did find a way to run the PipelineRun from my previous example.
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: echo
spec:
params: []
tasks:
- name: echo
taskRef:
resolver: cluster
params:
- name: kind
value: task
- name: name
value: echo
- name: namespace
value: hello-world
---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: test
spec:
pipelineRef:
name: echo
The Pipeline object should contain the cluster resolver reference to the source task, then we can create PipelineRun for this object. Seems like we cannot directly refer to Pipelines from different namespace using Cluster Resolver in PipelineRun as per the example - https://tekton.dev/docs/pipelines/cluster-resolver/#pipeline-resolution
@rh-rahulshetty it would also work if the echo pipeline is in the hello-world namespace.
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: echo
namespace: hello-world
spec:
params: []
tasks:
- name: echo
taskRef:
resolver: cluster
params:
- name: kind
value: task
- name: name
value: echo
- name: namespace
value: hello-world
---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: test
namespace: foo
spec:
pipelineRef:
resolver: cluster
params:
- name: kind
value: pipeline
- name: name
value: echo
- name: namespace
value: hello-world
This would work, aka you have to be "explicit" for all task if you want them coming from elsewhere than the current namespace. I do agree though that it is not user friendly 😅 and most likely need fixing.