build

build copied to clipboard

Given the simplicity of using Builds, it would be useful for a Tekton user to be able to use builds in a Tekton Pipeline as a part of a larger delivery process. Custom Tasks enable duck typing for plugging in other CRDs as tasks into a pipeline. It would be interesting to explore if Build resources could work as a custom task in Tekton.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: example
spec:
  params:
  - name: pipeline-param
    default: hello
  tasks:
  - name: example-task
    taskRef:
      apiVersion: build.dev/v1alpha1
      kind: Build
      name: buildah-golang
    params:
    - name: task-param

have you got a take yet @siamaksade on the priority of such an exploration? enough of a take to claim you preference on which milestone to target?

pending whatever answers you might have, grooming through the community meeting, etc. still seems warranted before any official assignment

@gabemontero It would be post GA IMO given that custom tasks are quite new (alpha) in Tekton.

@sbose78 asked for my input on this.

This certainly seems possible to do with Custom Tasks, but I think I need to understand Shipwright a bit better to understand how this improves the overall user experience.

My understanding is, when you tell Shipwright to run a Build, it turns around and asks Tekton to run a TaskRun, and Tekton tells K8s to run a Pod, etc. If that's all you get, adding Shipwright as as Custom Task would seem to only add a layer of abstraction, another hop to getting to the Pod that actually runs the work. This added layer of course adds another layer to debug when things inevitably go wrong.

Shipwright does provide a simpler API focused on building container images, so maybe the benefit is in constraining what the user can express to "build this image". The question is whether that's any simpler or more constrained than "run this Task that builds an image", which Tekton already supports without a Custom Task. I don't have enough context today to know for sure.

Perhaps the real benefit is in letting Shipwright users, who normally use Shipwright to simply build images in response to some trigger (source push, base image change, etc.), to integrate that with a Tekton Pipeline to explicitly kick off the build, instead of (or in addition to?) waiting for the trigger condition. I actually really like the simplicity that Shipwright promises -- "watch X, build an image" -- but I assume there are use cases where people want to graduate from the simple Shipwright case to a more complex pipeline, and this could help them slowly adopt that instead of having to throw everything out to restart with Tekton (or something else).

That turned into a lot more words than I originally expected, so I'm sorry about that -- tl;dr: I don't know if this is a good idea yet 😅 , let me learn some more and get back to you with ideas.

@siamaksade thanks for this issue, I also see potential here for post-ga.

@ImJasonH currently the flow is as follows:

where:

• an Strategy CRD holds a list of containers that define a series of steps in order to get-source-code+build-an-image+push-image. We embed this definition into the TaskRun during runtime. This provide us flexibility on defining different flavours on how to build, e.g. kaniko, buildpacks, buildkit, buildah, etc.
• A Build defines "what" to build, e.g. where is my source code, where I want to push my image and which strategy to use for building.
• A BuildRun triggers the whole building mechanism and reports back the state of the building.
• The above image layouts the flow: it moves from a Build, to a BuildRun, then it uses the strategy to define the TaskRun spec and eventually the pod is created.
• The BuildRun CRD reference a particular Build instance. This means you can actually have N BuildRuns referencing the same Build, we call this 1 to N.
• There is a Build controller, watches for Builds.
• There is a BuildRun controller, watches for BuildRuns and TaskRuns.

I took a look on EP Custom Tasks, if we translate the current state to that approach, it might look like:

where:

• Run CRD is manage by Tekton. I´m assuming this triggers the Pod creation in the background and in general runs to completion.
• We keep our minimal Build API, that defines the "what".
• We replace the BuildRun(shipwright) in favor of Run(Tekton), both should have the same goal, run to completion and report back the state. However not having full control on the Run API, will force us to define our current BuildRuns API fields somewhere else.
• We reduce our 2 controllers(Build/BuildRun), into a single one that monitors:
  • Build´s
  • Run´s , in order to see if we run to completion and to populate the Run Status.Condition.

The open questions I have(there might be more, I need to experiment with Custom Tasks from my side):

• How can we avoid losing the flexibility we have today around defining the content of the TaskRun via our Strategies CRD´s.
• We currently do not use Pipelines, we are only constrain to Tasks/TaskRuns
• Can we ensure the 1 to N relationship between the Build and the Run?

Ah I think I understand where I was misunderstanding. In my mental model, the custom task controller would reach out to the (otherwise unmodified) Shipwright installation to create the BuildRun, which would reach back out to Tekton to run the TaskRun/Pod.

Your diagram seems to move creation of TaskRuns (or, perhaps direct Pod creation 🤔 ) into the custom task controller, thus ~duplicating/sharing logic from Shipwright's codebase to translate the BuildRun into a TaskRun. This is certainly less indirect, which is a good thing, but we'd still need Shipwright to be running on the cluster to validate BuildStrategies etc., unless we also want to split out and bundle that functionality with the CT controller.

So I still think it's still possible, and useful especially to help smooth out the transition for users from Shipwright->Tekton. Let me know if you want to prototype this, I'd be happy to contribute/review/lgtm or anything else you need.

It sounds like Custom Tasks might also be our path to integrating with Tekton Triggers, which currently disallows triggered creation of any non-Tekton resources. However, Triggers can create Custom Tasks Runs, which our controller could respond to by creating BuildRuns.

I have a simple PoC here: https://github.com/ImJasonH/build-task

I'd like to get it in a better state and see about bringing into Shipwright directly, eventually.

It sounds like Custom Tasks might also be our path to integrating with Tekton Triggers, which currently disallows triggered creation of any non-Tekton resources. However, Triggers can create Custom Tasks Runs, which our controller could respond to by creating BuildRuns.

I have a simple PoC here: https://github.com/ImJasonH/build-task

I'd like to get it in a better state and see about bringing into Shipwright directly, eventually.

https://tektoncd.slack.com/archives/CKUSJ2A5D/p1616445589005000 for the tl;dr ;-)

@ImJasonH that sounds promising, checking ...