catalog Request: Proposal(s) for supporting Pipeline in catalog

The proposals we've had so far around the catalog have centered around supporting Tasks. Though we've always known we wanted to support more types, we haven't thought through how to do that well.

For example in https://github.com/tektoncd/catalog/pull/457 @mrutkows is adding a Pipeline and has run into complication that goes beyond what Tasks require, for example a Pipeline will require a number of Tasks, so it is not sufficient to apply the Pipeline to use it, you also need to apply the Tasks and so a deploy.sh has been created to make that easier.

We should create some proposals for improving our Pipeline support, including at least:

[ ] How to provide tests that verify the Pipelines work e.g. Test Infra proposal only includes Tasks
[ ] How to ensure that a user can easily apply the Pipeline and required Tasks (maybe generating something like the deploy.sh? or supporting this well through the hub via a "manifests" file that lists required Tasks? Or a mechanism to parse these out?)

Aug 06 '20 17:08 bobcatfish

@mrutkows if it was possible to specify the supporting Tasks via their location in an OCI registry, would you still need something like deploy.sh? (cuz in this case you wouldn't need to apply the Tasks before using the Pipeline)

Aug 06 '20 17:08 bobcatfish

@mrutkows if it was possible to specify the supporting Tasks via their location in an OCI registry, would you still need something like deploy.sh? (cuz in this case you wouldn't need to apply the Tasks before using the Pipeline)

Of course, I would like to endorse componentization and reusability, but the assumption seems to be one that assumes that all of my tasks (or other artifacts) must be put into a registry. That raises the bar for them IMO, such that it feels like you are forcing me to create versioned artifacts (as I commented to Sunil earlier when I was asked to relocate and version all my tasks).

My analogy are packages... that is if I am writing a program which is composed of many private packages that promote structure and good design methodologies I should not be forced to push them to a "registry" and support them for the world. Now of course I will use public packages created and maintained with the intent for reuse (typically to solve general problem spaces around specific data/services/interfaces/protocols). Indeed, some of my private packages might be candidates for public packages, but most will not be as they are specific to my data/intfs./services and now one in a general application programming ecosystem (GoLang, Python, Java) force me to register (and support) them. Even if private registries are allowed, I really do not want to necessarily consider having to support packaging them (against some spec.) and supporting them through some registry API.

I can see where you are going with asking if we could remove deploy.sh... and am all for removing scripts to install resources... but I would prefer a manifest style. That is, like a go mod or other language-level dependency manifests I would like to describe my dependencies which could be relative to a filesystem, a github repo. or even an S3 object store (as resources which they are). In this case, the resource descriptors already exist... we just need a manifest format point to wherever they may exist.

Aug 19 '20 15:08 mrutkows

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale with a justification. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.