Dynamic Task Generation

[fkorotkov]

Description

Allow tasks to produce artifacts with text/yaml or application/json types and new cirrus format. Such artifacts will be evaluated in the context of the parent build and new tasks will be created for the build.

Context

Advanced users can benefit from it

[RDIL]

Or it could output JSON which is then read since JSON generation is a lot easier than YAML.

[fkorotkov]

@RDIL agreed. Updated the original comment.

[waghanza]

I confirm that is a wonderful steps, to have a configurable matrix. I mean having that tasks (I understand that name as jobs in `travis-ci) dynamically create is awesome.

I have a pretty long (https://github.com/the-benchmarker/web-frameworks/blob/master/.travis.yml) list of jobs that I need to maintain (each PR needs to add one item, so could mis-lead if their is no rebase)

[RDIL]

Hey @waghanza, It looks like your specific use case doesn't require this, but rather a matrix.
I can help set it up if you want.

[fkorotkov]

@waghanza case is a bit different then just a matrix modification. Each task uses it's own Dockerfile generated from a template for each commit.

But yeah, in the current world @waghanza can try to use a mix of Matrix Modification and Dockerfile as a CI environment if it's possible to use more or less stable Dockerfiles (remove ADD and COPY instructions from them since it's possible just to mount a working directory). In this setup changesInclude function will also be handy.

[waghanza]

@RDIL @fkorotkov if you want to help me, feel free to open a PR https://github.com/the-benchmarker/web-frameworks. However, I think more of having a job that create a matrix.

Actually, I have ~150 items in my matrix. This matrix is built with a cli tool and I've to commit (and not forget to do it after any item addition).

If I can generate any matrix based on git history, it will be :heart:

PS : docker is NOT a hard requirement (I mean I have to build containers, no matter where I am, in a vm or in a container)