Add option to 'flatten' included taskfiles

[tommyalatalo]

trafficstars

It would be great to be able to set an option to avoid having to call the tasks using the task includename:task syntax when including a taskfile.

So this example assumes you have files like: ./Taskfile.yml ./common/Taskfile.yml

The use case for this is when creating a main Taskfile for a repository and importing some common tasks, lets say "list", "clean" and "version", right now I need to then call task common:clean etc to run them.

Instead I would like to be able to do something like this in my main taskfile:

includes:
  common: ./common
    flatten:true
    override: false

Where the 'flatten' property would define that the tasks from 'common' should be available as if they were present in the main taskfile, so "task clean" would actually call "task common:clean". This would probably have to throw a conflict error if the "clean" task exists in both the main taskfile and the 'common' one, meaning that anything from an included taskfile that would be flattened out needs to be unique. Possibly an "override" property could be added instead, to let the included task take precedence over any task with the same name in the main taskfile.

This kind of functionality would allow you to add taskfiles as a submodule to a repository, create a main taskfile for that repo where you include the submodule taskfiles, and still be able to run common tasks with a short syntax without having to point out the sub-taskfile. Or in other words include other taskfiles as transparent "libraries".

[andreynering]

Hi @tommyalatalo, thanks for opening this issue!

I'm still not convinced to add this feature. I think this brings too little benefit to be worth it: mostly allowing shorter task names (i.e. build instead of common:build).

I like how namespaced tasks allow you to immediately know where a task if defined (you just have to follow the import).

Am I missing something on other ways this would be useful?

[tommyalatalo]

Hi @tommyalatalo, thanks for opening this issue!

I'm still not convinced to add this feature. I think this brings too little benefit to be worth it: mostly allowing shorter task names (i.e. build instead of common:build).

I like how namespaced tasks allow you to immediately know where a task if defined (you just have to follow the import).

Am I missing something on other ways this would be useful?

I think there is a valid use case here. If I want to do multiple includes from a main taskfile, and those includes can vary, lets say like this:

Taskfile.yml
- build
- test

Taskfile.go.yml
- build
- test

Taskfile.another.yml
- build
- test

Then the main taskfile could benefit from the "flatten" and "override" function in that it would allow you to set default build and tests tasks in the main file, but you can include another taskfile which would override them as desired, depending on the content of the repository you're working in. That is to say if I work in a go repo I just import the "go" taskfile and set it to flatten and override, then it would automatically take over the targets in the main taskfile.

Another use for this is to clean up the --list output. When including multiple taskfiles from the main you could end up with something like this:

Taskfile.yml
- build
  -> go:build
- test
  -> go:test
- clean
  -> go:clean

Taskfile.go.yml
- build
- test
- clean

This would result in duplicate output of commands in --list like:

build - build binary
go:build - build binary
test - run tests
go:test - run tests
clean - remove build artifacts
go:clean - remove build artifacts

So you see in the above case I would be able to override all the non-"go" targets to keep the list output clean, since this list can get very long if including multiple other taskfiles. This duplicate list output can be confusing to someone who doesn't know as much about the importing of other taskfiles, and is simply cluttering. The main point of this would be to keep the standard targets "build", "test" and "clean" in this case always the same, so that you don't have to manually edit them to point to the "go" targets or whatever else you're importing.

I think this relates somewhat to #268 since if you do an override of a task, it would still be beneficial to see whether that task points to another task or if its still the default, so adding an "Alias" column to the --list output would provide that information in a good way.

[ghostsquad]

I kind of disagree that flattening should exist, as there's a couple things that I think might be confusing, how does a user reasonably figure out what tasks refer to what, especially since you can reference super.<task> using :taskname in an included file:

Taskfile.included.yaml

version: '3'

tasks:
  build:
    cmds:
      - task: welcome
      - task: :build # this calls the task named `build` of the `Taskfile` that _imports_ this Taskfile. (i.e. super.build)
      - task: fairwell

  welcome:
    cmds:
      - echo "Hello There!"

  fairwell:
    cmds:
      - echo "Until Next Time!"

Taskfile.yml

version: "3"

includes:
  i:
    taskfile: Taskfile.included.yml

tasks:
  build:
    cmds:
      - echo "building..."
      - echo "build complete!"

I don't think it would be very clear to an end user, if flatting was a thing, to figure out what's actually going on. The more I get into Golang development, the more I'm realizing that readability counts quite a bit. This is a case that would likely significantly hurt readability.

[Gowiem]

@pd93 thanks for sharing the dup and closing out my ticket #1271. I was actually just reading through this issue right after filing mine. Should've searched better before writing the whole thing up.

Anyway, I wanted to share that I think there is value in this and why. Particularly in the case that a lot of orgs are using Task (#770), which is where there is a centralized repo of tasks that get pulled into other repos as shared tasks and there aren't really any tasks in the consuming repos. In that use-case, we're creating a namespace hierarchy that is unnecessary and creates tedium. I get the argument against that everything should be explicit, but there is value in terseness as well.

[marverix]

Hi, I just was looking for Gradle alternative in my organization, which currently is using Gradle as a task manager. Currently, the lack of the "flattening" possibility in Task is a blocker for me to use it as a Gradle alternative. The use case is very similar to my previous speakers. As a CI/CD tech lead, I want to provide one file, which would be a Convention that we use in the company. For example, I can declare a Taskfile:

PipelineConvention.yaml

version: '3'

tasks:
  stage-init:
    desc: Stage run to initialize the pipeline
    cmd: exit 0
    silent: true
    
  stage-build:
    desc: Stage run to build your project
    cmd: docker build -f Dockerfile .
    silent: true
    
  stage-test:
    desc: Stage run to test your project
    cmd: dgoss run something something
    silent: true

And with that, I could do a git submodule in any repo of the organization and create Taskfile:

version: '3'

includes:
  PipelineConvention:
    taskfile: ./common-taskfiles/PipelineConvention.yaml
    flatten: true

tasks:
  stage-init:
    cmd: echo "I'm overwriting the task stage-init in this repo!"

This would give me the possibility to create a Jenkins Pipeline that will always just run for example task stage-init. In my opinion, this is a must-have feature to allow bigger organizations standardize tasks across all repos.

As I said, in Gradle it's easily achievable (and I'm thinking about Gradle mainly as a task manager, not the builder - because I don't care about Java, and it wasn't mine decision to use Gradle), because you can overwrite any task - even subtask.

So at least if you don't want to implement flatten, then maybe at least allow to overwrite included (namespaced) task from the taskfile that includes. Eg.

version: '3'

includes:
  PipelineConvention:
    taskfile: ./common-taskfiles/PipelineConvention.yaml
    flatten: true

tasks:
  "PipelineConvention:stage-init":
    cmd: echo "I'm overwriting the task stage-init in this repo!"

Br, Marek

[vmaerten]

We have the same use case as @marverix in my company

[Chi-teck]

My use case: There is a number of tasks included to the Taskfile that is stored in Git repository.

* build/alpha:  Builds alpha
* build/beta:  Builds beta

I need to be able to include additional tasks from local git ignored Task file. So the resulting list of tasks would look like follows

* build/alpha:  Builds alpha
* build/beta:  Builds beta
* build/gamma:  Builds gamma # Comes from local Taskfile

For consistency it's highly desired that all tasks share same namespace. No matter where they come from.

[Chi-teck]

I agree that that 'flatten' behavior should be optional and disabled by default.

[elocke]

This feature would be massive benefit to our team!