Add support for label/tags

[pditommaso]

Summary

The goal of this enhancement is to allow users to propagate label/tag annotations to the corresponding cloud execution platform.

This is a recurrent request from Nextflow users. See here, here, here and here.

This support could be implemented by extending the semantic of the label directive, in such a way that when a key-value is provided using the syntax name=value is implicitly interpreted as key-value pair that needs to be propagated to the corresponding job execution.

This is expected to be an opt-in feature therefore the user should enabled either using the env variable: NXF_ENABLE_LABELS_PROPAGATION=true or setting the attribute nextflow.enable.labelsPropagation=true in the nextflow config file.

Supported platforms

• AWS Batch
  • Job definition
  • Job submit
• Google LS
  • Run pipeline allow to specified the labels key-value pairs to be used
• Azure batch (it looks like it does not provide any API for tags/labels annotation)
• Kubernets
  • The task pod should include the user-provided key-value pairs pod labels.

Validation

Each platform has a slightly different set of validation rules for labels name and values. It's to be defined if nextflow should prevent the use of such of invalid pattern or delegate the check to the target platform

[kdallen1]

Is there an estimated time to merge the linked PR? Looking forward to testing it out :smiley:

[pditommaso]

Hey there, this is scheduled for 22.10 release which means the end of this quarter

[pditommaso]

Following the internal discussion the decision is to not change the current implementation of label directive, add instead a new directive named resourceLabel to propagate labels and tags to cloud resources.

The implementation should remain the same. The settings NXF_ENABLE_LABELS_PROPAGATION and nextflow.enable.labelsPropagation=true are not needed any more

[jorgeaguileraseqera]

Implemented in #2853

[pditommaso]

But not merged