tools icon indicating copy to clipboard operation
tools copied to clipboard

Support modules with `exec:` blocks

Open muffato opened this issue 6 months ago • 1 comments

With the current version, linting a module that has an exec: block yields the following error

╭─ [✗] 1 Module Test Failed ───────────────────────────────────────────────────╮
│                      ╷                          ╷                            │
│ Module name          │ File path                │ Test message               │
│╶─────────────────────┼──────────────────────────┼───────────────────────────╴│
│ hiccramalign/chunks  │ modules/sanger-tol/hicc… │ when: condition has too    │
│                      │                          │ many lines                 │
│                      ╵                          ╵                            │
╰──────────────────────────────────────────────────────────────────────────────╯

This is because the parser is not aware of exec: blocks.

In this PR I simply add code to recognise exec: blocks and check that they don't exist together with script: or shell:

Best, Matthieu

PR checklist

  • [x] This comment contains a description of changes (with reason)
  • [ ] CHANGELOG.md is updated
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
  • [ ] Documentation in docs is updated

muffato avatar Jun 20 '25 15:06 muffato

Codecov Report

:x: Patch coverage is 50.00000% with 4 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 77.29%. Comparing base (9bff2b9) to head (2872af9). :warning: Report is 5 commits behind head on dev.

Files with missing lines Patch % Lines
nf_core/modules/lint/main_nf.py 50.00% 4 Missing :warning:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Jun 20 '25 16:06 codecov[bot]

I think there is no explicit reason not to support it but :

  1. make sure that it is only recommend to use in certain contexts (e.g. not heavy jobs, as it runs on the head node)
  2. we need to ensure that the structure of the block itself is consistent... (harshil align etc)

For now I think it's fine to merge this (exec: is valid Nextflow), but we should have the maintainers team develop a couple extra points for the specs page :)

jfy133 avatar Jul 30 '25 10:07 jfy133

So far it seems people are half neutral half against supporting within nf-core/modules officially, but I dont think linting should complain about a valid nextflow syntax so again I would merge this now so people can happily have local modules

jfy133 avatar Jul 31 '25 07:07 jfy133

I agree with James's summary, we should support this in general but an nf-core module would need a convincing argument as to why we need exec there.

SPPearce avatar Jul 31 '25 10:07 SPPearce

complain about a valid nextflow syntax

agreed, maybe as an improved later on, we can have stricter linting on the nf-core side to at least warn for execs (to Simon's point), but allow disableing this linting in a config file

FriederikeHanssen avatar Jul 31 '25 12:07 FriederikeHanssen

complain about a valid nextflow syntax

agreed, maybe as an improved later on, we can have stricter linting on the nf-core side to at least warn for execs (to Simon's point), but allow disableing this linting in a config file

I think it is fine in nf-core pipelines, just not in the modules repo personally.

SPPearce avatar Jul 31 '25 13:07 SPPearce

ah correct, I meant modules linting only

FriederikeHanssen avatar Jul 31 '25 16:07 FriederikeHanssen

thanks for all the feedback! I am going to merge this PR then :)

mirpedrol avatar Aug 01 '25 14:08 mirpedrol