nextflow icon indicating copy to clipboard operation
nextflow copied to clipboard

Output file("*.foo") returns a single file or list

Open heuermh opened this issue 5 years ago • 3 comments

Bug report

Capturing wildcard files in an output: block

process split_foo_gz {
  input:
    set sample, file(foo) from foos
  output:
    set sample, file("*.foo.gz") into splitFoos
...

appears to result in either a single file or a list of files, which makes downstream code confusing or tricky, e.g. in this attempt to print the size of the list of files, it prints the size (in bytes presumably?) of a single file

$ nextflow split_fasta_gz.nf
N E X T F L O W  ~  version 19.04.1
Launching `split_fasta_gz.nf` [amazing_lalande] - revision: 3bb463322d
[warm up] executor > local
executor >  local (2)
[f6/b16ff4] process > split_fasta_gz [  0%] 0 of 2

Split reference to 117968 files:
work/a3/eaa67d4822b8dfe66b5292c4f65352/reference.0.fa.gz

Split transcript to 10 files:
[work/f6/b16ff41dc6aee15e8c45e5f851e6e5/transcript.0.fa.gz, work/f6
/b16ff41dc6aee15e8c45e5f851e6e5/transcript.1.fa.gz, ... >  local (2)

Expected behavior and actual behavior

Always return a list, even if it contains zero or one files.

heuermh avatar Jul 22 '19 03:07 heuermh

For example, run nextflow split_fasta_gz.nf from https://github.com/heuermh/dishevelled-bio-nextflow

heuermh avatar Jul 22 '19 04:07 heuermh

Yes, this was a bad design choice. Hopefully is going to solved soon in the next syntax review.

Related #969

pditommaso avatar Jul 25 '19 12:07 pditommaso

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Apr 27 '20 04:04 stale[bot]

Fixed by https://github.com/nextflow-io/nextflow/commit/42504d3c83145e16179e22df83e9d35ecc995eca

bentsherman avatar Sep 11 '23 14:09 bentsherman

Thank you for following up on this @pditommaso @bentsherman!

The arity option appears to be exactly what was needed

If the arity is 1, a sole file object will be emitted. Otherwise, a list will always be emitted, even if only one file is produced.

heuermh avatar Sep 11 '23 14:09 heuermh

It took only four years! 😆

pditommaso avatar Sep 11 '23 15:09 pditommaso