Reads dropped when demultiplexing a sample across lanes.

[AaronNHart]

Description of the bug

First off, thank you for maintaining this pipeline, it looks very useful!

While informally validating the pipeline on an old study, I believe I observed that fastq files from the same sample but run from different lanes overwrite one another due to being written to the same location for each lane.

My scenario is similar to the usage documentation with a samplesheet like:

id,samplesheet,lane,flowcell foo,s3://SampleSheet.csv,1,s3://foo/ foo,s3://SampleSheet.csv,2,s3://foo/

Command used and terminal output

nextflow run nf-core/demultiplex/dev/ \
   -config ./my.config \
   --input samplesheet.csv \
   --demultiplexer bclconvert \
   --outdir s3://bucket/out/ \
   -work-dir s3://bucket/work/

Relevant files

You can see here that the file shown is shortly overwritten, the timing corresponds to the moment when each bclconvert jobs completes.

I prefer to not directly post the whole log file, but if you have questions about it I can pull out some details as needed.

System information

• Nextflow 23.10.1 build 5891
• nf-core/demultiplex 1.4.1
• head job from my desktop (Ubuntu 22)
• AWS Batch executor (w/docker)