eager icon indicating copy to clipboard operation
eager copied to clipboard

DSL2: Ensure that all sharded fastQs make it into mapping.

Open TCLamnidis opened this issue 1 year ago • 3 comments

(Preface: I might be under caffeinated and seeing things, BUT)

I think the sharded FastQs never actually make it to mapping? It looks like a channel ch_input_for_mapping is correctly created with them, but then all mappers in the if/else section of the MAP SWF start with: ch_input_for_mapping = reads, which pulls the original unsharded reads again... 🤔

I also get the feeling that the mapping output files should actually contain the shard number, to ensure they are kept when merging lanes. 🤔 🤔

TCLamnidis avatar Sep 27 '24 08:09 TCLamnidis

@nf-core/eager Could I get a second opinion? Am I missing something?

TCLamnidis avatar Sep 27 '24 08:09 TCLamnidis

Yes you are right we are using the original unsharded reads. We should be using ch_input_for_mapping.reads instead. Under caffeinated brain works just as well as the caffeinated one ;)

shyama-mama avatar Sep 27 '24 09:09 shyama-mama

Made simple changes to fix the bug. https://github.com/nf-core/eager/pull/1090

shyama-mama avatar Sep 27 '24 10:09 shyama-mama