quantms icon indicating copy to clipboard operation
quantms copied to clipboard

We should dynamically configure some of the steps depending of the experiment size.

Open ypriverol opened this issue 1 year ago • 2 comments

Description of feature

@daichengxin, @jpfeuffer:

When running multiple datasets, I have found that it would be good to have some logic to configure memory and CPU depending on the number of files. We have done that approach for the proteomicsLFQ step, however other processes could also benefit from it. Here is a list of process to review:

  • [ ] NFCORE_QUANTMS:QUANTMS:TMT:PROTEININFERENCE:IDFILTER
  • [ ] NFCORE_QUANTMS:QUANTMS:DIA:ASSEMBLE_EMPIRICAL_LIBRARY Issue #339
  • [ ] ID:DATABASESEARCHENGINES:SEARCHENGINESAGE

ypriverol avatar Jan 12 '24 11:01 ypriverol

It should be possible to implement this in nextflow syntax. The logic can be divided two ways. What do you think?

  1. number * unit usage.
  2. range (24 G when number < 100, 128G when number<1000, 528G when number<10000, 1024G when number >10000)?
  3. other?

daichengxin avatar Jan 13 '24 07:01 daichengxin

Looks like a good strategy.

ypriverol avatar Jan 13 '24 10:01 ypriverol