Spectral library generation part is using more threads that it is supposed to be

[fcyu]

The EasyPQP convert command seems to be multithreading: each command is using all CPU cores. But the gen_con_spec_lib.py script also execute multiple convert commands in parallel, which makes the total thread equals #{CPU} * n, where n is the number of convert commands.

Best,

Fengchao

[guoci]

This is a EasyPQP/pyOpenMS issue. EasyPQP uses pyOpenMS for parsing LCMS files, but you cannot control the number of threads used by the parser from pyOpenMS.

[fcyu]

If the convert command always use all CPUs, the gen_con_spec_lib.py should not run multiple convert command in parallel.

Also, can you create an issue to EasyPOP or OpenMS letting them add an option to set the number of threads.

Thanks,

Fengchao

[guoci]

only the parsing is multithreaded, the rest of the code is single-threaded, so we might slow down the processing by doing that

[fcyu]

Then, please create an issue to EasyPQP or OpenMS to letting them add an option to set the number of threads. And then, set it to 1 in FragPipe. At any rate, we cannot have #{CPU} * n threads running. It will crash some HPC.

[guoci]

submitted an issue: grosenberger/easypqp#76

[guoci]

@fcyu update on the issue: https://github.com/grosenberger/easypqp/issues/76#issuecomment-1048932833 https://github.com/OpenMS/OpenMS/pull/5839