rnaseq icon indicating copy to clipboard operation
rnaseq copied to clipboard
verified publisher icon nf-core

Patching samtools sort process behaviour when samtools sort resorts to using on-disk temporary files

Open koustav-pal opened this issue 1 year ago • 2 comments

PR checklist

  • [ x] This comment contains a description of changes (with reason).
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
  • [ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • [ ] If necessary, also make a PR on the nf-core/rnaseq branch on the nf-core/test-datasets repository.
  • [ x] Make sure your code lints (nf-core lint).
  • [x ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • [ x] Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • [ ] Usage Documentation in docs/usage.md is updated.
  • [ ] Output Documentation in docs/output.md is updated.
  • [ ] CHANGELOG.md is updated.
  • [ ] README.md is updated (including new tool citations and authors/contributors).

samtools sort resorts to using a combination of on-disk and in-memory temporary files when it is unable to sort using memory only. In such cases, samtools issues an exit code of 140 and provides a suitable error message stating that it is using in-memory and on-disk temporary files. This patch modulates samtools sort errorStrategy when such a scenario is observed.

koustav-pal avatar Feb 23 '24 15:02 koustav-pal

nf-core lint overall result: Passed :white_check_mark: :warning:

Posted for pipeline commit f2a99c1

+| ✅ 170 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   7 tests had warnings |!

:heavy_exclamation_mark: Test warnings:

  • files_exist - File not found: assets/multiqc_config.yml
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

:grey_question: Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • nextflow_config - Config default ignored: params.ribo_database_manifest
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/rnaseq/rnaseq/.github/workflows/awstest.yml
  • multiqc_config - 'assets/multiqc_config.yml' not found

:white_check_mark: Tests passed:

Run details

  • nf-core/tools version 2.13
  • Run at 2024-02-23 15:15:44

github-actions[bot] avatar Feb 23 '24 15:02 github-actions[bot]

Hi @koustav-pal ! Thanks for the PR! Can you provide me an example of the stdout/ Nextflow log file when there is a failure here please? Is it a silent error?

In any case, this PR should be made to nf-core/modules to update this module there. It can then be installed in pipelines from there.

We also should be catching this exit status with the standard errorStrategy we have set across all processes: https://github.com/nf-core/rnaseq/blob/b89fac32650aacc86fcda9ee77e00612a1d77066/conf/base.config#L17

drpatelh avatar Mar 12 '24 09:03 drpatelh

Closing for now. Please feel free to re-open with a fix as suggested above. Be great to have a little more info on the failure and access to the logs generating it.

drpatelh avatar May 13 '24 08:05 drpatelh