atacseq icon indicating copy to clipboard operation
atacseq copied to clipboard

Reduce available memory for Picard Mark Duplicates

Open ryan-moreno opened this issue 1 year ago • 1 comments

Right now, the pipeline fails for me with the following error:

Error executing process > 'NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES (DMSO_REP4)'

Caused by:
  Process `NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES (DMSO_REP4)` terminated with an error exit status (1)

Command executed:

  picard \
      -Xmx36g \
      MarkDuplicates \
      --ASSUME_SORTED true --REMOVE_DUPLICATES false --VALIDATION_STRINGENCY LENIENT --TMP_DIR tmp \
      --INPUT DMSO_REP4.mLb.sorted.bam \
      --OUTPUT DMSO_REP4.mLb.mkD.sorted.bam \
      --REFERENCE_SEQUENCE genome.fa \
      --METRICS_FILE DMSO_REP4.mLb.mkD.sorted.MarkDuplicates.metrics.txt
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES":
      picard: $(echo $(picard MarkDuplicates --version 2>&1) | grep -o 'Version:.*' | cut -f2- -d:)
  END_VERSIONS

Command exit status:
  1

Command output:
  Error occurred during initialization of VM
  Could not reserve enough space for 37748736KB object heap

I ran into this issue with the cutandrun pipeline and the rnaseq pipeline. In both cases, the fix was to allocate only a fraction of the available memory when launching the process. Here is the relevant change in the cutandrun pipeline. Thanks @drpatelh for the change in the cutandrun repo.

PR checklist

  • [x] This comment contains a description of changes (with reason).
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
  • [ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs- [ ] If necessary, also make a PR on the nf-core/atacseq branch on the nf-core/test-datasets repository.
  • [ ] Make sure your code lints (nf-core lint).
  • [ ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • [ ] Usage Documentation in docs/usage.md is updated.
  • [ ] Output Documentation in docs/output.md is updated.
  • [ ] CHANGELOG.md is updated.
  • [ ] README.md is updated (including new tool citations and authors/contributors).

ryan-moreno avatar May 01 '23 16:05 ryan-moreno