Pilon issue (default_jvm_mem_opts)

[xlinxlin]

Hi!

I would like to know is there a way to set default_jvm_mem_opts in Pilon (which is a part of Unicycler) through nextflow run command line? Especially when one uses -profile conda or -profile docker. Or there always be a problem by dealing with large genomes.

[apeltzer]

Currently there is no such option, but we're happy to integrate one - or you could add one yourself and supply this as optional parameter and open a pull request? :-)

[apeltzer]

Ok this is not as easy as thought - the pilon recipe from bioconda sets this already to a hard value and changing this from within the pipeline isn't easily possible:

https://github.com/rrwick/Unicycler/issues/147

Maybe this is something we should report upstream in bioconda and then consider fixing this afterwards here.

[apeltzer]

Asked here: https://github.com/bioconda/bioconda-recipes/issues/24749

[apeltzer]

Should be possible to replace this pilon -Xmx64g

[tamuanand]

Hi @apeltzer @xlinxlin

A suggested fix (which I had to do): I used conda to install pilon and then hand edited the pilon file with your favorite editor

<path_to>/envs/my-conda_env_name/bin/pilon

# change this line below from
default_jvm_mem_opts = ['-Xms512m', '-Xmx1g']
 
# to 16GB or something bigger depending on what you want
default_jvm_mem_opts = ['-Xms512m', '-Xmx16g']

Probably the same could be done within unicycler?

[apeltzer]

Hi @tamuanand !

Yes, this is an option in general, but unfortunately not a reproducible one and not one that works with containers (which ship pilon, but don't allow us to modify how unicycler is calling pilon unfortunately. The other fixes proposed here https://github.com/rrwick/Unicycler/issues/212 and here https://github.com/rrwick/Unicycler/issues/147 describe pretty much the same. Changing this simply isn't possible in a container very easily and its actually even a bad practice to do so in terms of reproducibility.

I currently don't really see an easy / straightforward fix without modifying conda recipes, which is a bad thing to do .... one potential solution would be to add an option to unicycler that allows to modify the memory requested for pilon - that would be a.) a good solution and b.) reproducible / clean.

[tamuanand]

Hi @apeltzer

A slightly related question:

I am trying to build a container using Docker that has pilon, quast, multiqc, spades etc. I am unable to get the container built because of (probably python version conflicts) and incompatibility withh quast.

Any suggestions?

[apeltzer]

Exactly the reason why we have multiple containers - these tools are inherently using as you said different python versions and are hard to make Them work nicely along each other. Not much you can do about this :-(

[Daniel-VM]

Clossing this issue, please feel free to reopen it if needed