Export graph as web page. Pandocs memory limitation?

[Phaethon1]

Dear Sahir ,

Thank you for your wonderful work and for sharing the manhattanly code.

I am trying to export the plotly graph that is the output of manhattanly, but it seems that due to the size of the input package (~1M Snps -> ~50Mb.txt -not so big GWAS file after all-) , pandocs has tough time when it outputs to PNG or Web page.

I have tested it with a cut down version of the input file ~20K Snps, and it works fine, but the 1M file is impossible to be saved as PNG/html. The graph can be seen within R though.

Surely there are plenty of posts about difficulties using Pandoc and memory limitation workarounds (https://jonathanchang.org/blog/fixing-pandoc-out-of-memory-errors-on-windows/) but nothing so far has worked. I have used a 8Gb Ram PC, and a 24Gb RAM PC, but to no avail. Pandoc goes up to 4Gb RAM

Probably you have experienced this yourself. Is there any workaround?

RSTUDIO OUTPUT MESSAGE Error: pandoc document conversion failed with error 1 In addition: Warning message: running command '"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS "C:\Users\XXXXX\AppData\Local\Temp\RtmpMhYyrQ\viewhtml11f038b647b4\index.html" --from markdown_strict --output "C:\Users\XXXXX\Documents\XXXXX2\plotly_work2\manhattanly_lowP\low_P.html" --self-contained --template "C:\Users\XXXXX\AppData\Local\Temp\RtmpMhYyrQ\file11f08da3dd9.html"' had status 1

[sahirbhatnagar]

Thanks for your interest in this package and sorry for delayed response. I currently don't have a workaround. The only thing I have in mind is to create a manhattanly function that automatically shaves the data that doesn't add much information to the plot. For example, in a GWAS, perhaps remove approximately 50% of the SNPs with -log10(p) less than 2. I need to come up with a smart way of automatically doing this. Let me know what you think of this idea.

Unfortunately, I don't know the ins and outs of Pandoc to better answer your question.

[Phaethon1]

Thanks for your response. I am thinking also to try incremental image creation and superposition. Something that is similar to the approach I believe pheweb uses with ggplot2. Because plotly uses pandoc, I would need multiple steps in the process. ie A. -log10(p) less than 2, B. -log10(p) less than 3 and higher than 2 etc. just to be on the safe side.