writing a HTML report file sometimes crashes

[slukowski]

Hi Thanks for the excellent tool. It's extremely comprehensive. I really like the interactive HTML report as a way to share the analysis with non-computational colleagues. However, I've come across a small issue and I was hoping you could help and I hope I can provide enough information.

For some query sets, writing the HTML report causes an error, but writing the excel report is fine. In these cases, I receive an error after the analysis is complete, at the point of writing the report, such as:

[2021-05-19 17:26:04][INFO] Writing HTML file with report contents
Quitting from lines 23-39 (./query_verification.Rmd) 
Error in file(file, ifelse(append, "a", "w")) : 
  all connections are in use

This was happening during a loop for multiple query sets and also for single files. Is this something you've seen before?

Thanks for any guidance on this Sam

[slukowski]

Notably, it doesn't happen to all of them, but I'm not sure what the difference is between those that complete successfully and those that fail

[sigven]

HI @slukowski,

Thanks for reaching out! This is probably a bug somewhere, could you forward the queryset, or the complete command you are running when it fails?

Kind regards, Sigve

[sigven]

Might be an issue with the output filename (too long path?), just noticed this issue:

https://stackoverflow.com/questions/17156445/why-i-get-this-error-writing-data-to-a-file

[slukowski]

Thanks for looking into it more. After looking at my commands and trying a few things, it's not really any clearer, except that all of the excel reports for all analyses write out ok. Some of the HTML reports write fine as well, but not all, and it's not clear why. I wondered if one of the HTML files wasn't closing properly. I'll check the filename situation and report back.

Thanks!

[sigven]

Would perhaps be useful for me to see whether there is an underlying bug related to the nature of the queryset also (if the filename issue is not the cause)? You can send them to me by e-mail if you do not want to share the lists here.

[sigven]

One more thing: pandoc version might be a matter, i.e. what do you see when you type rmarkdown::pandoc_version()?

[slukowski]

Hi - I managed to find a workaround, which involved 1) running the enrichment step, 2) saving all the intermediate results as .RDS files, and 3) closing the entire session, reloading the results files and then running the HTML part. No idea why, but it worked and it doesn't fail on any of them. I can't explain it, but I'm happy I could get the results in HTML format.

Regarding the rmarkdown::pandoc version, it is 2.1.3. If it helps, these HTML files ended up being ~20MB each and they took about 3-5min to write per file.

[sigven]

Great! I'll point to this issue for a workaround solution if others experience the same. Closing this for now.