WARNING: html report file could not be created

[swuyts]

Hi there,

I was giving TORMES a shot but ran into some issues.

I invoked TORMES as follows, on one single sample:

# Run TORMES
tormes --metadata samples_metadata.txt \
	--output GENOME \
	--no_mlst \
	--no_pangenome \
	--threads 16

This is the tormes.logfile:

Annotation started at: 2019-07-04 13:40
Tormes report started at: 2019-07-04 13:40
WARNING: html report file could not be created
TORMES pipeline finished at: 2019-07-04 13:40

When I then inspect all results folders, I see that the majority of the analyses are not performed. I then further inspected the stdout and noticed that everything up until QUAST runs fine but that the program executed after QUAST starts acting weird.

Here's a short snapshot of this log file:

Thank you for using QUAST!
Loading database... complete.
314954 sequences (129.48 Mbp) processed in 3.744s (5047.0 Kseq/m, 2074.85 Mbp/m).
  77123 sequences classified (24.49%)
  237831 sequences unclassified (75.51%)
/home/tormes/anaconda3/envs/tormes-1.0/bin/tormes: line 853: /media/ssd/genome_data/wd/GENOME/tormes.log: No such file or directory
/home/tormes/anaconda3/envs/tormes-1.0/bin/tormes: line 854: /media/ssd/genome_data/wd/GENOME/tormes.log: No such file or directory
parallel: Error: Cannot open input file `/media/ssd/genome_data/wd/GENOME/list.tmp': No such file or directory.
sed: can't read /media/ssd/genome_data/wd/GENOME/antibiotic_resistance_genes/resfinder/GENOME_resfinder.tab: No such file or directory
parallel: Error: Cannot open input file `/media/ssd/genome_data/wd/GENOME/list.tmp': No such file or directory.
sed: can't read /media/ssd/genome_data/wd/GENOME/antibiotic_resistance_genes/card/GENOME_card.tab: No such file or directory
parallel: Error: Cannot open input file `/media/ssd/genome_data/wd/GENOME/list.tmp': No such file or directory.
sed: can't read /media/ssd/genome_data/wd/GENOME/antibiotic_resistance_genes/argannot/GENOME_argannot.tab: No such file or directory
parallel: Error: Cannot open input file `/media/ssd/genome_data/wd/GENOME/list.tmp': No such file or directory.
sed: can't read /media/ssd/genome_data/wd/GENOME/virulence_genes/GENOME_vfdb.tab: No such file or directory
sed: can't read /media/ssd/genome_data/wd/GENOME/draft_genomes/GENOME.fasta: No such file or directory

It is also weird that these errors are thrown before PROKKA annotation.

Any idea what is going on?

Many thanks! Sander

[nmquijada]

Dear Sander,

Sorry for the late reply. Could you please re-dun thesame analysis by redirecting the screen outputs and errors to a file? Like this: tormes --metadata samples_metadata.txt --output GENOME --no_mlst --no_pangenome --threads 16 &>>error-tormes.txt 2>>error-tormes.txt & Please, share with me the error-tormes.txt file. Bests, Narciso

[biobrad]

Dear Sander,

Sorry for the late reply. Could you please re-dun thesame analysis by redirecting the screen outputs and errors to a file? Like this: tormes --metadata samples_metadata.txt --output GENOME --no_mlst --no_pangenome --threads 16 &>>error-tormes.txt 2>>error-tormes.txt & Please, share with me the error-tormes.txt file. Bests, Narciso

Hi Narciso, I have had the exact same problem. I have done what you have advised, attached is the error-tormes.txt file. Cheers Brad error-tormes.txt

[nmquijada]

Hi Brad, I checked your error-tormes.txt file and it ends up in the middle of the assembly of SPAdes without any error. Are you sure that the tormes analysis is already finished? Cheers, Narciso

[biobrad]

Hi Narciso, it didn't appear to run again, it just created that folder and the error text... does the commands you provided make it run without logging to the shell?

[biobrad]

Ah, I see! it outputs the logging to the txt instead of the shell. It is still running, Attached is updated .txt as of this stage error-tormes.txt

[biobrad]

Thank you very much by the way for looking into this. Thank you for creating the pipeline as well. It was so easy to install and get running compared to other pipelines!

[nmquijada]

Hi Brad, The commands I provided make the action to be performed in the background of the computer, so it can keep on running unless you plug off your computer/server. You can trace if it's still running by commands such as top and you can stop it whenever you want by using kill + the ID of the job. You can also check the tormes.log located in /home/brad/Downloads/GENOME/ It the last line is not "TORMES pipeline finished at: " it means tormes is still running.

[nmquijada]

Hi Brad, Thanks, I really appreciate your words :) Thanks also for sharing the new error-tormes.txt file. Tormes is still running, I can see that the assembly finished witout errors. Please, check in some time the tormes.log located in /home/brad/Downloads/GENOME/ until the last line is "TORMES pipeline finished at: " and then share with me the error-tormes.txt file. Thus, we'll be sure that the pipeline finsihed and I can look at the entire error log file. Thanks! Narciso

[biobrad]

HI Narciso, Ok, all done, it looks like by skipping the pangenome it seemed to make it through to the end. I have attached the txt file. error-tormes.txt

[nmquijada]

Hi Brad, Thanks for sharing the file. It seems that it run without errors this time, right? For pangenome analysis one of more sample is needed in order to perform genome comparisons. It could be a bug at this step (that might be skipped) when using tormes for analyzing one sample. I'll try to fix this bug for further versions. Thanks! Narciso

[biobrad]

Hi Narciso, Yes, it ran perfect without the pangenome. Thank you so much for your responses. It is great pipeline and definitely the most user friendly one I have come across. I really appreciate you help! cheers Brad

[biobrad]

Narciso! Your pipeline does NOT have any bugs. I worked out what the issue is.

I didn't install the quast files.

If anyone else has this problem... Make sure you are in the tormes environment in conda: conda activate tormes-1.0 quast-download-gridss quast-download-silva quast-download-busco

One other thing as well, the config file needs a change for mauve. The config_file.txt has: mauve-2.4.0.r4736-0 You need to change the config_file.txt to have a 1 on the end instead of a 0 mauve-2.4.0.r4736-1

[nmquijada]

Hi Brad,

Thank you very much for sharing the solution! It will be very helpful for other users encountering the same issues. I will include this actions in the tormes-setup script for further TORMES releases. Thanks! Narciso

[swuyts]

Hi Narciso,

I performed these analysis in a previous function and on a machine to which I do not have easy access to. I am therefore sorry that I can not rerunthe pipeline anymore.

Thank you for chipping in Brad!

Sander

[sprusaka]

Narciso! Your pipeline does NOT have any bugs. I worked out what the issue is.

I didn't install the quast files.

If anyone else has this problem... Make sure you are in the tormes environment in conda: conda activate tormes-1.0 quast-download-gridss quast-download-silva quast-download-busco

One other thing as well, the config file needs a change for mauve. The config_file.txt has: mauve-2.4.0.r4736-0 You need to change the config_file.txt to have a 1 on the end instead of a 0 mauve-2.4.0.r4736-1

Is there any way to install and run these databases locally ? I have run these steps, but since I don't have many permissions on the server the databases where downloaded in my home directory. I have a feeling that's why TORMES still quits on Quast.

Many thanks

[nmquijada]

Hi @aartdvark Could you please re-run your analysis by redirecting the screen output/errors to a file and then share that file with me? You can do it like this: https://github.com/nmquijada/tormes/issues/6#issuecomment-538347884 If you any have problem with this scripting, please let me know and I will help. Cheers!

[sprusaka]

Hi @nmquijada

Please see the output files below.

Many thanks.

tormes.log error-tormes.txt

[nmquijada]

Hi @aartdvark I checked your error file and TORMES didn't quit after Quast, as all the analysis that have to be performed after Quast (annotation, antbiotic resistance and virulence screening, etc.) were done and you might find the results in /mnt/mydata/MINI/tormes/error_test/ However, there was a problem generating the report, as it reported this error when trying to load the "rmarkdown" package into R:

Error: package or namespace load failed for ‘rmarkdown’ in rbind(info, getNamespaceInfo(env, "S3methods")): number of columns of matrices must match (see arg 2)

It could be a problem of installation of that package in your system. Are you familiar with R? If so, please activate tormes environment and install the "rmarkdown" package. Otherwise, I can provide further instructions for you. Once you have rmarkdown corretly installed, try to generate a new report. In your case, it would be:

conda activate tormes-1.0
cd /mnt/mydata/MINI/tormes/error_test/
tar xzf report_files.tgz
cd report_files/
Rscript -e 'library(rmarkdown); rmarkdown::render("tormes_report.Rmd", "html_document", encoding="UTF-8")'

Let me know if it works!