slamdunk
slamdunk copied to clipboard
t-neumann
"More columns than column names" error message when running alleyoop utrrates
Dear Team,
I am new to SLAMseq and trying to use SLAMDUNK for the analysis of my Slamseq data. When I am running the command "alleyoop utrrates -o try/ -r /ref/genome.fa -mq 27 -m -t 64 -b /ref/genome.gtf.bed -l 151 S4U_filtered_dedup.bam", I am getting the following error message:
Running alleyoop utrrates for 1 files (64 threads)
Traceback (most recent call last):
File "/miniconda3/envs/slamdunk/bin/alleyoop", line 10, in
Multiprocessing exception:
...........................................................................
/miniconda3/envs/slamdunk/bin/alleyoop in
........................................................................... /miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/alleyoop.py in run() 490 n = args.threads 491 referenceFile = args.referenceFile 492 minMQ = args.mq 493 494 message("Running alleyoop utrrates for " + str(len(args.bam)) + " files (" + str(n) + " threads)") --> 495 results = Parallel(n_jobs=n, verbose=verbose)(delayed(runStatsRatesUTR)(tid, args.bam[tid], referenceFile, minMQ, args.strictTCs, outputDirectory, args.bed, args.maxLength) for tid in range(0, len(args.bam))) results = undefined n = 64 496 dunkFinished() 497 498 elif (command == "summary") : 499 message("Running alleyoop summary for " + str(len(args.bam)) + " files")
...........................................................................
/miniconda3/envs/slamdunk/lib/python2.7/site-packages/joblib/parallel.py in call(self=Parallel(n_jobs=64), iterable=<generator object
Sub-process traceback:
RuntimeError Tue Jun 21 13:21:45 2022 PID: 245343Python 2.7.13: /miniconda3/envs/slamdunk/bin/python2.7 ........................................................................... /miniconda3/envs/slamdunk/lib/python2.7/site-packages/joblib/parallel.py in call(self=<joblib.parallel.BatchedCalls object>) 126 def init(self, iterator_slice): 127 self.items = list(iterator_slice) 128 self._size = len(self.items) 129 130 def call(self): --> 131 return [func(*args, **kwargs) for func, args, kwargs in self.items] func = <function runStatsRatesUTR> args = (0, '../S4U_filtered_dedup.bam', '/X101SC21113754-Z01-J001/ref/genome.fa', 27, True, 'try/', '/X101SC21113754-Z01-J001/ref/genome.gtf.bed', 151) kwargs = {} self.items = [(<function runStatsRatesUTR>, (0, '../S4U_filtered_dedup.bam', '/X101SC21113754-Z01-J001/ref/genome.fa', 27, True, 'try/', '/X101SC21113754-Z01-J001/ref/genome.gtf.bed', 151), {})] 132 133 def len(self): 134 return self._size 135
........................................................................... /miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/alleyoop.py in runStatsRatesUTR(tid=0, bam='../S4U_filtered_dedup.bam', referenceFile='/X101SC21113754-Z01-J001/ref/genome.fa', minMQ=27, strictTCs=True, outputDirectory='try/', utrFile='/X101SC21113754-Z01-J001/ref/genome.gtf.bed', maxReadLength=151) 169 170 log = getLogFile(outputLOG) 171 172 print("Using " + str(maxReadLength) + " as maximum read length.",file=log) 173 --> 174 stats.statsComputeOverallRatesPerUTR(referenceFile, bam, minMQ, strictTCs, outputCSV, outputPDF, utrFile, maxReadLength, log) referenceFile = '/X101SC21113754-Z01-J001/ref/genome.fa' bam = '../S4U_filtered_dedup.bam' minMQ = 27 strictTCs = True outputCSV = 'try/S4U_filtered_dedup_mutationrates_utr.csv' outputPDF = 'try/S4U_filtered_dedup_mutationrates_utr.pdf' utrFile = '/X101SC21113754-Z01-J001/ref/genome.gtf.bed' maxReadLength = 151 log = <open file 'try/S4U_filtered_dedup_mutationrates_utr.log', mode 'a'> 175 closeLogFile(log) 176 stepFinished() 177 178 def runSNPeval(tid, bam, ref, bed, maxLength, minQual, coverageCutoff, variantFraction, strictTCs, outputDirectory, snpDirectory) :
........................................................................... /miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/dunks/stats.py in statsComputeOverallRatesPerUTR(referenceFile='/X101SC21113754-Z01-J001/ref/genome.fa', bam='../S4U_filtered_dedup.bam', minBaseQual=27, strictTCs=True, outputCSV='try/S4U_filtered_dedup_mutationrates_utr.csv', outputPDF='try/S4U_filtered_dedup_mutationrates_utr.pdf', utrBed='/X101SC21113754-Z01-J001/ref/genome.gtf.bed', maxReadLength=151, log=<open file 'try/S4U_filtered_dedup_mutationrates_utr.log', mode 'a'>, printOnly=False, verbose=True, force=False) 523 else: 524 f = tempfile.NamedTemporaryFile(mode='w',delete=False) 525 print(sampleInfo.Name, outputCSV, sep='\t', file=f) 526 f.close() 527 --> 528 callR(getPlotter("globalRatePlotter") + " -f " + f.name + " -O " + outputPDF, log, dry=printOnly, verbose=verbose) f.name = '/tmp/tmp6FnCBb' outputPDF = 'try/S4U_filtered_dedup_mutationrates_utr.pdf' log = <open file 'try/S4U_filtered_dedup_mutationrates_utr.log', mode 'a'> printOnly = False verbose = True 529 530 531 def readSummary(filteredFiles, countDirectory, outputFile, log, printOnly=False, verbose=True, force=False): 532
........................................................................... /miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/utils/misc.py in callR(cmd='/miniconda3/envs/slamdunk/li...filtered_dedup_mutationrates_utr.pdf', log=<open file 'try/S4U_filtered_dedup_mutationrates_utr.log', mode 'a'>, verbose=True, dry=False) 206 lines_iterator = iter(p.stdout.readline, b"") 207 for line in lines_iterator: 208 print(line, end="", file=log) # yield line 209 p.wait(); 210 if(p.returncode != 0): --> 211 raise RuntimeError("Error while executing command: "" + cmd + """) cmd = '/miniconda3/envs/slamdunk/li...filtered_dedup_mutationrates_utr.pdf' 212 213 def pysamIndex(outputBam): 214 pysam.index(outputBam) # @UndefinedVariable 215
RuntimeError: Error while executing command: "/miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f /tmp/tmp6FnCBb -O try/S4U_filtered_dedup_mutationrates_utr.pdf"
When I try to run " /miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f S4U_filtered_dedup_mutationrates_utr.csv -O try.pdf", I get the following error message: "More columns than column names"
How can I fix this problem?Looking forward to your reply. Thanks a lot!
What happens if you run this?
/miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f /tmp/tmp6FnCBb -O try/S4U_filtered_dedup_mutationrates_utr.pdf
Is there any content in the /tmp/tmp6FnCBb file?
Thank you for your quick reply! I have tried as you instructed and I get the following error:
/miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f /tmp/tmp6FnCBb -O S4U_filtered_dedup_mutationrates_utr.pdf [1] "Sample: sample_0" diff lwr upr p adj A>G-A>C 0.0863673036 0.0825425688 0.090192038 0.000000e+00 A>T-A>C 0.0130495043 0.0092247695 0.016874239 3.730349e-14 C>A-A>C 0.0133128031 0.0094880643 0.017137542 0.000000e+00 C>G-A>C 0.0003009161 -0.0035238228 0.004125655 1.000000e+00 C>T-A>C 0.0522307509 0.0484060121 0.056055490 0.000000e+00 G>A-A>C 0.1083475175 0.1045227644 0.112172271 0.000000e+00 G>C-A>C 0.0311174670 0.0272927139 0.034942220 0.000000e+00 G>T-A>C 0.0165039359 0.0126791828 0.020328689 0.000000e+00 T>A-A>C 0.0689461204 0.0651213754 0.072770865 0.000000e+00 T>C-A>C 2.3305313344 2.3267065894 2.334356079 0.000000e+00 T>G-A>C 0.0305043325 0.0266795875 0.034329077 0.000000e+00 A>T-A>G -0.0733177993 -0.0771425340 -0.069493064 0.000000e+00 C>A-A>G -0.0730545005 -0.0768792393 -0.069229762 0.000000e+00 C>G-A>G -0.0860663875 -0.0898911263 -0.082241649 0.000000e+00 C>T-A>G -0.0341365526 -0.0379612915 -0.030311814 0.000000e+00 G>A-A>G 0.0219802140 0.0181554608 0.025804967 0.000000e+00 G>C-A>G -0.0552498365 -0.0590745897 -0.051425083 0.000000e+00 G>T-A>G -0.0698633676 -0.0736881208 -0.066038615 0.000000e+00 T>A-A>G -0.0174211832 -0.0212459281 -0.013596438 0.000000e+00 T>C-A>G 2.2441640308 2.2403392859 2.247988776 0.000000e+00 T>G-A>G -0.0558629711 -0.0596877161 -0.052038226 0.000000e+00 C>A-A>T 0.0002632988 -0.0035614401 0.004088038 1.000000e+00 C>G-A>T -0.0127485882 -0.0165733271 -0.008923849 1.180167e-13 C>T-A>T 0.0391812466 0.0353565078 0.043005985 0.000000e+00 G>A-A>T 0.0952980132 0.0914732601 0.099122766 0.000000e+00 G>C-A>T 0.0180679627 0.0142432096 0.021892716 0.000000e+00 G>T-A>T 0.0034544316 -0.0003703215 0.007279185 1.232030e-01 T>A-A>T 0.0558966161 0.0520718711 0.059721361 0.000000e+00 T>C-A>T 2.3174818301 2.3136570851 2.321306575 0.000000e+00 T>G-A>T 0.0174548282 0.0136300832 0.021279573 0.000000e+00 C>G-C>A -0.0130118870 -0.0168366300 -0.009187144 4.529710e-14 C>T-C>A 0.0389179478 0.0350932049 0.042742691 0.000000e+00 G>A-C>A 0.0950347144 0.0912099572 0.098859472 0.000000e+00 G>C-C>A 0.0178046639 0.0139799067 0.021629421 0.000000e+00 G>T-C>A 0.0031911328 -0.0006336244 0.007015890 2.129671e-01 T>A-C>A 0.0556333173 0.0518085682 0.059458066 0.000000e+00 T>C-C>A 2.3172185313 2.3133937822 2.321043280 0.000000e+00 T>G-C>A 0.0171915294 0.0133667803 0.021016278 0.000000e+00 C>T-C>G 0.0519298349 0.0481050919 0.055754578 0.000000e+00 G>A-C>G 0.1080466014 0.1042218442 0.111871359 0.000000e+00 G>C-C>G 0.0308165510 0.0269917937 0.034641308 0.000000e+00 G>T-C>G 0.0162030198 0.0123782626 0.020027777 0.000000e+00 T>A-C>G 0.0686452043 0.0648204552 0.072469953 0.000000e+00 T>C-C>G 2.3302304183 2.3264056693 2.334055167 0.000000e+00 T>G-C>G 0.0302034164 0.0263786673 0.034028165 0.000000e+00 G>A-C>T 0.0561167666 0.0522920094 0.059941524 0.000000e+00 G>C-C>T -0.0211132839 -0.0249380411 -0.017288527 0.000000e+00 G>T-C>T -0.0357268150 -0.0395515722 -0.031902058 0.000000e+00 T>A-C>T 0.0167153695 0.0128906204 0.020540119 0.000000e+00 T>C-C>T 2.2783005835 2.2744758344 2.282125333 0.000000e+00 T>G-C>T -0.0217264185 -0.0255511675 -0.017901669 0.000000e+00 G>C-G>A -0.0772300505 -0.0810548220 -0.073405279 0.000000e+00 G>T-G>A -0.0918435816 -0.0956683531 -0.088018810 0.000000e+00 T>A-G>A -0.0394013971 -0.0432261605 -0.035576634 0.000000e+00 T>C-G>A 2.2221838169 2.2183590535 2.226008580 0.000000e+00 T>G-G>A -0.0778431851 -0.0816679484 -0.074018422 0.000000e+00 G>T-G>C -0.0146135311 -0.0184383026 -0.010788760 0.000000e+00 T>A-G>C 0.0378286534 0.0340038900 0.041653417 0.000000e+00 T>C-G>C 2.2994138674 2.2955891040 2.303238631 0.000000e+00 T>G-G>C -0.0006131346 -0.0044378979 0.003211629 9.999961e-01 T>A-G>T 0.0524421845 0.0486174211 0.056266948 0.000000e+00 T>C-G>T 2.3140273985 2.3102026352 2.317852162 0.000000e+00 T>G-G>T 0.0140003965 0.0101756332 0.017825160 0.000000e+00 T>C-T>A 2.2615852140 2.2577604588 2.265409969 0.000000e+00 T>G-T>A -0.0384417879 -0.0422665431 -0.034617033 0.000000e+00 T>G-T>C -2.3000270019 -2.3038517571 -2.296202247 0.000000e+00 Error in dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared object ‘/miniconda3/envs/slamdunk/lib/R/library/stringi/libs/stringi.so’：: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /miniconda3/envs/slamdunk/lib/R/library/stringi/libs/../../../../libicui18n.so.58) Calls: do.call ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>
I tried to update libm.so.6 but I am using a public server which I do not have root permission to update lib file and make, what can I do to proceed?Thanks a lot!
Hm that looks like your local R installation is interfering with the conda environment. Did you try running it out of the Docker container?
I can run rate,snpeval,tccontent,tcperreadpos and tcperutrpos, but not utrrates and summary. May I know what are the possible reasons for such interference of these two scripts? I made a new environment with R only and ran, it still failed for these two scripts. I will try to run the scripts out of the Docker container.
Hi Tobias, I updated python to version 3.6 in the conda environment with R only and tried this command "/miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f /tmp/tmp6FnCBb -O try/S4U_filtered_dedup_mutationrates_utr.pdf". I got the following message and a pdf file with the plot: [1] "Sample: sample_0" diff lwr upr p adj A>G-A>C 0.0863673036 0.0825425688 0.090192038 0.000000e+00 A>T-A>C 0.0130495043 0.0092247695 0.016874239 3.730349e-14 C>A-A>C 0.0133128031 0.0094880643 0.017137542 0.000000e+00 C>G-A>C 0.0003009161 -0.0035238228 0.004125655 1.000000e+00 C>T-A>C 0.0522307509 0.0484060121 0.056055490 0.000000e+00 G>A-A>C 0.1083475175 0.1045227644 0.112172271 0.000000e+00 G>C-A>C 0.0311174670 0.0272927139 0.034942220 0.000000e+00 G>T-A>C 0.0165039359 0.0126791828 0.020328689 0.000000e+00 T>A-A>C 0.0689461204 0.0651213754 0.072770865 0.000000e+00 T>C-A>C 2.3305313344 2.3267065894 2.334356079 0.000000e+00 T>G-A>C 0.0305043325 0.0266795875 0.034329077 0.000000e+00 A>T-A>G -0.0733177993 -0.0771425340 -0.069493064 0.000000e+00 C>A-A>G -0.0730545005 -0.0768792393 -0.069229762 0.000000e+00 C>G-A>G -0.0860663875 -0.0898911263 -0.082241649 0.000000e+00 C>T-A>G -0.0341365526 -0.0379612915 -0.030311814 0.000000e+00 G>A-A>G 0.0219802140 0.0181554608 0.025804967 0.000000e+00 G>C-A>G -0.0552498365 -0.0590745897 -0.051425083 0.000000e+00 G>T-A>G -0.0698633676 -0.0736881208 -0.066038615 0.000000e+00 T>A-A>G -0.0174211832 -0.0212459281 -0.013596438 0.000000e+00 T>C-A>G 2.2441640308 2.2403392859 2.247988776 0.000000e+00 T>G-A>G -0.0558629711 -0.0596877161 -0.052038226 0.000000e+00 C>A-A>T 0.0002632988 -0.0035614401 0.004088038 1.000000e+00 C>G-A>T -0.0127485882 -0.0165733271 -0.008923849 1.180167e-13 C>T-A>T 0.0391812466 0.0353565078 0.043005985 0.000000e+00 G>A-A>T 0.0952980132 0.0914732601 0.099122766 0.000000e+00 G>C-A>T 0.0180679627 0.0142432096 0.021892716 0.000000e+00 G>T-A>T 0.0034544316 -0.0003703215 0.007279185 1.232030e-01 T>A-A>T 0.0558966161 0.0520718711 0.059721361 0.000000e+00 T>C-A>T 2.3174818301 2.3136570851 2.321306575 0.000000e+00 T>G-A>T 0.0174548282 0.0136300832 0.021279573 0.000000e+00 C>G-C>A -0.0130118870 -0.0168366300 -0.009187144 4.529710e-14 C>T-C>A 0.0389179478 0.0350932049 0.042742691 0.000000e+00 G>A-C>A 0.0950347144 0.0912099572 0.098859472 0.000000e+00 G>C-C>A 0.0178046639 0.0139799067 0.021629421 0.000000e+00 G>T-C>A 0.0031911328 -0.0006336244 0.007015890 2.129671e-01 T>A-C>A 0.0556333173 0.0518085682 0.059458066 0.000000e+00 T>C-C>A 2.3172185313 2.3133937822 2.321043280 0.000000e+00 T>G-C>A 0.0171915294 0.0133667803 0.021016278 0.000000e+00 C>T-C>G 0.0519298349 0.0481050919 0.055754578 0.000000e+00 G>A-C>G 0.1080466014 0.1042218442 0.111871359 0.000000e+00 G>C-C>G 0.0308165510 0.0269917937 0.034641308 0.000000e+00 G>T-C>G 0.0162030198 0.0123782626 0.020027777 0.000000e+00 T>A-C>G 0.0686452043 0.0648204552 0.072469953 0.000000e+00 T>C-C>G 2.3302304183 2.3264056693 2.334055167 0.000000e+00 T>G-C>G 0.0302034164 0.0263786673 0.034028165 0.000000e+00 G>A-C>T 0.0561167666 0.0522920094 0.059941524 0.000000e+00 G>C-C>T -0.0211132839 -0.0249380411 -0.017288527 0.000000e+00 G>T-C>T -0.0357268150 -0.0395515722 -0.031902058 0.000000e+00 T>A-C>T 0.0167153695 0.0128906204 0.020540119 0.000000e+00 T>C-C>T 2.2783005835 2.2744758344 2.282125333 0.000000e+00 T>G-C>T -0.0217264185 -0.0255511675 -0.017901669 0.000000e+00 G>C-G>A -0.0772300505 -0.0810548220 -0.073405279 0.000000e+00 G>T-G>A -0.0918435816 -0.0956683531 -0.088018810 0.000000e+00 T>A-G>A -0.0394013971 -0.0432261605 -0.035576634 0.000000e+00 T>C-G>A 2.2221838169 2.2183590535 2.226008580 0.000000e+00 T>G-G>A -0.0778431851 -0.0816679484 -0.074018422 0.000000e+00 G>T-G>C -0.0146135311 -0.0184383026 -0.010788760 0.000000e+00 T>A-G>C 0.0378286534 0.0340038900 0.041653417 0.000000e+00 T>C-G>C 2.2994138674 2.2955891040 2.303238631 0.000000e+00 T>G-G>C -0.0006131346 -0.0044378979 0.003211629 9.999961e-01 T>A-G>T 0.0524421845 0.0486174211 0.056266948 0.000000e+00 T>C-G>T 2.3140273985 2.3102026352 2.317852162 0.000000e+00 T>G-G>T 0.0140003965 0.0101756332 0.017825160 0.000000e+00 T>C-T>A 2.2615852140 2.2577604588 2.265409969 0.000000e+00 T>G-T>A -0.0384417879 -0.0422665431 -0.034617033 0.000000e+00 T>G-T>C -2.3000270019 -2.3038517571 -2.296202247 0.000000e+00 Warning messages: 1: Removed 39801 rows containing non-finite values (stat_boxplot). 2: Removed 39801 rows containing non-finite values (stat_boxplot). null device 1
But when I tried to plot other graphs using below command, it gave me the same error again: /miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f utrrates/control_filtered_dedup_mutationrates_utr.csv -O utrrates/control_filtered_dedup_mutationrates_utr.pdf Error in read.table(opt$rateTab, stringsAsFactors = FALSE, col.names = c("sample", : more columns than column names
Hi Tobias, I kinda found the way to solve this problem though I have no idea if it is correct, but I got the plots. R needs to run under python 3, so I created a new environment with R and python 3.6 only. I ran the commands "alleyoop utrrates -o try/ -r /ref/genome.fa -mq 27 -m -t 64 -b /ref/genome.gtf.bed -l 151 S4U_filtered_dedup.bam" using slamdunk and got the same errors described above, so I activated the R environment and ran the command using /tmp/tmpXXXX file "miniconda3/envs/slamdunk/lib/python2.7/site-packages/slamdunk/plot/globalRatePlotter.R -f /tmp/tmpXXX -O utrrates/control_filtered_dedup_mutationrates_utr.pdf", and I got the plots. Same solution for Summary option. Please let me know if this is the right way doing it.
Hi @cz0430 if this is working for you, I would go ahead with this. I strongly suspect if you used the Docker container, everything would work nicely, because that's at least how it is for me or if you have a clean conda installation without any interfering local environments, but that might be hard to get.