rnaseq
rnaseq copied to clipboard
Published
20 hours ago •
nf-core
nf-core
Trimgalore -> cutadapt -> OSError: [Errno 40] Too many levels of symbolic links
Description of the bug
Running rnaseq -r 3.8.1 with -profile docker on a local machine gives this strange error that has me stumped.
"Too many levels of symbolic links" should really only happen with recursive links, which I quadruple checked.
And what is even more strange, this happens inconsistently on seemingly random files (e.g. mate pair 1 works, mate pair 2 breaks, in the same process)
The previous run did not have this problem, the only thing that is changed is that I set the strandedness from reverse to unstranded.
Any ideas?
Command used and terminal output
COMMAND:
########
nextflow run nf-core/rnaseq -r 3.8.1 -profile docker --input samplesheet.csv --outdir results_fullrun --fasta $MMUGENOME --gtf $MMUANNOT --deseq2_vst
SAMPLESHEET:
############
`sample,fastq_1,fastq_2,strandedness
no001-0_2W_B6j_F_HSC_Y_A1_S1,/data/fass2/reads/fli/20211028_688_KLR/no001-0_2W_B6j_F_HSC_Y_A1_S1_R1_001.fastq.gz,/data/fass2/reads/fli/20211028_688_KLR/no001-0_2W_B6j_F_HSC_Y_A1_S1_R2_001.fastq.gz,unstranded
no002-0_2W_B6j_F_HSC_Y_A2_S2,/data/fass2/reads/fli/20211028_688_KLR/no002-0_2W_B6j_F_HSC_Y_A2_S2_R1_001.fastq.gz,/data/fass2/reads/fli/20211028_688_KLR/no002-0_2W_B6j_F_HSC_Y_A2_S2_R2_001.fastq.gz,unstranded
`
etc
OUTPUT:
#######
Error executing process > 'NFCORE_RNASEQ:RNASEQ:FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (no025-0_2W_B6j_F_
HSC_O_A5_S25)'
Caused by:
Process `NFCORE_RNASEQ:RNASEQ:FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (no025-0_2W_B6j_F_HSC_O_A5_S25)` t
erminated with an error exit status (1)
Command executed:
[ ! -f no025-0_2W_B6j_F_HSC_O_A5_S25_1.fastq.gz ] && ln -s no025-0_2W_B6j_F_HSC_O_A5_S25_R1_001.fastq
.gz no025-0_2W_B6j_F_HSC_O_A5_S25_1.fastq.gz
[ ! -f no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz ] && ln -s no025-0_2W_B6j_F_HSC_O_A5_S25_R2_001.fastq
.gz no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz
trim_galore \
--fastqc \
--cores 4 \
--paired \
--gzip \
\
\
\
\
no025-0_2W_B6j_F_HSC_O_A5_S25_1.fastq.gz \
no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz
cat <<-END_VERSIONS > versions.yml
"NFCORE_RNASEQ:RNASEQ:FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE":
trimgalore: $(echo $(trim_galore --version 2>&1) | sed 's/^.*version //; s/Last.*$//')
cutadapt: $(cutadapt --version)
END_VERSIONS
Command exit status:
1
Command output:
pigz 2.6
Command error:
return _open_gz(filename, mode, compresslevel, threads)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 505, in _open_gz
return igzip.open(filename, mode)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 87, in open
binary_file = IGzipFile(filename, gz_mode, compresslevel)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 146, in __init__
super().__init__(filename, mode, compresslevel, fileobj, mtime)
File "/usr/local/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 552, in run
with self._opener.xopen(self.path, 'rb') as f:
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 186, in xopen
f = open_raise_limit(
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 51, in open_raise_limit
f = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 609, in xopen
return _open_gz(filename, mode, compresslevel, threads)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 505, in _open_gz
return igzip.open(filename, mode)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 87, in open
binary_file = IGzipFile(filename, gz_mode, compresslevel)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 146, in __init__
super().__init__(filename, mode, compresslevel, fileobj, mtime)
File "/usr/local/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 626, in run
raise e
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
Traceback (most recent call last):
File "/usr/local/bin/cutadapt", line 10, in <module>
sys.exit(main_cli())
File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 848, in main_cli
main(sys.argv[1:])
File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 913, in main
stats = r.run()
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 825, in run
raise e
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
Cutadapt terminated with exit signal: '256'.
Terminating Trim Galore run, please check error message(s) to get an idea what went wrong...
Work dir:
/data/fass5/sebastian/dietary_restriction_2020/RNASeq/nfcore_test/work/e8/2a43d104157343f39fa46fa64c27
76
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
TRIMGALORE LOG: (mate pair 1 finished successfully, omitted here)
###############
SUMMARISING RUN PARAMETERS
==========================
Input filename: no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz
Trimming mode: paired-end
Trim Galore version: 0.6.7
Cutadapt version: 3.4
Python version: could not detect
Number of cores used for trimming: 4
Quality Phred score cutoff: 20
Quality encoding type selected: ASCII+33
Adapter sequence: 'CTGTCTCTTATA' (Nextera Transposase sequence; auto-detected)
Maximum trimming error rate: 0.1 (default)
Minimum required adapter overlap (stringency): 1 bp
Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp
Running FastQC on the data once trimming has completed
Output file(s) will be GZIP compressed
Cutadapt seems to be fairly up-to-date (version 3.4). Setting -j -j 4
Writing final adapter and quality trimmed output to no025-0_2W_B6j_F_HSC_O_A5_S25_2_trimmed.fq.gz
>>> Now performing quality (cutoff '-q 20') and adapter trimming in a single pass for the adapter sequ
ence: 'CTGTCTCTTATA' from file no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz <<<
This is cutadapt 3.4 with Python 3.9.6
Command line parameters: -j 4 -e 0.1 -q 20 -O 1 -a CTGTCTCTTATA no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz
Processing reads on 4 cores in single-end mode ...
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 552, in run
with self._opener.xopen(self.path, 'rb') as f:
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 186, in xopen
f = open_raise_limit(
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 51, in open_raise_limit
f = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 609, in xopen
return _open_gz(filename, mode, compresslevel, threads)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 505, in _open_gz
return igzip.open(filename, mode)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 87, in open
binary_file = IGzipFile(filename, gz_mode, compresslevel)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 146, in __init__
super().__init__(filename, mode, compresslevel, fileobj, mtime)
File "/usr/local/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 552, in run
with self._opener.xopen(self.path, 'rb') as f:
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 186, in xopen
f = open_raise_limit(
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 51, in open_raise_limit
f = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 609, in xopen
return _open_gz(filename, mode, compresslevel, threads)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 505, in _open_gz
return igzip.open(filename, mode)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 87, in open
binary_file = IGzipFile(filename, gz_mode, compresslevel)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 146, in __init__
super().__init__(filename, mode, compresslevel, fileobj, mtime)
File "/usr/local/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 552, in run
with self._opener.xopen(self.path, 'rb') as f:
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 186, in xopen
f = open_raise_limit(
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 51, in open_raise_limit
f = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 609, in xopen
return _open_gz(filename, mode, compresslevel, threads)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 505, in _open_gz
return igzip.open(filename, mode)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 87, in open
binary_file = IGzipFile(filename, gz_mode, compresslevel)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 146, in __init__
super().__init__(filename, mode, compresslevel, fileobj, mtime)
File "/usr/local/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 552, in run
with self._opener.xopen(self.path, 'rb') as f:
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 186, in xopen
f = open_raise_limit(
File "/usr/local/lib/python3.9/site-packages/cutadapt/utils.py", line 51, in open_raise_limit
f = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 609, in xopen
return _open_gz(filename, mode, compresslevel, threads)
File "/usr/local/lib/python3.9/site-packages/xopen/__init__.py", line 505, in _open_gz
return igzip.open(filename, mode)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 87, in open
binary_file = IGzipFile(filename, gz_mode, compresslevel)
File "/usr/local/lib/python3.9/site-packages/isal/igzip.py", line 146, in __init__
super().__init__(filename, mode, compresslevel, fileobj, mtime)
File "/usr/local/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 626, in run
raise e
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
Traceback (most recent call last):
File "/usr/local/bin/cutadapt", line 10, in <module>
sys.exit(main_cli())
File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 848, in main_cli
main(sys.argv[1:])
File "/usr/local/lib/python3.9/site-packages/cutadapt/__main__.py", line 913, in main
stats = r.run()
File "/usr/local/lib/python3.9/site-packages/cutadapt/pipeline.py", line 825, in run
raise e
OSError: [Errno 40] Too many levels of symbolic links: 'no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz'
Cutadapt terminated with exit signal: '256'.
Terminating Trim Galore run, please check error message(s) to get an idea what went wrong...
STATE OF THE PROCESS WORK FOLDER:
#################################
drwxrwxr-x 2 ya86gul bioinf3 16 22. Jul 13:10 .
drwxrwxr-x 27 ya86gul bioinf3 25 22. Jul 09:40 ..
-rw-rw-r-- 1 ya86gul bioinf3 0 22. Jul 13:01 .command.begin
-rw-rw-r-- 1 ya86gul bioinf3 13K 22. Jul 13:10 .command.err
-rw-rw-r-- 1 ya86gul bioinf3 13K 22. Jul 13:10 .command.log
-rw-rw-r-- 1 ya86gul bioinf3 9 22. Jul 13:01 .command.out
-rw-rw-r-- 1 ya86gul bioinf3 11K 22. Jul 13:01 .command.run
-rw-rw-r-- 1 ya86gul bioinf3 741 22. Jul 13:01 .command.sh
-rw-rw-r-- 1 ya86gul root 0 22. Jul 13:01 .command.trace
-rw-rw-r-- 1 ya86gul bioinf3 1 22. Jul 13:10 .exitcode
lrwxrwxrwx 1 ya86gul root 45 22. Jul 13:01 no025-0_2W_B6j_F_HSC_O_A5_S25_1.fastq.gz -> no025-0_2W_B6j_F_HSC_O_A5_S25_R1_001.fastq.gz
-rw-rw-r-- 1 ya86gul root 4,3K 22. Jul 13:10 no025-0_2W_B6j_F_HSC_O_A5_S25_1.fastq.gz_trimming_report.txt
-rw-rw-r-- 1 ya86gul root 2,1G 22. Jul 13:10 no025-0_2W_B6j_F_HSC_O_A5_S25_1_trimmed.fq.gz
lrwxrwxrwx 1 ya86gul root 45 22. Jul 13:01 no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz -> no025-0_2W_B6j_F_HSC_O_A5_S25_R2_001.fastq.gz
-rw-rw-r-- 1 ya86gul root 6,5K 22. Jul 13:10 no025-0_2W_B6j_F_HSC_O_A5_S25_2.fastq.gz_trimming_report.txt
-rw-rw-r-- 1 ya86gul root 20 22. Jul 13:10 no025-0_2W_B6j_F_HSC_O_A5_S25_2_trimmed.fq.gz
lrwxrwxrwx 1 ya86gul bioinf3 84 22. Jul 13:01 no025-0_2W_B6j_F_HSC_O_A5_S25_R1_001.fastq.gz -> /data/fass2/reads/fli/20211028_688_KLR/no025-0_2W_B6j_F_HSC_O_A5_S25_R1_001.fastq.gz
lrwxrwxrwx 1 ya86gul bioinf3 84 22. Jul 13:01 no025-0_2W_B6j_F_HSC_O_A5_S25_R2_001.fastq.gz -> /data/fass2/reads/fli/20211028_688_KLR/no025-0_2W_B6j_F_HSC_O_A5_S25_R2_001.fastq.gz
Relevant files
No response
System information
Nextflow version: 22.04.0 build 5697
nf-core/rnaseq version: 3.8.1
Hardware: local Debian machine, 48 threads, 500GB RAM
Container engine: docker
This looks to be docker specific, did not run into this with -profile singularity
Hi @RaverJay ! Did you manage to figure out why this was happening? I haven't seen this issue before and will need a way to reproduce this issue locally in-order to investigate further. Will close this for now but please re-open if you have any updates.
Please feel free to reach out to us in the #rnaseq channel on the nf-core Slack for pipeline related issues in the future for more real-time help. Thanks!
