OpenGene
segmentation fault: core dumped during adapter trimming
Hello, I am using fastp to clean my reads before alignment. I am currently processing 48 samples as part of a bash workflow, and two files have given me a segmentation fault error, and they are not the two largest files of the 48 (each read file is ~1.2GB). I watch my memory consumption and it does not get maxed out (64GB RAM available). Here is the command I used:
for SAMPLE_ID in ${SAMPLENAMES}; do
printf "\nPIPELINE ALERT: Trimming %s\n" "${SAMPLE_ID}"
${FASTP} --in1 ${RAWREADSDIR}/${SAMPLE_ID}_R1.fastq.gz --in2 ${RAWREADSDIR}/${SAMPLE_ID}_R2.fastq.gz
--out1 ${OUTDIR}/${POP}/Clean_fastqs/${SAMPLE_ID}_R1_clean.fq.gz --out2 ${OUTDIR}/${POP}/Clean_fastqs/${SAMPLE_ID}_R2_clean.fq.gz
--detect_adapter_for_pe -q ${PHREDQ} -u ${BADBASES} -l ${MINREADLENGTH} --correction
--cut_right --cut_right_window_size 4 --cut_right_mean_quality 20
--html "${OUTDIR}/${POP}/FastqQC/${SAMPLE_ID}_fastq_trim_report.html"
rm ./fastp.json;
done
And here is the screen output (with some of my own workflow output) with the error for both files:
PIPELINE ALERT: Trimming CW-N-008 Detecting adapter sequence for read1... No adapter detected for read1
Detecting adapter sequence for read2...
Illumina TruSeq Adapter Read 2 AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT
Error to read gzip file 0809.sh: line 7: 21733 Segmentation fault (core dumped) ${FASTP} --in1 ${RAWREADSDIR}/${SAMPLE_ID}_R1.fastq.gz --in2 ${RAWREADSDIR}/${SAMPLE_ID}_R2.fastq.gz --out1 ${OUTDIR}/${POP}/Clean_fastqs/${SAMPLE_ID}_R1_clean.fq.gz --out2 ${OUTDIR}/${POP}/Clean_fastqs/${SAMPLE_ID}_R2_clean.fq.gz --detect_adapter_for_pe -q ${PHREDQ} -u ${BADBASES} -l ${MINREADLENGTH} --correction --cut_right --cut_right_window_size 4 --cut_right_mean_quality 20 --html "${OUTDIR}/${POP}/FastqQC/${SAMPLE_ID}_fastq_trim_report.html" rm: cannot remove './fastp.json': No such file or directory
PIPELINE ALERT: Trimming CW-N-009 Detecting adapter sequence for read1... No adapter detected for read1
Detecting adapter sequence for read2...
Illumina TruSeq Adapter Read 2 AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT
Error to read gzip file 0809.sh: line 7: 21890 Segmentation fault (core dumped) ${FASTP} --in1 ${RAWREADSDIR}/${SAMPLE_ID}_R1.fastq.gz --in2 ${RAWREADSDIR}/${SAMPLE_ID}_R2.fastq.gz --out1 ${OUTDIR}/${POP}/Clean_fastqs/${SAMPLE_ID}_R1_clean.fq.gz --out2 ${OUTDIR}/${POP}/Clean_fastqs/${SAMPLE_ID}_R2_clean.fq.gz --detect_adapter_for_pe -q ${PHREDQ} -u ${BADBASES} -l ${MINREADLENGTH} --correction --cut_right --cut_right_window_size 4 --cut_right_mean_quality 20 --html "${OUTDIR}/${POP}/FastqQC/${SAMPLE_ID}_fastq_trim_report.html" rm: cannot remove './fastp.json': No such file or directory
I'm running fastp v 0.21.0 on an Ubuntu 18.x machine. Please let me know what you think the error might be due to.
Many thanks!
Follow-up: I just tried the same commands on the same files with v 0.23 and it doesn't give the segmentation fault error, but files that are 30-60MB (gzipped) are produced and fastp hangs and doesn't complete processing the file.
Hello, I was able to determine that this was occurring because the fastq.gz files were damaged (truncated/incomplete), so it wasn't actually a problem with fastp! But, it seems like it would be nice if the fastp error message indicated that vs. just a core dump. I'm assuming that could be written into the code. Thanks for your time.