HTStream icon indicating copy to clipboard operation
HTStream copied to clipboard

"no such file or directory" error

Open Emmashipman opened this issue 1 year ago • 2 comments

Hello,

I am trying to use HTStream. I have based my code off the tutorial here: https://ucdavis-bioinformatics-training.github.io/2020-mRNA_Seq_Workshop/data_reduction/01-preproc_htstream_mm

and I referred to the sbatch example script heavily:

#!/bin/bash

#SBATCH --job-name=htstream # Job name
#SBATCH --nodes=1
#SBATCH --ntasks=9
#SBATCH --time=60
#SBATCH --mem=3000 # Memory pool for all cores (see also --mem-per-cpu)
#SBATCH --partition=production
#SBATCH --reservation=mrnaseq_workshop
#SBATCH --account=mrnaseq_workshop
#SBATCH --array=1-22
#SBATCH --output=slurmout/htstream_%A_%a.out # File to which STDOUT will be written
#SBATCH --error=slurmout/htstream_%A_%a.err # File to which STDERR will be written
#SBATCH --mail-type=ALL
#SBATCH [email protected]

start=`date +%s`
echo $HOSTNAME
echo "My SLURM_ARRAY_TASK_ID: " $SLURM_ARRAY_TASK_ID

sample=`sed "${SLURM_ARRAY_TASK_ID}q;d" samples.txt`

inpath="00-RawData"
outpath="01-HTS_Preproc"
[[ -d ${outpath} ]] || mkdir ${outpath}
[[ -d ${outpath}/${sample} ]] || mkdir ${outpath}/${sample}

echo "SAMPLE: ${sample}"

module load htstream/1.3.2

call="hts_Stats -L ${outpath}/${sample}/${sample}.json -N 'initial stats' \
          -1 ${inpath}/${sample}/*R1.fastq.gz \
          -2 ${inpath}/${sample}/*R2.fastq.gz | \
      hts_SeqScreener -A ${outpath}/${sample}/${sample}.json -N 'screen phix' | \
      hts_SeqScreener -A ${outpath}/${sample}/${sample}.json -N 'count the number of rRNA reads'\
          -r -s References/mouse_rrna.fasta | \
      hts_SuperDeduper -A ${outpath}/${sample}/${sample}.json -N 'remove PCR duplicates' | \
      hts_AdapterTrimmer -A ${outpath}/${sample}/${sample}.json -N 'trim adapters' | \
      hts_PolyATTrim --no-left --skip_polyT -A ${outpath}/${sample}/${sample}.json -N 'remove polyAT tails' | \
      hts_NTrimmer -A ${outpath}/${sample}/${sample}.json -N 'remove any remaining N characters' | \
      hts_QWindowTrim -A ${outpath}/${sample}/${sample}.json -N 'quality trim the ends of reads' | \
      hts_LengthFilter -A ${outpath}/${sample}/${sample}.json -N 'remove reads < 50bp' \
          -n -m 50 | \
      hts_Stats -A ${outpath}/${sample}/${sample}.json -N 'final stats' \
          -f ${outpath}/${sample}/${sample}"

echo $call
eval $call

end=`date +%s`
runtime=$((end-start))
echo $runtime

When I run my script, I get an error for the line that has the
 hts_Stats -L 
command that says "no such file or directory"
I have tried running the program with and without a blank file that I've named but the error is the same. It seems the program cannot create or see the log.

Here is how my script looks:


#!/bin/bash

#SBATCH --job-name=htstream_N #SBATCH --nodes=1 #SBATCH --ntasks=10 #SBATCH --time=12:00:00 #SBATCH --mem=32000 #SBATCH --output=slurmout/htstream_N_%A_%a.out # File to which STDOUT will be written #SBATCH --error=slurmout/htstream_N_%A_%a.err # File to which STDERR #SBATCH --mail-user= #SBATCH --mail-type=ALL

start=date +%s

echo $HOSTNAME echo "My SLURM_ARRAY_TASK_ID: " $SLURM_ARRAY_TASK_ID

module load deprecated/HTStream/1.3.3

call=" hts_Stats -L /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "initial stats" \

-1 /home/user/RNASEQDATADUMP/NWTsalt8/NWTsalt8_CKDL220024596-1A_HJ2W5BBXX_L6_1.fq.gz
-2 /home/user/RNASEQDATADUMP/NWTsalt8/NWTsalt8_CKDL220024596-1A_HJ2W5BBXX_L6_1.fq.gz | \

hts_SeqScreener -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "screen phix" |
hts_SeqScreener -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "count rRNAs"
--record --seq /home/user/RNASEQDATADUMP/wheat_rrna.fasta |
hts_Overlapper -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "overlap PE reads" |
hts_AdapterTrimmer -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "trim adapters" |
hts_PolyATTrim -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "trim polyAT" |
hts_NTrimmer -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "remove N chars" |
hts_QWindowTrim -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "quality trim ends" |
hts_LengthFilter -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "remove reads < 50 bp"
-n -m 50 |
hts_Stats -A /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "final stats" \ -f /home/user/RNASEQDATADUMP/HTS_outputs/N_hts_paired.htstream "

echo $call eval $call

end=date +%s runtime=$((end-start)) echo $runtime


My filepath is correct and it fails on the line for hts_Stats -L /home/user/RNASEQDATADUMP/HTS_outputs/N_htsStats.json -N "initial stats" \

I would really appreciate any help. my IT department suggested something could be being incorrectly send to my slurmout folder, but the .out and .err files are writing normally. 

The error is like this:
/var/spool/slurmd/job66287601/slurm_script: line 23: 50: No such file or directory

I am not a programmer so I am not sure how to proceeed. I also tried this with the 1.3.2 version and had the same problem.

Thank you for any insights or corrections.

Emmashipman avatar Jun 08 '23 01:06 Emmashipman