pod5-file-format icon indicating copy to clipboard operation
pod5-file-format copied to clipboard

pod5 convert fast5 does not gracefully exit after an error when using "strict"

Open billytcl opened this issue 2 years ago • 1 comments

Python 3.7, pod5 0.2.4

Using this command to catch exceptions on a SLURM HPC cluster:

pod5 convert fast5 --strict -f -o pod5/converted.pod5 -r -t 20 ./Seq_Output/

It catches an error early on, but seems to hang and not end the job:

(base) [billylau@sh02-ln04 login /scratch/groups/hanleeji/20221104_PRM_1295]$ cat slurm-30777430.out
Converting 30037 Fast5s:   5%|4         | 5752514/120088914 [10:27<3:27:59, 9161.58Reads/s]ERROR:pod5:Input PAM91900_pass_barcode01_9bb047c3_5303.fast5 is not a multi-read fast5
Converting 30037 Fast5s:   5%|4         | 5752914/120088914 [10:28<3:28:01, 9160.68Reads/s]ERROR:pod5:Encountered an exception in Seq_Output/20221104_1730_3B_PAM91900_1eee8ef4/fast5_pass/barcode01/PAM91900_pass_barcode01_9bb047c3_5303.fast5 - Seq_Output/20221104_1730_3B_PAM91900_1eee8ef4/fast5_pass/barcode01/PAM91900_pass_barcode01_9bb047c3_5303.fast5 is not a multi-read fast5 file.
NoneType: None
Seq_Output/20221104_1730_3B_PAM91900_1eee8ef4/fast5_pass/barcode01/PAM91900_pass_barcode01_9bb047c3_5303.fast5 is not a multi-read fast5 file.
Converting 30037 Fast5s:   5%|4         | 5753314/120088914 [10:28<3:28:01, 9160.76Reads/s]
CRITICAL:pod5:Exception raised and --strict set
An unexpected error occurred: Seq_Output/20221104_1730_3B_PAM91900_1eee8ef4/fast5_pass/barcode01/PAM91900_pass_barcode01_9bb047c3_5303.fast5 is not a multi-read fast5 file.

POD5 has encountered an error: 'Seq_Output/20221104_1730_3B_PAM91900_1eee8ef4/fast5_pass/barcode01/PAM91900_pass_barcode01_9bb047c3_5303.fast5 is not a multi-read fast5 file.'

For detailed information set POD5_DEBUG=1'

It's hung for about 30 minutes with no change to the pod5 file. SSHing into the node itself shows that the CPU is not doing anything, and the job hasn't exited.

billytcl avatar Sep 14 '23 07:09 billytcl

Thanks for reporting this @billytcl , we'll take a look at improving the error handling of the converter

HalfPhoton avatar Sep 14 '23 15:09 HalfPhoton