Error in pod5 conversion

[KristinaGagalova]

Multiple issues for pod5 conversion

Hi, I have a dataset of fast5 reads that I am trying to convert to pod5 with the following command:

# path to fast5 files, exists
pod5 convert fast5 \
       mydir/*.fast5 \
        --output converted.nanopore.pod5

During the run, I had the following issues.

....is not a multi-read fast5 file.
"Unable to synchronously open object (object 'Raw' doesn't exist)"

I tried to run several tools on the dataset, but without success.

1. Convert to multi-read fasta - ERROR

ERROR:ont_fast5_api.conversion_tools.single_to_multi_fast5:Failed to initialise single-read Fast5File: '.../FAL03712_a37eb560_171.fast5': Cannot input MultiRead files to single_to_multi:

2. Troubleshooting of corrupted reads using the script find_fas5_issues.py mentioned in github.com/nanoporetech/pod5-file-format/issues/97

Searching .../2024_PN155_nanopore_raw_fast5 for fast5 files
double free or corruption (out)
Aborted (core dumped)

At this stage, I don't know how to tackle this issue. Could you please provide some feedback?

Logs

FAL03712_a37eb560_169.fast5 is not a multi-read fast5 file. FAL03712_a37eb560_20.fast5 is not a multi-read fast5 file. Converting 435 Fast5s: 0%| | 0/1676000 [00:00<?, ?Reads/s]free(): invalid pointer Converting 435 Fast5s: 0%| | 0/1668000 [00:01<?, ?Reads/s]free(): invalid pointer Converting 435 Fast5s: 0%| | 6800/1668000 [00:18<1:14:35, 371.20Reads/s]free(): invalid size An unexpected error occurred: No progress in 600 seconds - quitting Converting 435 Fast5s: 0%| | 7200/1668000 [10:18<39:37:47, 11.64Reads/s] POD5 has encountered an error: 'No progress in 600 seconds - quitting'

Specifications

• Pod5 Version: 0.3.23
• Python Version:
• Platform: SUSE Linux Enterprise Server 15 SP5

[KristinaGagalova]

I just wanted to attach the complete log file:

2025-02-23--17-12-35-main-pod5.log

Since the multifast5 error message, I also tried to convert the single to multiple fast5 reads with github.com/nanoporetech/ont_fast5_api.

 single_to_multi_fast5 --input_path /path/to/fast5/ --save_path /path/to/out/ --recursive

Which gave

ERROR:ont_fast5_api.conversion_tools.single_to_multi_fast5:Failed to initialise single-read Fast5File: '/path/to/name.fast5': Cannot input MultiRead files to single_to_multi: '/path/to/name.fast5'

I also tried to remove the files that may have caused the issues, the ones flagged as "single-read fast5," but nothing changed. Other read files caused problems.

Please let me know if this helps

[0x55555555]

Hi @KristinaGagalova,

What is the origin of these files? Is it possible they are corrupt (given the output of find_fas5_issues).

If you open the file in a tool like hdfview or h5ls, what is the structure of the file?

Thanks

• George

[KristinaGagalova]

Hi, Looks like we have a series of errors

h5ls -d -e FAL03712_a37eb560_104.fast5
HDF5-DIAG: Error detected in HDF5 (1.10.7) thread 1:
  #000: ../../../src/H5L.c line 1226 in H5Literate_by_name(): link iteration failed
    major: Links
    minor: Iteration failed
  #001: ../../../src/H5L.c line 3350 in H5L__iterate(): link iteration failed
    major: Links
    minor: Iteration failed
  #002: ../../../src/H5Gint.c line 853 in H5G_iterate(): error iterating over links
    major: Symbol table
    minor: Iteration failed
  #003: ../../../src/H5Gobj.c line 677 in H5G__obj_iterate(): can't iterate over dense links
    major: Symbol table
    minor: Iteration failed
  #004: ../../../src/H5Gdense.c line 1020 in H5G__dense_iterate(): error building table of links
    major: Symbol table
    minor: Can't get value
  #005: ../../../src/H5Gdense.c line 820 in H5G__dense_build_table(): error iterating over links
    major: Symbol table
    minor: Can't move to next iterator location
  #006: ../../../src/H5Gdense.c line 1011 in H5G__dense_iterate(): link iteration failed
    major: Symbol table
    minor: Iteration failed
  #007: ../../../src/H5B2.c line 438 in H5B2_iterate(): node iteration failed
    major: B-Tree node
    minor: Unable to list node
  #008: ../../../src/H5B2int.c line 1569 in H5B2__iterate_node(): node iteration failed
    major: B-Tree node
    minor: Unable to list node
  #009: ../../../src/H5B2int.c line 1569 in H5B2__iterate_node(): node iteration failed
    major: B-Tree node
    minor: Unable to list node
  #010: ../../../src/H5B2int.c line 1574 in H5B2__iterate_node(): iterator function failed
    major: B-Tree node
    minor: Unable to list node
  #011: ../../../src/H5Gdense.c line 903 in H5G__dense_iterate_bt2_cb(): heap op callback failed
    major: Symbol table
    minor: Can't operate on object
  #012: ../../../src/H5HF.c line 707 in H5HF_op(): can't operate on object from fractal heap
    major: Heap
    minor: Can't operate on object
  #013: ../../../src/H5HFman.c line 536 in H5HF__man_op(): unable to operate on heap object
    major: Heap
    minor: Can't operate on object
  #014: ../../../src/H5HFman.c line 400 in H5HF__man_op_real(): unable to protect fractal heap direct block
    major: Heap
    minor: Unable to protect metadata
  #015: ../../../src/H5HFdblock.c line 496 in H5HF__man_dblock_protect(): unable to protect fractal heap direct block
    major: Heap
    minor: Unable to protect metadata
  #016: ../../../src/H5AC.c line 1517 in H5AC_protect(): H5C_protect() failed
    major: Object cache
    minor: Unable to protect metadata
  #017: ../../../src/H5C.c line 2501 in H5C_protect(): can't load entry
    major: Object cache
    minor: Unable to load metadata into cache
  #018: ../../../src/H5C.c line 7647 in H5C_load_entry(): incorrect metadatda checksum after all read attempts
    major: Object cache
    minor: Read failed
H5tools-DIAG: Error detected in HDF5:tools (1.10.7) thread 1:
  #000: ../../../../tools/lib/h5trav.c line 1060 in h5trav_visit(): traverse failed
    major: Failure in tools library
    minor: error in function
  #001: ../../../../tools/lib/h5trav.c line 295 in traverse(): H5Literate_by_name failed
    major: Failure in tools library
    minor: error in function

Any suggestions on how to treat these file or fix them?

[0x55555555]

Hello,

Do you know the origin of the files? If there is an original uncorrupt source you could copy from there?

Thanks,

• George

[KristinaGagalova]

I believe these are copies from the original, but I expect something is wrong with the originals, too. Do you think the error message shows corrupted files? I will need to check better what has happened there Thanks for the feedback

[0x55555555]

Yes, from the messages posted I believe there is a problem with the files themselves - I cant be too sure what has happened though, sorry.

• George

[Patricie34]

Hi, did anybody proceed with this issue? Because, I've just encountered exactly the same error in one of my old samples, which I need to convert from fast5 to pod5. @KristinaGagalova how did you solve it? Thanks!

[KristinaGagalova]

Hi @Patricie34, In my case the files were corrupted. I found an older back up and it worked out