dorado
dorado copied to clipboard
nanoporetech
dorado v0.9.5 GLIB_2.29 dependency?
Seems like the upgrade in torch has then lost support on our RockyLinux system due to GLIB dependencies. I grabbed the package like this:
$ wget https://cdn.oxfordnanoportal.com/software/analysis/dorado-0.9.5-linux-x64.tar.gz
$ tar xvf dorado-0.9.5-linux-x64.tar.gz
$ ./dorado-0.9.0-linux-x64/bin/dorado --version
[2025-04-01 19:33:14.955] [info] Running: "--version"
0.9.0+9dc15a8
$ ./dorado-0.9.5-linux-x64/bin/dorado --version
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by ./dorado-0.9.5-linux-x64/bin/dorado)
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./dorado-0.9.5-linux-x64/bin/dorado)
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /bioinfo/apps/all_apps/dorado-0.9.5-linux-x64/bin/../lib/libdorado_torch_lib.so)
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /bioinfo/apps/all_apps/dorado-0.9.5-linux-x64/bin/../lib/libdorado_torch_lib.so)
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /bioinfo/apps/all_apps/dorado-0.9.5-linux-x64/bin/../lib/../lib/libdorado_cudnn_lib.so)
$ ldd --version
ldd (GNU libc) 2.28
Hej @nextgenusfs , I ran into the same issue and helped myself by building an apptainer image for dorado (see snippets from the recipe below)
To run dorado using apptainer, you must tell it where to find the 12.8 cuda libs:
apptainer exec --cleanenv --nv --bind /path/to/cuda/12.8.0/lib64/ --bind /path/to/cuda/12.8.0/bin/ dorado_v0.9.5.sif dorado basecaller
Best! Philipp
The apptainer file:
bootstrap: docker From: ubuntu:noble
%post export DEBIAN_FRONTEND=noninteractive apt-get update -y apt install -y build-essential zlib1g-dev libbz2-dev wget
newVersion="0.9.5" cd /opt wget https://cdn.oxfordnanoportal.com/software/analysis/dorado-${newVersion}-linux-x64.tar.gz tar xzvf dorado-${newVersion}-linux-x64.tar.gz rm dorado-${newVersion}-linux-x64.tar.gz cd /usr/local/bin/ ln -s /opt/dorado-0.9.5-linux-x64/bin/dorado .
%runscript dorado "${@}"
Thanks @phpeters for the suggestion, but unfortunately a container won't work in this particular system.
This particular call (dorado --version) didn't have the GPU attached. When we run it on data we launch on an instance with GPU (here I was just checking if install was working, notably different behavior than v0.9.0) but I assume it wouldn't matter as the system doesn't have GLIBC_2.29 when CUDA/GPU is attached. Our base image currently has CUDA 12.2 and 535.183.01- so I know I need to upgrade to 12.8 as it said in the release docs, but didn't want to do that if it isn't going to run on our image as is. Rocky Linux 8 is shipped with GLIB 2.28.
You only need to meet the minimum driver requirement (which 535 does) to run dorado (we have moved to using the 12.8 cuda toolkit for building).
I'll assume compute capability < 9.0. Here's a build https://cdn.oxfordnanoportal.com/software/analysis/dorado-0.9.5-linux-x64-cuda-11.8.tar.gz
Awesome, thanks. Initial test seems to work, will try a base calling run shortly.
$ ./dorado-0.9.5-linux-x64-cuda-11.8/bin/dorado --version
[2025-04-02 16:14:04.024] [info] Running: "--version"
0.9.5+bd7fb21
This is working for me as well with the cuda-11.8 , thanks a lot.
You only need to meet the minimum driver requirement (which 535 does) to run dorado (we have moved to using the 12.8 cuda toolkit for building).
I'll assume compute capability < 9.0. Here's a build https://cdn.oxfordnanoportal.com/software/analysis/dorado-0.9.5-linux-x64-cuda-11.8.tar.gz
This release raises two questions, i) will it be mandatory to update the dependencies in the future (I imagine so as you did the build in version 12.8). ii) Does using the shared version with cuda 11.8 make a difference in performance?
i) Yes, we've moved from building in a ~10-year-old environment to ~5 (which is reasonable). We need to increase our toolkit versions to support the latest software and hardware features, so we are somewhat at the mercy of nvidia here.
ii) The CUDA 11.8 build has no support for Blackwell, and hac will be slower on Hopper and Orin but I would really hope in practice nobody has Hopper with old drivers and OS (glibc).
Looks like base calling working okay with the dorado-0.9.5-linux-x64-cuda-11.8 above. However, I'm seeing an early termination of dorado correct, its not erroring but seems like the overlaps are not working properly (whether that is this specific build or an issue with 0.9.5 in general I can't say).
This is running on cpu (32 cpus / 256 GB RAM):
$ dorado correct --verbose --to-paf sample.ont.fastq 1> sample.overlaps.paf
[2025-04-03 16:02:24.992] [info] Running: "correct" "--verbose" "--to-paf" "sample.ont.fastq"
[2025-04-03 16:02:25.035] [info] Failed to load NVML
[2025-04-03 16:02:25.039] [debug] Aligner threads 32, corrector threads 8, writer threads 1
[2025-04-03 16:02:25.039] [debug] furthest_skip_header = '', furthest_skip_id = -1
[2025-04-03 16:02:25.039] [debug] Initialized index options.
[2025-04-03 16:02:25.039] [debug] Loading index...
[2025-04-03 16:03:42.528] [debug] Loaded index with 322220 target seqs
[2025-04-03 16:03:43.999] [debug] Loaded mm2 index.
[2025-04-03 16:03:43.999] [debug] Initial index block set to: 0
[2025-04-03 16:03:43.999] [info] Starting
[2025-04-03 16:03:43.999] [debug] Align with index 0
[2025-04-03 16:03:45.457] [debug] Read 10000 reads
[2025-04-03 16:03:45.459] [debug] Alignments processed 10001, total m_corrected_records size 0.17516327 MB
[2025-04-03 16:03:47.015] [debug] Read 20000 reads
[2025-04-03 16:03:47.017] [debug] Alignments processed 20000, total m_corrected_records size 0.26559448 MB
[2025-04-03 16:03:48.531] [debug] Read 30000 reads
[2025-04-03 16:03:48.532] [debug] Alignments processed 30000, total m_corrected_records size 0.27727127 MB
[2025-04-03 16:03:49.999] [debug] Read 40000 reads
[2025-04-03 16:03:50.001] [debug] Alignments processed 40000, total m_corrected_records size 0.33927536 MB
[2025-04-03 16:03:51.571] [debug] Read 50000 reads
[2025-04-03 16:03:51.573] [debug] Alignments processed 50000, total m_corrected_records size 0.5354309 MB
[2025-04-03 16:03:53.050] [debug] Read 60000 reads
[2025-04-03 16:03:53.051] [debug] Alignments processed 60000, total m_corrected_records size 0.6121025 MB
[2025-04-03 16:03:54.499] [debug] Read 70000 reads
[2025-04-03 16:03:54.499] [debug] Alignments processed 70001, total m_corrected_records size 0.67445374 MB
[2025-04-03 16:03:55.777] [debug] Read 80000 reads
[2025-04-03 16:03:55.777] [debug] Alignments processed 80001, total m_corrected_records size 0.7263756 MB
[2025-04-03 16:03:56.416] [debug] Read 90000 reads
[2025-04-03 16:03:56.417] [debug] Alignments processed 90001, total m_corrected_records size 0.736496 MB
[2025-04-03 16:03:58.071] [debug] Read 100000 reads
[2025-04-03 16:03:58.073] [debug] Alignments processed 100000, total m_corrected_records size 0.8158493 MB
[2025-04-03 16:03:59.697] [debug] Read 110000 reads
[2025-04-03 16:03:59.700] [debug] Alignments processed 110000, total m_corrected_records size 0.9138222 MB
[2025-04-03 16:04:01.361] [debug] Read 120000 reads
[2025-04-03 16:04:01.364] [debug] Alignments processed 120000, total m_corrected_records size 1.078289 MB
[2025-04-03 16:04:02.943] [debug] Read 130000 reads
[2025-04-03 16:04:02.950] [debug] Alignments processed 130000, total m_corrected_records size 1.155716 MB
[2025-04-03 16:04:04.579] [debug] Read 140000 reads
[2025-04-03 16:04:04.582] [debug] Alignments processed 140000, total m_corrected_records size 1.2683945 MB
[2025-04-03 16:04:06.196] [debug] Read 150000 reads
[2025-04-03 16:04:06.199] [debug] Alignments processed 150000, total m_corrected_records size 1.319519 MB
[2025-04-03 16:04:07.781] [debug] Read 160000 reads
[2025-04-03 16:04:07.783] [debug] Alignments processed 160000, total m_corrected_records size 1.4117165 MB
[2025-04-03 16:04:09.358] [debug] Read 170000 reads
[2025-04-03 16:04:09.360] [debug] Alignments processed 170000, total m_corrected_records size 1.5471802 MB
[2025-04-03 16:04:10.051] [debug] Read 180000 reads
[2025-04-03 16:04:10.051] [debug] Alignments processed 180002, total m_corrected_records size 1.593277 MB
[2025-04-03 16:04:11.317] [debug] Read 190000 reads
[2025-04-03 16:04:11.319] [debug] Alignments processed 190000, total m_corrected_records size 1.8453293 MB
[2025-04-03 16:04:12.869] [debug] Read 200000 reads
[2025-04-03 16:04:12.870] [debug] Alignments processed 200001, total m_corrected_records size 1.9583511 MB
[2025-04-03 16:04:14.371] [debug] Read 210000 reads
[2025-04-03 16:04:14.373] [debug] Alignments processed 210000, total m_corrected_records size 2.0131073 MB
[2025-04-03 16:04:15.982] [debug] Read 220000 reads
[2025-04-03 16:04:15.983] [debug] Alignments processed 220000, total m_corrected_records size 2.1407242 MB
[2025-04-03 16:04:17.542] [debug] Read 230000 reads
[2025-04-03 16:04:17.543] [debug] Alignments processed 230000, total m_corrected_records size 2.2466278 MB
[2025-04-03 16:04:19.101] [debug] Read 240000 reads
[2025-04-03 16:04:19.102] [debug] Alignments processed 240000, total m_corrected_records size 2.3775826 MB
[2025-04-03 16:04:20.611] [debug] Read 250000 reads
[2025-04-03 16:04:20.613] [debug] Alignments processed 250001, total m_corrected_records size 2.4399033 MB
[2025-04-03 16:04:22.040] [debug] Read 260000 reads
[2025-04-03 16:04:22.042] [debug] Alignments processed 260000, total m_corrected_records size 2.5553017 MB
[2025-04-03 16:04:22.431] [debug] Read 270000 reads
[2025-04-03 16:04:22.431] [debug] Alignments processed 270000, total m_corrected_records size 2.5691757 MB
[2025-04-03 16:04:23.817] [debug] Read 280000 reads
[2025-04-03 16:04:23.819] [debug] Alignments processed 280001, total m_corrected_records size 2.6754875 MB
[2025-04-03 16:04:25.412] [debug] Read 290000 reads
[2025-04-03 16:04:25.414] [debug] Alignments processed 290000, total m_corrected_records size 2.7829247 MB
[2025-04-03 16:04:26.978] [debug] Read 300000 reads
[2025-04-03 16:04:26.981] [debug] Alignments processed 300001, total m_corrected_records size 2.9323425 MB
[2025-04-03 16:04:28.547] [debug] Read 310000 reads
[2025-04-03 16:04:28.551] [debug] Alignments processed 310000, total m_corrected_records size 3.0207748 MB
[2025-04-03 16:04:29.623] [debug] Read 320000 reads
[2025-04-03 16:04:29.623] [debug] Alignments processed 320000, total m_corrected_records size 3.0772476 MB
[2025-04-03 16:04:29.844] [debug] Pushing 198 records downstream of mapping.
[2025-04-03 16:04:29.845] [debug] Pushed 198 non-skipped records for correction.
[2025-04-03 16:04:30.005] [info] Finished
Which you can see finished in a few minutes, whereas the dorado-0.9.0-linux-x64/bin/dorado binary running on the same dataset on separate instance is still running after a few hours -- typically this has taken 3-4 hours in similar datasets.
Thanks @iiSeymour for the cuda-11.8 build. This is absolutely essential for those of us using 24GB A30 GPUs for dorado-correct (the official build was using >22GB GPU RAM and managed to utilize only 2-6% of the gpu).
dorado correct runs much more quickly with this dorado 0.9.5 version than the older versions (I'm seeing <50% of the previous runtime). All in all, a great update.
Hey @colindaven, just want to check I've understood correctly.. are you saying cuda-11.8-v0.9.5 is ~twice as fast as previous dorado releases but the cuda-12.8-v0.9.5 is using a lot more memory and performing dreadfully?
Hi @iiSeymour , well, current tests indicate cuda-11.8-v0.9.5 is about 7x as fast for some datasets, but that is likely due to our gpu RAM limit being hit and gpu utilization going down. Issue with details here - https://github.com/nanoporetech/dorado/issues/1297
A current set of test datasets for a 2.5GB genome with 40X coverage took 5-7 days in the past, now these are going through inside 1 day.
I can't test on old and new containers in parallel on this setup, so please take all these with a grain of salt. The key message is I'm very happy to see the cuda-11.8-v0.9.5 version out, since otherwise I'd have to stop using dorado correct since jobs were failing with gpu segfaults - maybe out of RAM errors - (at least with A30 24G RAM GPUs, but I don't have a better alternative at present).
I can say that this is definitely true: "cuda-12.8-v0.9.5 is using a lot more memory and performing dreadfully?".
I'm surprised but happy to see big speedups in cpu and gpu though. Of course part of this can be cpu and storage utilization on a heavily used cluster ....
Many thanks @iiSeymour
I had the same GLIB dependency problem which has been fixed and now working very well with dorado-0.9.5-linux-x64-cuda-11.8.tar.gz
I am using NCI (National Computational Infrastructure) in Australia, which is/was one of the biggest servers in the southern hemisphere. https://nci.org.au/
Hi, all!
I'd been experiencing the same issue as the OP, running Dorado v0.9.5 on a Rocky 8 local server.
I have tested the dorado-0.9.5-linux-x64-cuda-11.8 build and all process seems run smoothly. I'll run a couple experiments with this version to verify it.
So thanks @iiSeymour for the solution!
We have added a Cuda 12.8 Rocky 8 preview build in v0.9.6, and we can look to make this the new default Linux x86 build environment (over Ubuntu 20.04 from v0.9.5) in the next release. Please can you test and provide any feedback..
https://cdn.oxfordnanoportal.com/software/analysis/preview/dorado-0.9.6-linux-x64-cuda-12.8-rocky.tar.gz
Thanks a ton for the cuda-11 version of v0.9.5 and the new update, @iiSeymour !
Sadly, the cuda11-version keeps stalling on arbitrary files, usually the first one of a couple, not particularly big or with long long reads. But it seems that mainly the GPU Quadro RTX 6000 is affected (See the log below).
The containered version of v0.9.5 is also much slower, I assume that the computational resources are not properly recognized by apptainer, or it has some container-induced overhead.
On our local HPC, centOS 7 is running. cuda 12.8 is available. Any idea or chance how to make this work? Also for future releases.
Thanks and all the best! Philipp
[2025-04-22 12:28:11.987] [info] Running: "basecaller" "-v" "--min-qscore" "0" "--sample-sheet" "sampleSheet.csv" "--kit-name" "SQK-PCB114-24" "[email protected]" "pod5/"
[2025-04-22 12:28:12.020] [debug] Found CUDA_VISIBLE_DEVICES=0
[2025-04-22 12:28:12.043] [info] > Creating basecall pipeline
[2025-04-22 12:28:13.435] [debug] TxEncoderStack: use_koi_tiled false.
[2025-04-22 12:28:13.435] [debug] TxEncoderStack: use_koi_volta_tiled false.
[2025-04-22 12:28:15.405] [debug] cuda:0 memory available: 23.39GB
[2025-04-22 12:28:15.405] [debug] cuda:0 memory limit 22.39GB
[2025-04-22 12:28:15.405] [debug] cuda:0 maximum safe estimated batch size at chunk size 12288 is 256
[2025-04-22 12:28:15.405] [debug] cuda:0 maximum safe estimated batch size at chunk size 6144 is 544
[2025-04-22 12:28:15.405] [debug] Auto batchsize cuda:0: testing up to 544 in steps of 32
[2025-04-22 12:28:15.405] [info] Calculating optimized batch size for GPU "Quadro RTX 6000" and model [email protected]. Full benchmarking will run for this device, which may take some time.
[2025-04-22 12:29:38.876] [debug] Auto batchsize cuda:0: 32, time per chunk 1300.263794 ms
[2025-04-22 12:33:15.889] [debug] Auto batchsize cuda:0: 64, time per chunk 1694.296387 ms
[2025-04-22 12:39:29.641] [debug] Auto batchsize cuda:0: 96, time per chunk 1940.637695 ms
[2025-04-22 12:48:06.644] [debug] Auto batchsize cuda:0: 128, time per chunk 2014.291626 ms
[2025-04-22 12:59:22.535] [debug] Auto batchsize cuda:0: 160, time per chunk 2098.929688 ms
[2025-04-22 13:13:10.549] [debug] Auto batchsize cuda:0: 192, time per chunk 2148.178467 ms
[2025-04-22 13:29:20.643] [debug] Auto batchsize cuda:0: 224, time per chunk 2160.156250 ms
[2025-04-22 13:47:56.126] [debug] Auto batchsize cuda:0: 256, time per chunk 2178.087402 ms
[2025-04-22 14:09:48.783] [debug] Auto batchsize cuda:0: 288, time per chunk 2272.747070 ms
[2025-04-22 14:33:19.318] [debug] Auto batchsize cuda:0: 320, time per chunk 2201.056152 ms
[2025-04-22 14:59:12.747] [debug] Auto batchsize cuda:0: 352, time per chunk 2205.819336 ms
[2025-04-22 15:27:23.252] [debug] Auto batchsize cuda:0: 384, time per chunk 2188.702393 ms
[2025-04-22 15:58:08.021] [debug] Auto batchsize cuda:0: 416, time per chunk 2215.648682 ms
[2025-04-22 16:31:05.213] [debug] Auto batchsize cuda:0: 448, time per chunk 2205.892822 ms
[2025-04-22 17:06:10.693] [debug] Auto batchsize cuda:0: 480, time per chunk 2191.490479 ms
[2025-04-22 17:43:46.945] [debug] Auto batchsize cuda:0: 512, time per chunk 2199.140625 ms
[2025-04-22 18:23:44.870] [debug] Auto batchsize cuda:0: 544, time per chunk 2184.651367 ms
[2025-04-22 18:23:44.870] [debug] Adding chunk timings to internal cache for GPU Quadro RTX 6000, model [email protected] (1 entries)
[2025-04-22 18:23:44.870] [debug] Largest batch size for cuda:0: 32, time per chunk 1300.263794 ms
[2025-04-22 18:23:44.870] [debug] Final batch size for cuda:0[0]: 32
[2025-04-22 18:23:44.870] [debug] Final batch size for cuda:0[1]: 32
[2025-04-22 18:23:44.870] [info] cuda:0 using chunk size 12288, batch size 32
[2025-04-22 18:23:44.870] [debug] cuda:0 Model memory 2.28GB
[2025-04-22 18:23:44.870] [debug] cuda:0 Decode memory 0.28GB
[2025-04-22 18:25:36.996] [info] cuda:0 using chunk size 6144, batch size 32
[2025-04-22 18:25:36.996] [debug] cuda:0 Model memory 1.14GB
[2025-04-22 18:25:36.996] [debug] cuda:0 Decode memory 0.14GB
[2025-04-22 18:26:20.283] [debug] BasecallerNode chunk size 12288
[2025-04-22 18:26:20.283] [debug] BasecallerNode chunk size 6144
[2025-04-22 18:26:20.305] [debug] Load reads from file pod5/PBC45370_17505c24_a47ae386_15.pod5
[2025-04-22 18:28:05.118] [debug] > Kits to evaluate: 1```
Hi @phpeters ,
CentOS 7 end of life was June 30th 2024, so I would strongly recommend that you update to a new distribution to continue receiving CVE patches and updates. The CUDA 12.8 Rocky 8 Preview which @iiSeymour linked above requires glibc >= 2.28, so will not work on CentOS 7, as it is on glibc version 2.17.
We chose Rocky 8 and CUDA 12.8 as a candidate build environment because it supports a broad range of non-EOL Linux distributions. We're interested in any feedback on issues with the preview release, as it would be highly preferrable to have a single Linux x86 binary distribution for dorado.
Thanks for all your feedback so far!
Hej @MarkBicknellONT ,
That's a very good point, apologies, CentOS 7's end-of-life wasn't on my radar - our IT is most likely onto it. Thanks for your quick reply!
Philipp
Hi @colindaven,
I can say that this is definitely true: "cuda-12.8-v0.9.5 is using a lot more memory and performing dreadfully?".
We've ran benchmarks of the two, and it does look like the CUDA 12.8 build consumes slightly more GPU memory than the 11.8 one (a few hundred MB in our tests; but I can imagine that in some cases it's just enough to go overboard) and does run slower. It's possible that Torch is using a different code path for CUDA 12 under the hood. We will investigate this further. Thanks for reporting this!
Also, in case you need lower the memory consumption, you can try reducing the number of concurrent inference workers (--infer-threads; which is set to 3 by default), or the batch size (--batch-size).
Resolved: export LD_LIBRARY_PATH fixed this issue.
We have added a Cuda 12.8 Rocky 8 preview build in v0.9.6, and we can look to make this the new default Linux x86 build environment (over Ubuntu 20.04 from v0.9.5) in the next release. Please can you test and provide any feedback..
https://cdn.oxfordnanoportal.com/software/analysis/preview/dorado-0.9.6-linux-x64-cuda-12.8-rocky.tar.gz
@iiSeymour finally had a chance to test this on our Rocky 8 system. Seem like it needs a non-standard library:
./dorado-0.9.6-linux-x64/bin/dorado: error while loading shared libraries: libsz.so.2: cannot open shared object file: No such file or directory
Interestingly, this is only available via the powertools library/repo and the libaec package (Adaptive Entropy Coding Library):
$ dnf provides libsz.so.2
Rocky Linux 8 - AppStream 12 MB/s | 17 MB 00:01
Rocky Linux 8 - BaseOS 21 MB/s | 22 MB 00:01
Rocky Linux 8 - Extras 74 kB/s | 15 kB 00:00
Rocky Linux 8 - PowerTools 7.9 MB/s | 4.1 MB 00:00
Extra Packages for Enterprise Linux 8 - x86_64 6.4 MB/s | 14 MB 00:02
libaec-1.0.2-3.el8.i686 : Adaptive Entropy Coding library
Repo : powertools
Matched from:
Provide : libsz.so.2
So then after running sudo dnf install libaec it seems I can run it. However, on this system our GPU queue actually runs from AWS-batch, so I will have to rebuild our base image to include libaec to get this to work (which will take an IT request...). Is this expected?
Hi @nextgenusfs ,
It does look like the dorado exe is not finding the libsz.so.2 that's shipped in the lib folder for the Rocky archive, which it where we'd expect it to get picked up from. As you mentioned, adding the lib folder to LD_LIBRARY_PATH works around this issue for now:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./dorado-0.9.6-linux-x64/lib/
This dependency comes in from the HDF5 library, and we are actually in the process of removing support for FAST5 from dorado, so this will not be an issue for the next release.
You only need to meet the minimum driver requirement (which 535 does) to run dorado (we have moved to using the 12.8 cuda toolkit for building).
I'll assume compute capability < 9.0. Here's a build https://cdn.oxfordnanoportal.com/software/analysis/dorado-0.9.5-linux-x64-cuda-11.8.tar.gz
I'm getting this issue on dorado-0.9.6 using cuda 11.8.0 - is this resolvable?
Sorry - I have many issues.
The pertinent one in this case are the missing libraries:
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libm.so.6: version GLIBC_2.29' not found (required by ./dorado-0.9.5-linux-x64/bin/dorado) ./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libstdc++.so.6: version GLIBCXX_3.4.26' not found (required by ./dorado-0.9.5-linux-x64/bin/dorado)
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libm.so.6: version GLIBC_2.29' not found (required by /bioinfo/apps/all_apps/dorado-0.9.5-linux-x64/bin/../lib/libdorado_torch_lib.so) ./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libstdc++.so.6: version GLIBCXX_3.4.26' not found (required by /bioinfo/apps/all_apps/dorado-0.9.5-linux-x64/bin/../lib/libdorado_torch_lib.so)
./dorado-0.9.5-linux-x64/bin/dorado: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /bioinfo/apps/all_apps/dorado-0.9.5-linux-x64/bin/../lib/../lib/libdorado_cudnn_lib.so)
(copied from above)
Except I am using the dorado-0.9.6 release. I was able to solve this with the cuda 11.8 version you provided for dorado-0.9.5 but the 0.9.6 release brings back the issue.
Hope that makes sense...
[edited due to typing incompetence]
Dorado v0.9.5 (cuda 11.8) was built in CentOS 7 so requires glibc >= 2.17 for v0.9.6 we released a preview build on Rocky 8 which is glibc >= 2.28 but the mainline build is Ubuntu 20.04 so glibc >= 2.31.
What OS are you running on @mattloose?
NAME="Red Hat Enterprise Linux" VERSION="8.6 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.6" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
Okay so RHEL 8.6 has glibc 2.28 so this preview build of v0.9.6 should work https://cdn.oxfordnanoportal.com/software/analysis/preview/dorado-0.9.6-linux-x64-cuda-12.8-rocky.tar.gz
Let us know if that's all good as we plan to make this the default in the next release.
Sorry for the delay in testing:
./dorado --help
./dorado: error while loading shared libraries: libsz.so.2: cannot open shared object file: No such file or directory
Hi @mattloose ,
There was an issue with the Rocky preview release, where on some RHEL 8 systems this error would come up. It can be worked around by adding the lib folder to the ldd search path like so:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./dorado-0.9.6-linux-x64/lib/
Another +1 for this frustrating library issue @iiSeymour - same with me on RHEL 8.2 on our HPC environment. I have downloaded the preview build and tried, will just roll back to our old version and try compiling from source as well
