Medaka not detecting installed tools

[BMTTMC]

Describe the bug When attempting to run medaka_consensus within a script, it is failing due to a failure to detect installed tools (i.e. samtools, bcftools, etc.). minimap2 is found, but others are not (see the logging section).
Interestingly, samtools is used on its own from within the script without any issue. This issue is focusing on samtools, but the issue involves the other tools listed as well. Any help would be appreciated.

Logging When trying to run medaka_consensus (note that this behavior is consistent whether or not I specify a model):

$ medaka_consensus  -i ../testing_pipeline/output/IMP9/BRCA1/BRCA1_exon16_41242915_244_110_189/d2e6f6e1-bf1d-47cf-b1db-a4887ee99025.fastq -d ../testing_pipeline/output/IMP9/BRCA1/BRCA1_exon16_41242915_244_110_189/BRCA1_exon16_41242915_244_110_189_reference.fasta -o ../testing_pipeline/output/IMP9/BRCA1/BRCA1_exon16_41242915_244_110_189/d2e6f6e1-bf1d-47cf-b1db-a4887ee99025 -q
Attempting to automatically select model version.
WARNING: Failed to detect a model version, will use default: 'r1041_e82_400bps_sup_v5.0.0'
Checking program versions
This is medaka 2.0.0
Program    Version    Required   Pass     
bcftools   Not found  1.11       False    
bgzip      Not found  1.11       False    
minimap2   2.24       2.11       True     
samtools   Not found  1.11       False    
tabix      Not found  1.11       False   

To confirm that samtools (for example) is installed:

$ samtools --version
samtools 1.13

$ which samtools
/usr/bin/samtools

I've also confirmed that the PATH includes samtools:

$ echo $PATH
/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ubuntu/.local/bin:/home/ubuntu/.local/bin:/usr/bin/samtools

Environment:

• Installation method: pypi
• OS: Ubuntu 22.04.5
• medaka version: 2.0.0
• AWS EC2 m3.2xlarge instance

[cjw85]

The code that performs these checks is here. You'll see it doesn't do much other than run the commands. I'm afraid I can't suggest a reason why these checks are failing.

[BMTTMC]

That's true, but it's called in report_binaries(), which will throw an error if anything is missing, preventing the script from running.

[BMTTMC]

Update: By installing all of the tools using apt (as opposed to manually installing them), they are now all detected except for samtools. I have uninstalled and reinstalled samtools, but to no avail.

[vineeth-s]

@BMTTMC if you have not solved this yet, this is an encoding issue I stepped through the code and the decoding of what subprocessing returns is the issue for samtools

(samtools version 1.13 the issue seems to be in the Samtools compilation details section where the CFLAGS line is CFLAGS: -g -O2 -ffile-prefix-map=�BUILDPATH�=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security The culprits are the 2 �)

So in my medaka installation, I had to edit lib/python3.10/site-packages/medaka/__init__.py and add decoding parameters to the line You have to change first_line = proc.stdout.decode().split("\n", 1)[0] to first_line = proc.stdout.decode('utf-8',errors='replace').split("\n", 1)[0]

and then I am able to run medaka just fine

[zwets]

Confirming the fix by @vineeth-s for distro-provided samtools on Ubuntu 24.04. proc.stdout.decode() chokes on the tofu in the output of samtools --version.

[Agridibuu]

If my data was basecalled with Dorado version 0.9.1 with super accurate (sup) model and Flowcell: FLO-MIN114 (R10.4.1) was used so what code should I use to run medaka. or I can no longer use it in 2025 ? my genome is 40mb and its a fungus.

[rickbeeloo]

Yup same bug here, samtools installed but not detected

[rickbeeloo]

A hacky way to "pretend" the version is fine:

nano ~/mybin/samtools

We create a script to return samtools 1.11 if a command is samtools version but without version we call the original executable:

nano ~/mybin/samtools

and paste the following:

#!/bin/bash

# If any argument contains "version", return fake version
for arg in "$@"; do
    if [[ "$arg" == *version* ]]; then
        echo "samtools 1.11"
        exit 0
    fi
done

# Call normal samtools (MATCH YOURs, see whereis samtools)
/usr/bin/samtools "$@"

make executable

chmod +x ~/mybin/samtools

Replace samtools in Path by this patch:

export PATH=~/mybin:$PATH

Now running samtools --version should give you samtools 1.11, and just running samtools should call your original executable

(this is temporary, probably you don't want to add it to ~/.bashrc as it might mess up your other tools)