bonito icon indicating copy to clipboard operation
bonito copied to clipboard

bonito model downloading error

Open madhubioinfo opened this issue 2 years ago • 16 comments

I have used the following command for models bonito download --models --list

I am having this error of usage: bonito [-h] [-v] {basecaller,evaluate,train,view,tune} ... bonito: error: argument command: invalid choice: 'download' (choose from 'basecaller', 'evaluate', 'train', 'view', 'tune')

Then I used https://github.com/nanoporetech/bonito/issues/69 tips to download the model and try to run bonito like this

bonito basecaller /home/minion/bonito_models/[email protected] /mnt/nas2/raw_sequence_data/nanopore/220124_MC110090/fast5_pass/barcode02 > test_bonito_basecalled.fastq

I am having the following error message

loading model Traceback (most recent call last): File "/home/minion/miniconda3/bin/bonito", line 8, in sys.exit(main()) File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/init.py", line 31, in main args.func(args) File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/basecaller.py", line 19, in main model = load_model(args.model_directory, args.device, weights=int(args.weights), half=args.half) File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/util.py", line 185, in load_model model = Model(toml.load(config)) File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/model.py", line 25, in init self.stride = config['block'][0]['stride'][0] KeyError: 'block'

can you please tell me what is wrong in here ? is it the version of installed bonito or models?

Thanks in advance

madhubioinfo avatar Jun 09 '22 17:06 madhubioinfo

@madhubioinfo which version of bonito do you have installed (you can check with bonito -v)? It looks like you might have an old version. Are you able to update to the latest version?

iiSeymour avatar Jun 10 '22 10:06 iiSeymour

Its is bonito 0.0.7 and i did with the pip installation of this. where can i find the recent one? how to upgrade?

madhubioinfo avatar Jun 10 '22 15:06 madhubioinfo

All versions can be installed with pip and you can upgrade with:

$ pip install --upgrade ont-bonito

iiSeymour avatar Jun 10 '22 15:06 iiSeymour

when i do pip install --upgrade ont-bonito

Requirement already satisfied: ont-bonito in ./miniconda3/lib/python3.8/site-packages (0.0.7)
Collecting ont-bonito
  Using cached ont-bonito-0.5.3.tar.gz (50 kB)
  Preparing metadata (setup.py) ... done
Collecting mappy==2.24
  Using cached mappy-2.24.tar.gz (140 kB)
  Preparing metadata (setup.py) ... done
Collecting toml==0.10.2
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: tqdm<5,>4 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (4.31.1)
Collecting scipy==1.7.3
  Using cached scipy-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.3 MB)
Collecting numpy~=1.21.5
  Using cached numpy-1.21.6-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Requirement already satisfied: pysam==0.18.0 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.18.0)
Requirement already satisfied: parasail==1.2 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.2)
Collecting pandas<2,>1
  Using cached pandas-1.4.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
Collecting requests==2.25.1
  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting ont-koi==0.0.9
  Using cached ont_koi-0.0.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (561 kB)
Collecting onnxruntime==1.11.0
  Using cached onnxruntime-1.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB)
Collecting ont-remora==1.1.0
  Using cached ont_remora-1.1.0-cp38-cp38-linux_x86_64.whl
Collecting ont-fast5-api==3.3.0
  Using cached ont_fast5_api-3.3.0-py3-none-any.whl (2.0 MB)
Collecting pod5-format==0.0.14
  Using cached pod5_format-0.0.14-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)
Collecting pod5-format-tools==0.0.14
  Using cached pod5_format_tools-0.0.14-py3-none-any.whl (8.3 kB)
Collecting fast-ctc-decode==0.3.2
  Using cached fast_ctc_decode-0.3.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (517 kB)
Collecting python-dateutil==2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting torch~=1.11
  Using cached torch-1.11.0-cp38-cp38-manylinux1_x86_64.whl (750.6 MB)
Collecting flatbuffers
  Using cached flatbuffers-2.0-py2.py3-none-any.whl (26 kB)
Collecting protobuf
  Using cached protobuf-4.21.1-cp37-abi3-manylinux2014_x86_64.whl (407 kB)
Requirement already satisfied: progressbar33>=2.3.1 in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (2.4)
Requirement already satisfied: h5py>=2.8 in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (3.7.0)
Collecting packaging
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Requirement already satisfied: cffi in ./miniconda3/lib/python3.8/site-packages (from ont-koi==0.0.9->ont-bonito) (1.14.3)
Collecting thop
  Downloading thop-0.1.0.post2206102148-py3-none-any.whl (15 kB)
Collecting scikit-learn
  Using cached scikit_learn-1.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (31.2 MB)
Collecting onnx
  Using cached onnx-1.11.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.8 MB)
Requirement already satisfied: tabulate in ./miniconda3/lib/python3.8/site-packages (from ont-remora==1.1.0->ont-bonito) (0.8.9)
Requirement already satisfied: wheel in ./miniconda3/lib/python3.8/site-packages (from parasail==1.2->ont-bonito) (0.35.1)
Collecting more-itertools
  Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting iso8601
  Using cached iso8601-1.0.2-py3-none-any.whl (9.7 kB)
Requirement already satisfied: pytz in ./miniconda3/lib/python3.8/site-packages (from pod5-format==0.0.14->ont-bonito) (2022.1)
Collecting pyarrow~=7.0.0
  Using cached pyarrow-7.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.7 MB)
Requirement already satisfied: six>=1.5 in ./miniconda3/lib/python3.8/site-packages (from python-dateutil==2.8.2->ont-bonito) (1.15.0)
Requirement already satisfied: idna<3,>=2.5 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (3.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2021.10.8)
Collecting typing-extensions
  Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Requirement already satisfied: pycparser in ./miniconda3/lib/python3.8/site-packages (from cffi->ont-koi==0.0.9->ont-bonito) (2.20)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./miniconda3/lib/python3.8/site-packages (from packaging->ont-fast5-api==3.3.0->ont-bonito) (3.0.9)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: joblib>=1.0.0 in ./miniconda3/lib/python3.8/site-packages (from scikit-learn->ont-remora==1.1.0->ont-bonito) (1.1.0)
Building wheels for collected packages: ont-bonito, mappy
  Building wheel for ont-bonito (setup.py) ... error
  error: subprocess-exited-with-error

[11:38 a.m.] Noor Shubair
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [103 lines of output]
      Building with CUDA 10.2
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib
      creating build/lib/bonito
      copying bonito/schedule.py -> build/lib/bonito
      copying bonito/nn.py -> build/lib/bonito
      copying bonito/reader.py -> build/lib/bonito
      copying bonito/data.py -> build/lib/bonito
      copying bonito/io.py -> build/lib/bonito
      copying bonito/pod5.py -> build/lib/bonito
      copying bonito/util.py -> build/lib/bonito
      copying bonito/mod_util.py -> build/lib/bonito
      copying bonito/fast5.py -> build/lib/bonito
      copying bonito/training.py -> build/lib/bonito
      copying bonito/multiprocessing.py -> build/lib/bonito
      copying bonito/__init__.py -> build/lib/bonito
      copying bonito/aligner.py -> build/lib/bonito
      creating build/lib/bonito/cli
      copying bonito/cli/evaluate.py -> build/lib/bonito/cli
      copying bonito/cli/convert.py -> build/lib/bonito/cli
      copying bonito/cli/download.py -> build/lib/bonito/cli
      copying bonito/cli/export.py -> build/lib/bonito/cli
      copying bonito/cli/train.py -> build/lib/bonito/cli
      copying bonito/cli/view.py -> build/lib/bonito/cli
      copying bonito/cli/basecaller.py -> build/lib/bonito/cli
      copying bonito/cli/__init__.py -> build/lib/bonito/cli
      copying bonito/cli/duplex.py -> build/lib/bonito/cli
      creating build/lib/bonito/ctc
      copying bonito/ctc/__init__.py -> build/lib/bonito/ctc
      copying bonito/ctc/model.py -> build/lib/bonito/ctc
      copying bonito/ctc/basecall.py -> build/lib/bonito/ctc
      creating build/lib/bonito/crf
      copying bonito/crf/__init__.py -> build/lib/bonito/crf
      copying bonito/crf/model.py -> build/lib/bonito/crf
      copying bonito/crf/basecall.py -> build/lib/bonito/crf
      running egg_info
      writing ont_bonito.egg-info/PKG-INFO
      writing entry points to ont_bonito.egg-info/entry_points.txt
      writing requirements to ont_bonito.egg-info/requires.txt
      writing top-level names to ont_bonito.egg-info/top_level.txt
      reading manifest file 'ont_bonito.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      writing manifest file 'ont_bonito.egg-info/SOURCES.txt'
      creating build/lib/bonito/data
      copying bonito/data/README.md -> build/lib/bonito/data
      creating build/lib/bonito/models
      copying bonito/models/README.md -> build/lib/bonito/models
      creating build/lib/bonito/models/configs
      copying bonito/models/configs/[email protected] -> build/lib/bonito/models/configs
      copying bonito/models/configs/[email protected] -> build/lib/bonito/models/configs
      copying bonito/models/configs/[email protected] -> build/lib/bonito/models/configs
      installing to build/bdist.linux-x86_64/wheel
      running install
      running install_lib
      creating build/bdist.linux-x86_64
      creating build/bdist.linux-x86_64/wheel
      creating build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/evaluate.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/convert.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/download.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/export.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/train.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/view.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/basecaller.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/__init__.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/duplex.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/schedule.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/ctc/__init__.py -> build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/ctc/model.py -> build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/ctc/basecall.py -> build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/nn.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/reader.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/data.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/io.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/crf/__init__.py -> build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/crf/model.py -> build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/crf/basecall.py -> build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/pod5.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/models
      copying build/lib/bonito/models/README.md -> build/bdist.linux-x86_64/wheel/bonito/models
      creating build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/models/configs/[email protected] -> build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/models/configs/[email protected] -> build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/models/configs/[email protected] -> build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/util.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/data
      copying build/lib/bonito/data/README.md -> build/bdist.linux-x86_64/wheel/bonito/data
      copying build/lib/bonito/mod_util.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/fast5.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/training.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/multiprocessing.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/__init__.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/aligner.py -> build/bdist.linux-x86_64/wheel/bonito
      running install_egg_info
      Copying ont_bonito.egg-info to build/bdist.linux-x86_64/wheel/ont_bonito-0.5.3-py3.8.egg-info
      running install_scripts
      error: [Errno 2] No such file or directory: 'build/bdist.linux-x86_64/wheel/ont_bonito-0.5.3.dist-info/dependency_links.txt'
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ont-bonito
  Running setup.py clean for ont-bonito
  Building wheel for mappy (setup.py) ... error
  error: subprocess-exited-with-error
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      running bdist_wheel
      running build
      running build_ext
      cythoning python/mappy.pyx to python/mappy.c
      /tmp/pip-install-vobu08zl/mappy_70c81feb799747f696549b53a8f24f70/.eggs/Cython-0.29.30-py3.8-linux-x86_64.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-vobu08zl/mappy_70c81feb799747f696549b53a8f24f70/python/mappy.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)
      building 'mappy' extension
      creating build
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/python
      gcc -pthread -B /home/minion/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/home/minion/miniconda3/include/python3.8 -c python/mappy.c -o build/temp.linux-x86_64-3.8/python/mappy.o -DHAVE_KALLOC -msse4.1
      In file included from python/mappy.c:700:0:
      python/cmappy.h:6:18: fatal error: zlib.h: No such file or directory
      compilation terminated.
      error: command 'gcc' failed with exit status 1
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mappy
  Running setup.py clean for mappy
Failed to build ont-bonito mappy
Installing collected packages: mappy, flatbuffers, fast-ctc-decode, typing-extensions, toml, threadpoolctl, requests, python-dateutil, protobuf, packaging, numpy, more-itertools, iso8601, torch, scipy, pyarrow, pandas, ont-koi, onnxruntime, onnx, thop, scikit-learn, pod5-format, ont-fast5-api, pod5-format-tools, ont-remora, ont-bonito
  Running setup.py install for mappy ... error
  error: subprocess-exited-with-error
  × Running setup.py install for mappy did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      running install
      running build
      running build_ext
      skipping 'python/mappy.c' Cython extension (up-to-date)
      building 'mappy' extension
      creating build
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/python
      gcc -pthread -B /home/minion/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/home/minion/miniconda3/include/python3.8 -c python/mappy.c -o build/temp.linux-x86_64-3.8/python/mappy.o -DHAVE_KALLOC -msse4.1
      In file included from python/mappy.c:700:0:
      python/cmappy.h:6:18: fatal error: zlib.h: No such file or directory
      compilation terminated.
      error: command 'gcc' failed with exit status 1
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure× Encountered error while trying to install package.
╰─> mappynote: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

madhubioinfo avatar Jun 10 '22 15:06 madhubioinfo

Can you try upgrading pip first -

 $ pip install --upgrade pip
 $ pip install --upgrade ont-bonito

iiSeymour avatar Jun 10 '22 15:06 iiSeymour

when i do pip install --upgrade pip it says requirement already satisfied. and when i try to upgrade the bonito installation this is what i'm receiving

Collecting ont-bonito
  Using cached ont-bonito-0.5.3.tar.gz (50 kB)
  Preparing metadata (setup.py) ... done
Collecting mappy==2.24
  Using cached mappy-2.24.tar.gz (140 kB)
  Preparing metadata (setup.py) ... done
Collecting toml==0.10.2
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: tqdm<5,>4 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (4.31.1)
Collecting scipy==1.7.3
  Using cached scipy-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.3 MB)
Collecting numpy~=1.21.5
  Using cached numpy-1.21.6-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Requirement already satisfied: pysam==0.18.0 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.18.0)
Requirement already satisfied: parasail==1.2 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.2)
Collecting pandas<2,>1
  Using cached pandas-1.4.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
Collecting requests==2.25.1
  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting ont-koi==0.0.9
  Using cached ont_koi-0.0.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (561 kB)
Collecting onnxruntime==1.11.0
  Using cached onnxruntime-1.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB)
Collecting ont-remora==1.1.0
  Using cached ont_remora-1.1.0-cp38-cp38-linux_x86_64.whl
Collecting ont-fast5-api==3.3.0
  Using cached ont_fast5_api-3.3.0-py3-none-any.whl (2.0 MB)
Collecting pod5-format==0.0.14
  Using cached pod5_format-0.0.14-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)
Collecting pod5-format-tools==0.0.14
  Using cached pod5_format_tools-0.0.14-py3-none-any.whl (8.3 kB)
Collecting fast-ctc-decode==0.3.2
  Using cached fast_ctc_decode-0.3.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (517 kB)
Collecting python-dateutil==2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting torch~=1.11
  Using cached torch-1.11.0-cp38-cp38-manylinux1_x86_64.whl (750.6 MB)
Collecting flatbuffers
  Using cached flatbuffers-2.0-py2.py3-none-any.whl (26 kB)
Collecting protobuf
  Using cached protobuf-4.21.1-cp37-abi3-manylinux2014_x86_64.whl (407 kB)
Requirement already satisfied: h5py>=2.8 in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (3.7.0)
Requirement already satisfied: progressbar33>=2.3.1 in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (2.4)
Collecting packaging
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Requirement already satisfied: cffi in ./miniconda3/lib/python3.8/site-packages (from ont-koi==0.0.9->ont-bonito) (1.14.3)
Collecting thop
  Using cached thop-0.1.0.post2206102148-py3-none-any.whl (15 kB)
Collecting onnx
  Using cached onnx-1.11.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.8 MB)
Requirement already satisfied: tabulate in ./miniconda3/lib/python3.8/site-packages (from ont-remora==1.1.0->ont-bonito) (0.8.9)
Collecting scikit-learn
  Using cached scikit_learn-1.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (31.2 MB)
Requirement already satisfied: wheel in ./miniconda3/lib/python3.8/site-packages (from parasail==1.2->ont-bonito) (0.35.1)
Collecting iso8601
  Using cached iso8601-1.0.2-py3-none-any.whl (9.7 kB)
Requirement already satisfied: pytz in ./miniconda3/lib/python3.8/site-packages (from pod5-format==0.0.14->ont-bonito) (2022.1)
Collecting more-itertools
  Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting pyarrow~=7.0.0
  Using cached pyarrow-7.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.7 MB)
Requirement already satisfied: six>=1.5 in ./miniconda3/lib/python3.8/site-packages (from python-dateutil==2.8.2->ont-bonito) (1.15.0)
Requirement already satisfied: chardet<5,>=3.0.2 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (3.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2021.10.8)
Requirement already satisfied: idna<3,>=2.5 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2.10)
Collecting typing-extensions
  Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Requirement already satisfied: pycparser in ./miniconda3/lib/python3.8/site-packages (from cffi->ont-koi==0.0.9->ont-bonito) (2.20)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./miniconda3/lib/python3.8/site-packages (from packaging->ont-fast5-api==3.3.0->ont-bonito) (3.0.9)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: joblib>=1.0.0 in ./miniconda3/lib/python3.8/site-packages (from scikit-learn->ont-remora==1.1.0->ont-bonito) (1.1.0)
Building wheels for collected packages: ont-bonito, mappy
  Building wheel for ont-bonito (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [103 lines of output]
      Building with CUDA 10.2
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib
      creating build/lib/bonito
      copying bonito/schedule.py -> build/lib/bonito
      copying bonito/nn.py -> build/lib/bonito
      copying bonito/reader.py -> build/lib/bonito
      copying bonito/data.py -> build/lib/bonito
      copying bonito/io.py -> build/lib/bonito
      copying bonito/pod5.py -> build/lib/bonito
      copying bonito/util.py -> build/lib/bonito
      copying bonito/mod_util.py -> build/lib/bonito
      copying bonito/fast5.py -> build/lib/bonito
      copying bonito/training.py -> build/lib/bonito
      copying bonito/multiprocessing.py -> build/lib/bonito
      copying bonito/__init__.py -> build/lib/bonito
      copying bonito/aligner.py -> build/lib/bonito
      creating build/lib/bonito/cli
      copying bonito/cli/evaluate.py -> build/lib/bonito/cli
      copying bonito/cli/convert.py -> build/lib/bonito/cli
      copying bonito/cli/download.py -> build/lib/bonito/cli
      copying bonito/cli/export.py -> build/lib/bonito/cli
      copying bonito/cli/train.py -> build/lib/bonito/cli
      copying bonito/cli/view.py -> build/lib/bonito/cli
      copying bonito/cli/basecaller.py -> build/lib/bonito/cli
      copying bonito/cli/__init__.py -> build/lib/bonito/cli
      copying bonito/cli/duplex.py -> build/lib/bonito/cli
      creating build/lib/bonito/ctc
      copying bonito/ctc/__init__.py -> build/lib/bonito/ctc
      copying bonito/ctc/model.py -> build/lib/bonito/ctc
      copying bonito/ctc/basecall.py -> build/lib/bonito/ctc
      creating build/lib/bonito/crf
      copying bonito/crf/__init__.py -> build/lib/bonito/crf
      copying bonito/crf/model.py -> build/lib/bonito/crf
      copying bonito/crf/basecall.py -> build/lib/bonito/crf
      running egg_info
      writing ont_bonito.egg-info/PKG-INFO
      writing entry points to ont_bonito.egg-info/entry_points.txt
      writing requirements to ont_bonito.egg-info/requires.txt
      writing top-level names to ont_bonito.egg-info/top_level.txt
      reading manifest file 'ont_bonito.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      writing manifest file 'ont_bonito.egg-info/SOURCES.txt'
      creating build/lib/bonito/data
      copying bonito/data/README.md -> build/lib/bonito/data
      creating build/lib/bonito/models
      copying bonito/models/README.md -> build/lib/bonito/models
      creating build/lib/bonito/models/configs
      copying bonito/models/configs/[email protected] -> build/lib/bonito/models/configs
      copying bonito/models/configs/[email protected] -> build/lib/bonito/models/configs
      copying bonito/models/configs/[email protected] -> build/lib/bonito/models/configs
      installing to build/bdist.linux-x86_64/wheel
      running install
      running install_lib
      creating build/bdist.linux-x86_64
      creating build/bdist.linux-x86_64/wheel
      creating build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/evaluate.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/convert.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/download.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/export.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/train.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/view.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/basecaller.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/__init__.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/cli/duplex.py -> build/bdist.linux-x86_64/wheel/bonito/cli
      copying build/lib/bonito/schedule.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/ctc/__init__.py -> build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/ctc/model.py -> build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/ctc/basecall.py -> build/bdist.linux-x86_64/wheel/bonito/ctc
      copying build/lib/bonito/nn.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/reader.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/data.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/io.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/crf/__init__.py -> build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/crf/model.py -> build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/crf/basecall.py -> build/bdist.linux-x86_64/wheel/bonito/crf
      copying build/lib/bonito/pod5.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/models
      copying build/lib/bonito/models/README.md -> build/bdist.linux-x86_64/wheel/bonito/models
      creating build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/models/configs/[email protected] -> build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/models/configs/[email protected] -> build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/models/configs/[email protected] -> build/bdist.linux-x86_64/wheel/bonito/models/configs
      copying build/lib/bonito/util.py -> build/bdist.linux-x86_64/wheel/bonito
      creating build/bdist.linux-x86_64/wheel/bonito/data
      copying build/lib/bonito/data/README.md -> build/bdist.linux-x86_64/wheel/bonito/data
      copying build/lib/bonito/mod_util.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/fast5.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/training.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/multiprocessing.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/__init__.py -> build/bdist.linux-x86_64/wheel/bonito
      copying build/lib/bonito/aligner.py -> build/bdist.linux-x86_64/wheel/bonito
      running install_egg_info
      Copying ont_bonito.egg-info to build/bdist.linux-x86_64/wheel/ont_bonito-0.5.3-py3.8.egg-info
      running install_scripts
      error: [Errno 2] No such file or directory: 'build/bdist.linux-x86_64/wheel/ont_bonito-0.5.3.dist-info/dependency_links.txt'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ont-bonito
  Running setup.py clean for ont-bonito
  Building wheel for mappy (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      running bdist_wheel
      running build
      running build_ext
      cythoning python/mappy.pyx to python/mappy.c
      /tmp/pip-install-neaaq_wj/mappy_03b582389bb54e22a675185d46374ffd/.eggs/Cython-0.29.30-py3.8-linux-x86_64.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-neaaq_wj/mappy_03b582389bb54e22a675185d46374ffd/python/mappy.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)
      building 'mappy' extension
      creating build
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/python
      gcc -pthread -B /home/minion/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/home/minion/miniconda3/include/python3.8 -c python/mappy.c -o build/temp.linux-x86_64-3.8/python/mappy.o -DHAVE_KALLOC -msse4.1
      In file included from python/mappy.c:700:0:
      python/cmappy.h:6:18: fatal error: zlib.h: No such file or directory
      compilation terminated.
      error: command 'gcc' failed with exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mappy
  Running setup.py clean for mappy
Failed to build ont-bonito mappy
Installing collected packages: mappy, flatbuffers, fast-ctc-decode, typing-extensions, toml, threadpoolctl, requests, python-dateutil, protobuf, packaging, numpy, more-itertools, iso8601, torch, scipy, pyarrow, pandas, ont-koi, onnxruntime, onnx, thop, scikit-learn, pod5-format, ont-fast5-api, pod5-format-tools, ont-remora, ont-bonito
  Running setup.py install for mappy ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for mappy did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      running install
      running build
      running build_ext
      skipping 'python/mappy.c' Cython extension (up-to-date)
      building 'mappy' extension
      creating build
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/python
      gcc -pthread -B /home/minion/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/home/minion/miniconda3/include/python3.8 -c python/mappy.c -o build/temp.linux-x86_64-3.8/python/mappy.o -DHAVE_KALLOC -msse4.1
      In file included from python/mappy.c:700:0:
      python/cmappy.h:6:18: fatal error: zlib.h: No such file or directory
      compilation terminated.
      error: command 'gcc' failed with exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> mappy

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

madhubioinfo avatar Jun 14 '22 15:06 madhubioinfo

I see the issue - mappy doesn't provide precompiled wheels https://github.com/lh3/minimap2/issues/449. It's compiled on your system at install time and requires the zlib headers.

python/cmappy.h:6:18: fatal error: zlib.h: No such file or directory

See https://stackoverflow.com/questions/36374267/how-to-fix-fatal-error-zlib-h-no-such-file-or-directory

iiSeymour avatar Jun 14 '22 15:06 iiSeymour

I was able to install zlib and i tried installing after that. I am getting different error

error: [Errno 2] No such file or directory: 'build/bdist.linux-x86_64/wheel/ont_bonito-0.5.3.dist-info/dependency_links.txt'
     [end of output]
 
 note: This error originates from a subprocess, and is likely not a problem with pip.
 ERROR: Failed building wheel for ont-bonito
 Running setup.py clean for ont-bonito
 Building wheel for mappy (setup.py) ... done
 Created wheel for mappy: filename=mappy-2.24-cp38-cp38-linux_x86_64.whl size=543730 sha256=635e017fa69e3959fb27e88a201e53be043b05aba0984876ce4a5c9f60642d79
 Stored in directory: /home/minion/.cache/pip/wheels/df/ca/6a/7be4760503025a76f7e52b3487f622e2698ae87598f8d8e0c5
Successfully built mappy
Failed to build ont-bonito
Installing collected packages: mappy, flatbuffers, fast-ctc-decode, typing-extensions, toml, threadpoolctl, requests, python-dateutil, protobuf, packaging, numpy, more-itertools, iso8601, torch, scipy, pyarrow, pandas, ont-koi, onnxruntime, onnx, thop, scikit-learn, pod5-format, ont-fast5-api, pod5-format-tools, ont-remora, ont-bonito
 Attempting uninstall: fast-ctc-decode
   Found existing installation: fast_ctc_decode 0.1.3
   Uninstalling fast_ctc_decode-0.1.3:
     Successfully uninstalled fast_ctc_decode-0.1.3
 Attempting uninstall: toml
   Found existing installation: toml 0.10.0
   Uninstalling toml-0.10.0:
     Successfully uninstalled toml-0.10.0
 Attempting uninstall: requests
   Found existing installation: requests 2.24.0
   Uninstalling requests-2.24.0:
     Successfully uninstalled requests-2.24.0
 Attempting uninstall: numpy
   Found existing installation: numpy 1.22.4
   Uninstalling numpy-1.22.4:
     Successfully uninstalled numpy-1.22.4
 Attempting uninstall: torch
   Found existing installation: torch 1.4.0
   Uninstalling torch-1.4.0:
     Successfully uninstalled torch-1.4.0
 Attempting uninstall: scipy
   Found existing installation: scipy 1.8.1
   Uninstalling scipy-1.8.1:
     Successfully uninstalled scipy-1.8.1
 Attempting uninstall: ont-fast5-api
   Found existing installation: ont-fast5-api 3.0.1
   Uninstalling ont-fast5-api-3.0.1:
     Successfully uninstalled ont-fast5-api-3.0.1
 Attempting uninstall: ont-bonito
   Found existing installation: ont-bonito 0.0.7
   Uninstalling ont-bonito-0.0.7:
     Successfully uninstalled ont-bonito-0.0.7
 Running setup.py install for ont-bonito ... done
 DEPRECATION: ont-bonito was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. Discussion can be found at https://github.com/pypa/pip/issues/8368
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
conda 4.12.0 requires ruamel_yaml_conda>=0.11.14, which is not installed.
Successfully installed fast-ctc-decode-0.3.2 flatbuffers-2.0 iso8601-1.0.2 mappy-2.24 more-itertools-8.13.0 numpy-1.21.6 onnx-1.11.0 onnxruntime-1.11.0 ont-bonito-0.5.3 ont-fast5-api-3.3.0 ont-koi-0.0.9 ont-remora-1.1.0 packaging-21.3 pandas-1.4.2 pod5-format-0.0.14 pod5-format-tools-0.0.14 protobuf-4.21.1 pyarrow-7.0.0 python-dateutil-2.8.2 requests-2.25.1 scikit-learn-1.1.1 scipy-1.7.3 thop-0.1.0.post2206102148 threadpoolctl-3.1.0 toml-0.10.2 torch-1.11.0 typing-extensions-4.2.0

madhubioinfo avatar Jun 14 '22 16:06 madhubioinfo

@madhubioinfo it's not clear to me this failed. Can you try bonito -v now?

iiSeymour avatar Jun 14 '22 16:06 iiSeymour

when i do check the version this is what i'm getting

$ bonito -v
Traceback (most recent call last):
  File "/home/minion/miniconda3/bin/bonito", line 33, in <module>
    sys.exit(load_entry_point('ont-bonito==0.5.3', 'console_scripts', 'bonito')())
  File "/home/minion/miniconda3/bin/bonito", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/home/minion/miniconda3/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/home/minion/miniconda3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/__init__.py", line 2, in <module>
    from bonito.cli import basecaller, train, evaluate, view, convert, download, export
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 18, in <module>
    from bonito.mod_util import call_mods, load_mods_model
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/mod_util.py", line 6, in <module>
    from remora.model_util import load_model
  File "/home/minion/miniconda3/lib/python3.8/site-packages/remora/model_util.py", line 11, in <module>
    import onnx
  File "/home/minion/miniconda3/lib/python3.8/site-packages/onnx/__init__.py", line 11, in <module>
    from onnx.external_data_helper import load_external_data_for_model, write_external_data_tensors, convert_model_to_external_data
  File "/home/minion/miniconda3/lib/python3.8/site-packages/onnx/external_data_helper.py", line 14, in <module>
    from .onnx_pb import TensorProto, ModelProto
  File "/home/minion/miniconda3/lib/python3.8/site-packages/onnx/onnx_pb.py", line 8, in <module>
    from .onnx_ml_pb2 import *  # noqa
  File "/home/minion/miniconda3/lib/python3.8/site-packages/onnx/onnx_ml_pb2.py", line 33, in <module>
    _descriptor.EnumValueDescriptor(
  File "/home/minion/miniconda3/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

when i do pip install --upgrade ont-bonito
I am getting following error
Requirement already satisfied: ont-bonito in ./miniconda3/lib/python3.8/site-packages (0.5.3)
Requirement already satisfied: mappy==2.24 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (2.24)
Requirement already satisfied: toml==0.10.2 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.10.2)
Requirement already satisfied: tqdm<5,>4 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (4.31.1)
Requirement already satisfied: scipy==1.7.3 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.7.3)
Requirement already satisfied: numpy~=1.21.5 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.21.6)
Requirement already satisfied: pysam==0.18.0 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.18.0)
Requirement already satisfied: parasail==1.2 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.2)
Requirement already satisfied: pandas<2,>1 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.4.2)
Requirement already satisfied: requests==2.25.1 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (2.25.1)
Requirement already satisfied: ont-koi==0.0.9 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.0.9)
Requirement already satisfied: onnxruntime==1.11.0 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.11.0)
Requirement already satisfied: ont-remora==1.1.0 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.1.0)
Requirement already satisfied: ont-fast5-api==3.3.0 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (3.3.0)
Requirement already satisfied: pod5-format==0.0.14 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.0.14)
Requirement already satisfied: pod5-format-tools==0.0.14 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.0.14)
Requirement already satisfied: fast-ctc-decode==0.3.2 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (0.3.2)
Requirement already satisfied: python-dateutil==2.8.2 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (2.8.2)
Requirement already satisfied: torch~=1.11 in ./miniconda3/lib/python3.8/site-packages (from ont-bonito) (1.11.0)
Requirement already satisfied: protobuf in ./miniconda3/lib/python3.8/site-packages (from onnxruntime==1.11.0->ont-bonito) (4.21.1)
Requirement already satisfied: flatbuffers in ./miniconda3/lib/python3.8/site-packages (from onnxruntime==1.11.0->ont-bonito) (2.0)
Requirement already satisfied: packaging in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (21.3)
Requirement already satisfied: h5py>=2.8 in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (3.7.0)
Requirement already satisfied: progressbar33>=2.3.1 in ./miniconda3/lib/python3.8/site-packages (from ont-fast5-api==3.3.0->ont-bonito) (2.4)
Requirement already satisfied: cffi in ./miniconda3/lib/python3.8/site-packages (from ont-koi==0.0.9->ont-bonito) (1.14.3)
Requirement already satisfied: onnx in ./miniconda3/lib/python3.8/site-packages (from ont-remora==1.1.0->ont-bonito) (1.11.0)
Requirement already satisfied: scikit-learn in ./miniconda3/lib/python3.8/site-packages (from ont-remora==1.1.0->ont-bonito) (1.1.1)
Requirement already satisfied: tabulate in ./miniconda3/lib/python3.8/site-packages (from ont-remora==1.1.0->ont-bonito) (0.8.9)
Requirement already satisfied: thop in ./miniconda3/lib/python3.8/site-packages (from ont-remora==1.1.0->ont-bonito) (0.1.0.post2206102148)
Requirement already satisfied: wheel in ./miniconda3/lib/python3.8/site-packages (from parasail==1.2->ont-bonito) (0.35.1)
Requirement already satisfied: iso8601 in ./miniconda3/lib/python3.8/site-packages (from pod5-format==0.0.14->ont-bonito) (1.0.2)
Requirement already satisfied: pytz in ./miniconda3/lib/python3.8/site-packages (from pod5-format==0.0.14->ont-bonito) (2022.1)
Requirement already satisfied: more-itertools in ./miniconda3/lib/python3.8/site-packages (from pod5-format==0.0.14->ont-bonito) (8.13.0)
Requirement already satisfied: pyarrow~=7.0.0 in ./miniconda3/lib/python3.8/site-packages (from pod5-format==0.0.14->ont-bonito) (7.0.0)
Requirement already satisfied: six>=1.5 in ./miniconda3/lib/python3.8/site-packages (from python-dateutil==2.8.2->ont-bonito) (1.15.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2021.10.8)
Requirement already satisfied: idna<3,>=2.5 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in ./miniconda3/lib/python3.8/site-packages (from requests==2.25.1->ont-bonito) (3.0.4)
Requirement already satisfied: typing-extensions in ./miniconda3/lib/python3.8/site-packages (from torch~=1.11->ont-bonito) (4.2.0)
Requirement already satisfied: pycparser in ./miniconda3/lib/python3.8/site-packages (from cffi->ont-koi==0.0.9->ont-bonito) (2.20)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./miniconda3/lib/python3.8/site-packages (from packaging->ont-fast5-api==3.3.0->ont-bonito) (3.0.9)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./miniconda3/lib/python3.8/site-packages (from scikit-learn->ont-remora==1.1.0->ont-bonito) (3.1.0)
Requirement already satisfied: joblib>=1.0.0 in ./miniconda3/lib/python3.8/site-packages (from scikit-learn->ont-remora==1.1.0->ont-bonito) (1.1.0)

madhubioinfo avatar Jun 14 '22 16:06 madhubioinfo

@madhubioinfo see https://github.com/nanoporetech/bonito/issues/260 - can you downgrade protobuf and check bonito -v shows 0.5.3.

$ pip install protobuf==3.20.1
$ bonito -v

iiSeymour avatar Jun 14 '22 16:06 iiSeymour

yes now it shows bonito 0.5.3 and yes this worked. but when i run my test command this is what I'm getting it

/home/minion/miniconda3/lib/python3.8/site-packages/torch/cuda/__init__.py:82: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 9000). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at  ../c10/cuda/CUDAFunctions.cpp:112.)
  return torch._C._cuda_getDeviceCount() > 0
Traceback (most recent call last):
  File "/home/minion/miniconda3/bin/bonito", line 33, in <module>
    sys.exit(load_entry_point('ont-bonito==0.5.3', 'console_scripts', 'bonito')())
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/__init__.py", line 34, in main
    args.func(args)
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 26, in main
    init(args.seed, args.device)
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/util.py", line 53, in init
    assert(torch.cuda.is_available())
AssertionError

madhubioinfo avatar Jun 14 '22 17:06 madhubioinfo

What GPU and driver version do you have? Can you provide the output from nvidia-smi?

iiSeymour avatar Jun 14 '22 17:06 iiSeymour

yes now it shows bonito 0.5.3 and yes this worked. but when i run my test command this is what I'm getting it

/home/minion/miniconda3/lib/python3.8/site-packages/torch/cuda/__init__.py:82: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 9000). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at  ../c10/cuda/CUDAFunctions.cpp:112.)
  return torch._C._cuda_getDeviceCount() > 0
Traceback (most recent call last):
  File "/home/minion/miniconda3/bin/bonito", line 33, in <module>
    sys.exit(load_entry_point('ont-bonito==0.5.3', 'console_scripts', 'bonito')())
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/__init__.py", line 34, in main
    args.func(args)
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 26, in main
    init(args.seed, args.device)
  File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/util.py", line 53, in init
    assert(torch.cuda.is_available())
AssertionError

so further I have installed a PyTorch version that has been compiled with your version of the CUDA driver and run the test command , i got following error raceback (most recent call last): File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 567, in _build_master ws.require(requires) File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 884, in require needed = self.resolve(parse_requirements(requirements)) File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 775, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (numpy 1.22.3 (/home/minion/miniconda3/lib/python3.8/site-packages), Requirement.parse('numpy~=1.21.5'), {'ont-bonito'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/minion/miniconda3/bin/bonito", line 33, in sys.exit(load_entry_point('ont-bonito==0.5.3', 'console_scripts', 'bonito')()) File "/home/minion/miniconda3/bin/bonito", line 25, in importlib_load_entry_point return next(matches).load() File "/home/minion/miniconda3/lib/python3.8/importlib/metadata.py", line 77, in load module = import_module(match.group('module')) File "/home/minion/miniconda3/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 783, in exec_module File "", line 219, in _call_with_frames_removed File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/init.py", line 2, in from bonito.cli import basecaller, train, evaluate, view, convert, download, export File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/cli/basecaller.py", line 18, in from bonito.mod_util import call_mods, load_mods_model File "/home/minion/miniconda3/lib/python3.8/site-packages/bonito/mod_util.py", line 6, in from remora.model_util import load_model File "/home/minion/miniconda3/lib/python3.8/site-packages/remora/model_util.py", line 7, in import pkg_resources File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 3239, in def _initialize_master_working_set(): File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 3222, in _call_aside f(*args, **kwargs) File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 3251, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 569, in _build_master return cls._build_from_requirements(requires) File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 582, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/home/minion/miniconda3/lib/python3.8/site-packages/pkg_resources/init.py", line 770, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'wheel' distribution was not found and is required by parasail

madhubioinfo avatar Jun 14 '22 17:06 madhubioinfo

I have this version of nvdia

----------------------------------------------------------------------------+
| NVIDIA-SMI 384.130                Driver Version: 384.130                   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro 5000         Off  | 00000000:05:00.0  On |                  Off |
| 30%   79C    P0    N/A /  N/A |    344MiB /  2489MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1207      G   /usr/lib/xorg/Xorg                           142MiB |
|    0      2089      G   compiz                                       136MiB |
|    0      2859      G   ...AAgAAAAAAAAACAAAAAAAAAA= --shared-files    54MiB |
|    0      7340      G   /usr/lib/firefox/firefox                       2MiB |
|    0     10265      G   /usr/lib/firefox/firefox                       2MiB |

madhubioinfo avatar Jun 14 '22 17:06 madhubioinfo

Your GPU is several generations old and unfortunately it won't run any modern CUDA applications. If you upgrade your GPU this will work. We recommend Ampere architecture-based GPUs (eg RTX A6000).

vellamike avatar Jun 14 '22 21:06 vellamike