Python errors when running lifton

[zgb963]

Hello,

I'm having an issue with running the lifton software. I'm running it on an HPC environment using 100GB memory and a computer node that has 2000 cores. The below bash script has the command I'm using to run liftton. The target genome is rhemac10 FASTA (rheMac10.fa) and I've also inputed the human genome hg38 FASTA (hg38.fa) and human genome annotation in GTF format from NCBI (hg38.ncbiRefSeq.gtf). I want to output a lifton rhemac10 annotation (hg38_lifton_rhemac10.gff3)

#!/bin/bash

SECONDS=0

cd ~/macaque_snRNAseq

#make sure than conda env is sourced
. /home/genevieve.baddoo1-umw/miniconda3/etc/profile.d/conda.sh

#activate lifton_pip conda env 
conda activate lifton_pip

lifton -g liftoff/hg38.ncbiRefSeq.gtf -o lifton/hg38_lifton_rhemac10.gff3 -copies -infer-genes liftoff/rheMac10.fa liftoff/hg38.fa

duration=$SECONDS
echo "$(($duration / 3600)) hours and $((($duration % 3600) / 60)) minutes and $(($duration % 60)) seconds elapsed."

Here is the bsub command I'm using to run lifton

bsub -q long -R rusage[mem=25G] -R span[hosts=1] -W 96:00 -n 4 -o ~/macaque_snRNAseq/lifton/my_out.%J -e ~/macaque_snRNAseq/lifton/my_err.%J ~/macaque_snRNAseq/scripts/lifton.sh

I've installed lifton in a conda environment using pip. It does run for over an hour but then I get the following python errors

252893 of 414578 (61%)
257039 of 414578 (62%)
261185 of 414578 (63%)
265330 of 414578 (64%)
2024-07-31 17:48:31,795 - INFO - Committing changes
2024-07-31 17:48:32,102 - INFO - Creating relations(parent) index
2024-07-31 17:48:36,819 - INFO - Creating relations(child) index
2024-07-31 17:48:41,709 - INFO - Creating features(featuretype) index
2024-07-31 17:48:44,560 - INFO - Creating features (seqid, start, end) index
2024-07-31 17:48:48,643 - INFO - Creating features (seqid, start, end, strand) index
2024-07-31 17:48:53,080 - INFO - Running ANALYZE features
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/align_features.py", line 61, in align_single_chroms
    minimap2_index = build_minimap2_index(target_file, args, threads_arg, minimap2_path)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/align_features.py", line 109, in build_minimap2_index
    subprocess.run(
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/subprocess.py", line 1720, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'minimap2'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/bin/lifton", line 8, in <module>
    sys.exit(main())
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/lifton.py", line 352, in main
    run_all_lifton_steps(args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/lifton.py", line 267, in run_all_lifton_steps
    liftoff_annotation = lifton_utils.exec_liftoff(lifton_outdir, args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/lifton_utils.py", line 113, in exec_liftoff
    liftoff_annotation = run_liftoff.run_liftoff(outdir, args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/run_liftoff.py", line 25, in run_liftoff
    liftoff_main.run_all_liftoff_steps(liftoff_args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/liftoff_main.py", line 19, in run_all_liftoff_steps
    feature_db, feature_hierarchy, ref_parent_order = liftover_types.lift_original_annotation(ref_chroms, target_chroms,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/liftover_types.py", line 15, in lift_original_annotation
    align_and_lift_features(ref_chroms, target_chroms, args, feature_hierarchy, liftover_type, unmapped_features,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/liftover_types.py", line 23, in align_and_lift_features
    aligned_segments= align_features.align_features_to_target(ref_chroms, target_chroms, args,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/align_features.py", line 24, in align_features_to_target
    for result in pool.imap_unordered(func, np.arange(0, len(target_chroms))):
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
FileNotFoundError: [Errno 2] No such file or directory: 'minimap2'

Am I using enough memory and cores to run the software? Does the input genome annotation have to be in GFF3 format instead of GTF? Also, can lifton output an annotation in GTF format or does it only output in GFF3 format?