No cleaning with exact matches even when lineage is provided

[taylorreiter]

When running charcoal to clean contigs, I noticed:

python -m charcoal run inputs/charcoal-conf.yml all_clean_contigs

filter rank is none; not doing any cleaning.
wrote 2171793 clean bp to /home/tereiter/github/2020-ibd/outputs/charcoal/GCA_900758465.1_genomic.fna.gz.clean.fa.gz

This is stemming from the stage1_hitlist.csv file, which states there is no lineage provided.

GCF_008121495.1_genomic.fna.gz,none,,0,0,0,0,0,0,0,1.000,1.000,,"found exact match: GCF_008121495.1 [Ruminococcus] gnavus ATCC 29149 strain=JCM6515, ASM812149v1. but no provided lineage! cannot analyze."

However, I do provide lineages in my config file:

# location for all generated files
output_dir: /home/tereiter/github/2020-ibd/outputs/charcoal/
# list of genome filenames to decontaminate
genome_list: /home/tereiter/github/2020-ibd/outputs/charcoal_conf/charcoal.genome-list.txt
# directory in which genome filenames live
genome_dir: /home/tereiter/github/2020-ibd/genbank_genomes
# (optional) list of lineages for input genomes. comment out or leave
# blank if none.
provided_lineages: /home/tereiter/github/2020-ibd/outputs/genbank/gather_vita_vars_gtdb_shared_assemblies.x.genbank.lineages.csv
# match_rank is the rank _above_ which cross-lineage matches are considered
# contamination. e.g. if set to 'superkingdom', then Archaeal matches in
# Bacterial genomes will be contamination, but nothing else.
#
# values can be superkingdom, phylum, class, order, family, or genus.
match_rank: order
# sourmash query databases for contamination (SBTs, LCAs, or signatures)
gather_db:
 - /group/ctbrowngrp/gtdb/databases/gtdb-rs202.genomic.k31.zip
# lineages CSV (see `sourmash lca index`) for signatures in query databases
lineages_csv: /group/ctbrowngrp/gtdb/gtdb-rs202.taxonomy.csv
strict: 1

When I removed the stage1/*.hitlist_for_filtering.csv, rule compare_taxonomy_single is re-run, and I think this is where the problem stems from. It loads the 54 provided lineages, but doesn't do anything with them.

$ python -m charcoal run inputs/charcoal-conf.yml all_clean_contigs -j 1
** read 54 provided lineages
** config file checks PASSED!
** from here on out, it's all snakemake...
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
        count   jobs
        1       all_clean_contigs
        54      clean_contigs
        1       combine_hit_list
        52      compare_taxonomy_single
        108
Select jobs to execute...

[Wed Jul  7 09:44:56 2021]
rule compare_taxonomy_single:
    input: /home/tereiter/github/2020-ibd/outputs/charcoal/stage1/GCF_003479605.1_genomic.fna.gz.contigs-tax.json, /home/tereiter/github/2020-ibd/outputs/charcoal/stage1/GCF_003479605.1_genomic.fna.gz.matches.zip, /group/c$browngrp/gtdb/gtdb-rs202.taxonomy.csv, /home/tereiter/github/2020-ibd/outputs/genbank/gather_vita_vars_gtdb_shared_assemblies.x.genbank.lineages.csv, /home/tereiter/github/2020-ibd/outputs/charcoal_conf/charcoal.genome-lis$.txt
    output: /home/tereiter/github/2020-ibd/outputs/charcoal/stage1/GCF_003479605.1_genomic.fna.gz.hitlist_for_filtering.csv, /home/tereiter/github/2020-ibd/outputs/charcoal/stage1/GCF_003479605.1_genomic.fna.gz.genome_summ$ry.csv, /home/tereiter/github/2020-ibd/outputs/charcoal/stage1/GCF_003479605.1_genomic.fna.gz.contam_summary.json
    jobid: 155
    wildcards: g=GCF_003479605.1_genomic.fna.gz

Activating conda environment: /home/tereiter/github/2020-ibd/.snakemake/conda/11b6d58027966e25866faeb8fd08fa61
examining spreadsheet headers...
** assuming column 'ident' is identifiers in spreadsheet
loaded 258406 tax assignments.
working on GCF_003479605.1_genomic.fna.gz
loaded 54 provided lineages
found exact match: GCF_003479605.1 Roseburia sp. AF12-17LB strain=AF12-17LB, ASM347960v1. but no provided lineage!
examining GCF_003479605.1_genomic.fna.gz for contamination
   superkingdom: 0 contigs w/ 0kb
   phylum: 0 contigs w/ 0kb
   class: 0 contigs w/ 0kb
   order: 0 contigs w/ 0kb
   family: 0 contigs w/ 0kb
   genus: 0 contigs w/ 0kb
   (total): 0 contigs w/ 0kb
processed GCF_003479605.1_genomic.fna.gz.
saving contamination summary to /home/tereiter/github/2020-ibd/outputs/charcoal/stage1/GCF_003479605.1_genomic.fna.gz.contam_summary.json

Charcoal version/installation environment:

This is charcoal version v0.1.dev324+g66e6635

Package install path: /home/tereiter/github/2020-ibd/.snakemake/conda/59cc953d/lib/python3.9/site-packages/charcoal
Install-wide config file: /home/tereiter/github/2020-ibd/.snakemake/conda/59cc953d/lib/python3.9/site-packages/charcoal/conf/system.conf
snakemake Snakefile: /home/tereiter/github/2020-ibd/.snakemake/conda/59cc953d/lib/python3.9/site-packages/charcoal/Snakefile