Replace Bowtie2 custom modules with official nf-core ones

Open jfy133 opened this issue 11 months ago • 4 comments

To hopefully close #694

TODOs:

[x] Finish converting ASSEMBLY_ALIGN
[ ] Test iGenomes for host removal alternative
[ ] Compare outputs of dev versus this PR
[ ] Consider deprecating or adding extra module for generating read_ids.txt stats

PR checklist

[ ] This comment contains a description of changes (with reason).
[ ] If you've fixed a bug or added code that should be tested, add tests!
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] If necessary, also make a PR on the nf-core/mag branch on the nf-core/test-datasets repository.
[ ] Make sure your code lints (nf-core pipelines lint).
[ ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
[ ] Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

Jan 24 '25 13:01 jfy133

`nf-core pipelines lint` overall result: Passed :white_check_mark: :warning:

Posted for pipeline commit e17cf94

+| ✅ 329 tests passed       |+
#| ❔   2 tests were ignored |#
!| ❗  52 tests had warnings |!

:heavy_exclamation_mark: Test warnings:

pipeline_todos - TODO string in main.nf: Remove this line if you don't need a FASTA file [TODO: try and test using for --host_fasta and --host_genome]
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
local_component_structure - rename_predastool.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - busco_save_download.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - quast_bins.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - pool_single_reads.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - centrifuge.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - bowtie2_assembly_build.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - combine_tsv.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - busco.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - bowtie2_assembly_align.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - busco_summary.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - tiara_classify.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - quast.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - split_fasta.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - cat_db_generate.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - gtdbtk_summary.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - pool_paired_reads.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - convert_depths.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - bin_summary.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - bowtie2_removal_build.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - adjust_maxbin2_ext.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - quast_bins_summary.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - busco_db_preparation.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - rename_postdastool.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - cat.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - cat_db.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - gtdbtk_db_preparation.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - mag_depths.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - mag_depths_plot.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - kraken2_db_preparation.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - mag_depths_summary.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - cat_summary.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - nanolyse.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - bowtie2_removal_align.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - krona.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - kraken2.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - shortread_preprocessing.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - binning_preparation.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - binning_refinement.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - virus_identification.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - domain_classification.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - longread_preprocessing.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - ancient_dna.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - depths.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - binning.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - gtdbtk.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - bin_qc.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - tiara.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

:grey_question: Tests ignored:

nextflow_config - Config default ignored: params.phix_reference
nextflow_config - Config default ignored: params.lambda_reference

:white_check_mark: Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-mag_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-mag_logo_light.png
files_exist - File found: docs/images/nf-core-mag_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: conf/igenomes_ignored.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-mag_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowMag.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: validation.help.enabled
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable found: validation.help.beforeText
nextflow_config - Config variable found: validation.help.afterText
nextflow_config - Config variable found: validation.help.command
nextflow_config - Config variable found: validation.summary.beforeText
nextflow_config - Config variable found: validation.summary.afterText
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 3.3.2dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
nextflow_config - Config default value correct: params.spades_fix_cpus= -1
nextflow_config - Config default value correct: params.spadeshybrid_fix_cpus= -1
nextflow_config - Config default value correct: params.metabat_rng_seed= 1
nextflow_config - Config default value correct: params.clip_tool= fastp
nextflow_config - Config default value correct: params.reads_minlength= 15
nextflow_config - Config default value correct: params.fastp_qualified_quality= 15
nextflow_config - Config default value correct: params.fastp_cut_mean_quality= 15
nextflow_config - Config default value correct: params.adapterremoval_minquality= 2
nextflow_config - Config default value correct: params.adapterremoval_adapter1= AGATCGGAAGAGCACACGTCTGAACTCCAGTCACNNNNNNATCTCGTATGCCGTCTTCTGCTTG
nextflow_config - Config default value correct: params.adapterremoval_adapter2= AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
nextflow_config - Config default value correct: params.bbnorm_target= 100
nextflow_config - Config default value correct: params.bbnorm_min= 5
nextflow_config - Config default value correct: params.longreads_min_length= 1000
nextflow_config - Config default value correct: params.longreads_keep_percent= 90
nextflow_config - Config default value correct: params.longreads_length_weight= 10
nextflow_config - Config default value correct: params.longread_adaptertrimming_tool= porechop_abi
nextflow_config - Config default value correct: params.longread_filtering_tool= filtlong
nextflow_config - Config default value correct: params.gtdb_db= https://data.gtdb.ecogenomic.org/releases/release220/220.0/auxillary_files/gtdbtk_package/full_package/gtdbtk_r220_data.tar.gz
nextflow_config - Config default value correct: params.gtdbtk_min_completeness= 50.0
nextflow_config - Config default value correct: params.gtdbtk_max_contamination= 10.0
nextflow_config - Config default value correct: params.gtdbtk_min_perc_aa= 10.0
nextflow_config - Config default value correct: params.gtdbtk_min_af= 0.65
nextflow_config - Config default value correct: params.gtdbtk_pplacer_cpus= 1
nextflow_config - Config default value correct: params.genomad_min_score= 0.7
nextflow_config - Config default value correct: params.genomad_splits= 1
nextflow_config - Config default value correct: params.binning_map_mode= group
nextflow_config - Config default value correct: params.min_contig_size= 1500
nextflow_config - Config default value correct: params.min_length_unbinned_contigs= 1000000
nextflow_config - Config default value correct: params.max_unbinned_contigs= 100
nextflow_config - Config default value correct: params.bin_domain_classification_tool= tiara
nextflow_config - Config default value correct: params.tiara_min_length= 3000
nextflow_config - Config default value correct: params.binqc_tool= busco
nextflow_config - Config default value correct: params.checkm_download_url= https://zenodo.org/records/7401545/files/checkm_data_2015_01_16.tar.gz
nextflow_config - Config default value correct: params.checkm2_db_version= 5571251
nextflow_config - Config default value correct: params.refine_bins_dastool_threshold= 0.5
nextflow_config - Config default value correct: params.postbinning_input= raw_bins_only
nextflow_config - Config default value correct: params.gunc_database_type= progenomes
nextflow_config - Config default value correct: params.pydamage_accuracy= 0.5
nextflow_config - Config default value correct: params.freebayes_ploidy= 1
nextflow_config - Config default value correct: params.freebayes_min_basequality= 20
nextflow_config - Config default value correct: params.freebayes_minallelefreq= 0.33
nextflow_config - Config default value correct: params.bcftools_view_high_variant_quality= 30
nextflow_config - Config default value correct: params.bcftools_view_medium_variant_quality= 20
nextflow_config - Config default value correct: params.bcftools_view_minimal_allelesupport= 3
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-mag_logo_light.png matches the template
files_unchanged - docs/images/nf-core-mag_logo_light.png matches the template
files_unchanged - docs/images/nf-core-mag_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 24.04.2, Config: 24.04.2
readme - README Zenodo placeholder was replaced with DOI.
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: template_version_comment.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - BOWTIE2_HOST_REMOVAL_BUILD found in conf/base.config and Nextflow scripts.
base_config - BOWTIE2_HOST_REMOVAL_ALIGN found in conf/base.config and Nextflow scripts.
base_config - BOWTIE2_PHIX_REMOVAL_ALIGN found in conf/base.config and Nextflow scripts.
base_config - PORECHOP_PORECHOP found in conf/base.config and Nextflow scripts.
base_config - NANOLYSE found in conf/base.config and Nextflow scripts.
base_config - FILTLONG found in conf/base.config and Nextflow scripts.
base_config - CENTRIFUGE_CENTRIFUGE found in conf/base.config and Nextflow scripts.
base_config - KRAKEN2 found in conf/base.config and Nextflow scripts.
base_config - KRONA_KTIMPORTTAXONOMY found in conf/base.config and Nextflow scripts.
base_config - CAT_DB_GENERATE found in conf/base.config and Nextflow scripts.
base_config - CAT found in conf/base.config and Nextflow scripts.
base_config - GTDBTK_CLASSIFYWF found in conf/base.config and Nextflow scripts.
base_config - MEGAHIT found in conf/base.config and Nextflow scripts.
base_config - SPADES found in conf/base.config and Nextflow scripts.
base_config - SPADESHYBRID found in conf/base.config and Nextflow scripts.
base_config - BOWTIE2_ASSEMBLY_ALIGN found in conf/base.config and Nextflow scripts.
base_config - METABAT2_METABAT2 found in conf/base.config and Nextflow scripts.
base_config - MAG_DEPTHS found in conf/base.config and Nextflow scripts.
base_config - BUSCO found in conf/base.config and Nextflow scripts.
base_config - MAXBIN2 found in conf/base.config and Nextflow scripts.
base_config - DASTOOL_DASTOOL found in conf/base.config and Nextflow scripts.
base_config - CHECKM2_PREDICT found in conf/base.config and Nextflow scripts.
modules_config - conf/modules.config found and not ignored.
modules_config - FASTQC_RAW found in conf/modules.config and Nextflow scripts.
modules_config - FASTP found in conf/modules.config and Nextflow scripts.
modules_config - ADAPTERREMOVAL_PE found in conf/modules.config and Nextflow scripts.
modules_config - ADAPTERREMOVAL_SE found in conf/modules.config and Nextflow scripts.
modules_config - BOWTIE2_PHIX_REMOVAL_ALIGN found in conf/modules.config and Nextflow scripts.
modules_config - BOWTIE2_HOST_REMOVAL_ALIGN found in conf/modules.config and Nextflow scripts.
modules_config - FASTQC_TRIMMED found in conf/modules.config and Nextflow scripts.
modules_config - BBMAP_BBNORM found in conf/modules.config and Nextflow scripts.
modules_config - PORECHOP_PORECHOP found in conf/modules.config and Nextflow scripts.
modules_config - PORECHOP_ABI found in conf/modules.config and Nextflow scripts.
modules_config - FILTLONG found in conf/modules.config and Nextflow scripts.
modules_config - NANOQ found in conf/modules.config and Nextflow scripts.
modules_config - NANOLYSE found in conf/modules.config and Nextflow scripts.
modules_config - CHOPPER found in conf/modules.config and Nextflow scripts.
modules_config - NANOPLOT_RAW found in conf/modules.config and Nextflow scripts.
modules_config - NANOPLOT_FILTERED found in conf/modules.config and Nextflow scripts.
modules_config - CENTRIFUGE_CENTRIFUGE found in conf/modules.config and Nextflow scripts.
modules_config - CENTRIFUGE_KREPORT found in conf/modules.config and Nextflow scripts.
modules_config - KRAKEN2 found in conf/modules.config and Nextflow scripts.
modules_config - KREPORT2KRONA_CENTRIFUGE found in conf/modules.config and Nextflow scripts.
modules_config - KRONA_KTIMPORTTAXONOMY found in conf/modules.config and Nextflow scripts.
modules_config - MEGAHIT found in conf/modules.config and Nextflow scripts.
modules_config - METASPADES found in conf/modules.config and Nextflow scripts.
modules_config - METASPADESHYBRID found in conf/modules.config and Nextflow scripts.
modules_config - QUAST found in conf/modules.config and Nextflow scripts.
modules_config - GENOMAD_ENDTOEND found in conf/modules.config and Nextflow scripts.
modules_config - BOWTIE2_ASSEMBLY_BUILD found in conf/modules.config and Nextflow scripts.
modules_config - BOWTIE2_ASSEMBLY_ALIGN found in conf/modules.config and Nextflow scripts.
modules_config - SAMTOOLS_INDEX found in conf/modules.config and Nextflow scripts.
modules_config - MAG_DEPTHS_PLOT found in conf/modules.config and Nextflow scripts.
modules_config - BIN_SUMMARY found in conf/modules.config and Nextflow scripts.
modules_config - BUSCO_DB_PREPARATION found in conf/modules.config and Nextflow scripts.
modules_config - BUSCO found in conf/modules.config and Nextflow scripts.
modules_config - BUSCO_SAVE_DOWNLOAD found in conf/modules.config and Nextflow scripts.
modules_config - BUSCO_SUMMARY found in conf/modules.config and Nextflow scripts.
modules_config - ARIA2_UNTAR found in conf/modules.config and Nextflow scripts.
modules_config - CHECKM_LINEAGEWF found in conf/modules.config and Nextflow scripts.
modules_config - CHECKM_QA found in conf/modules.config and Nextflow scripts.
modules_config - COMBINE_BINQC_TSV found in conf/modules.config and Nextflow scripts.
modules_config - CHECKM2_DATABASEDOWNLOAD found in conf/modules.config and Nextflow scripts.
modules_config - CHECKM2_PREDICT found in conf/modules.config and Nextflow scripts.
modules_config - GUNC_DOWNLOADDB found in conf/modules.config and Nextflow scripts.
modules_config - GUNC_RUN found in conf/modules.config and Nextflow scripts.
modules_config - GUNC_MERGECHECKM found in conf/modules.config and Nextflow scripts.
modules_config - CAT_DB_GENERATE found in conf/modules.config and Nextflow scripts.
modules_config - CAT found in conf/modules.config and Nextflow scripts.
modules_config - CAT_SUMMARY found in conf/modules.config and Nextflow scripts.
modules_config - GTDBTK_CLASSIFYWF found in conf/modules.config and Nextflow scripts.
modules_config - GTDBTK_SUMMARY found in conf/modules.config and Nextflow scripts.
modules_config - PROKKA found in conf/modules.config and Nextflow scripts.
modules_config - PRODIGAL found in conf/modules.config and Nextflow scripts.
modules_config - FREEBAYES found in conf/modules.config and Nextflow scripts.
modules_config - BCFTOOLS_VIEW found in conf/modules.config and Nextflow scripts.
modules_config - BCFTOOLS_CONSENSUS found in conf/modules.config and Nextflow scripts.
modules_config - BCFTOOLS_INDEX found in conf/modules.config and Nextflow scripts.
modules_config - PYDAMAGE_ANALYZE found in conf/modules.config and Nextflow scripts.
modules_config - PYDAMAGE_FILTER found in conf/modules.config and Nextflow scripts.
modules_config - SAMTOOLS_FAIDX found in conf/modules.config and Nextflow scripts.
modules_config - METABAT2_JGISUMMARIZEBAMCONTIGDEPTHS found in conf/modules.config and Nextflow scripts.
modules_config - METABAT2_METABAT2 found in conf/modules.config and Nextflow scripts.
modules_config - MAXBIN2 found in conf/modules.config and Nextflow scripts.
modules_config - ADJUST_MAXBIN2_EXT found in conf/modules.config and Nextflow scripts.
modules_config - CONCOCT_ found in conf/modules.config and Nextflow scripts.
modules_config - SPLIT_FASTA found in conf/modules.config and Nextflow scripts.
modules_config - DASTOOL_FASTATOCONTIG2BIN_METABAT2 found in conf/modules.config and Nextflow scripts.
modules_config - DASTOOL_FASTATOCONTIG2BIN_MAXBIN2 found in conf/modules.config and Nextflow scripts.
modules_config - DASTOOL_FASTATOCONTIG2BIN_CONCOCT found in conf/modules.config and Nextflow scripts.
modules_config - DASTOOL_FASTATOCONTIG2BIN_TIARA found in conf/modules.config and Nextflow scripts.
modules_config - DASTOOL_DASTOOL found in conf/modules.config and Nextflow scripts.
modules_config - RENAME_POSTDASTOOL found in conf/modules.config and Nextflow scripts.
modules_config - TIARA_TIARA found in conf/modules.config and Nextflow scripts.
modules_config - TIARA_CLASSIFY found in conf/modules.config and Nextflow scripts.
modules_config - TIARA_SUMMARY found in conf/modules.config and Nextflow scripts.
modules_config - MMSEQS_DATABASES found in conf/modules.config and Nextflow scripts.
modules_config - METAEUK_EASYPREDICT found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.2.0

Run details

nf-core/tools version 3.2.0
Run at 2025-03-13 13:13:35

Jan 24 '25 13:01 github-actions[bot]

[!WARNING] Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.1.2. Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

Feb 05 '25 05:02 nf-core-bot

@nf-core-bot fix linting

Feb 06 '25 08:02 jfy133

Next steps (as it's been a while since I worked on this):

[ ] Run dev vs bowtie2fixes
- [x] nextflow run ../main.nf -profile test,docker --outdir ./results-bowtie2fixes
- [x] nextflow run nf-core/mag -r dev -profile test,docker --outdir ./results-dev
[ ] Compare outputs:
- [x] Log files
- [x] Output files
- [x] Numbers approximately same (e.g. in logs, MultiQC)
[ ] Run with host genome removal (check numbers))
[ ] Run with selecting host_genome from iGenomes to make sure that is accepted by the new modules (check numbers)
[ ] Check aDNA special bowtie2 assembly alignment parmaeters work
[ ] Check all --save_*_reads works
[ ] Check @MeriamOs 's issue actually fixed
[ ] Add extra local module for generating *.read_ids.txt stats from BOWTIE2_REMOVAL_ALIGN
- [ ] Or use https://bioinf.shenwei.me/seqkit/usage/#seq
[ ] Delete old local bowtie2 modules

To invesigate before merge

[ ] Test profile results in missing half expected bins (19) in this branch (9), seems to be MetaBAT
- The depths file looks like it shows that for bowtie2fixes results that it only was calculated based on mapping against MEGAHIT-test_minigut-test_minigut_sample2.bam and not itself
- The test profile should be mapping against own so I think somehow the join/combine combination isn't working in BINNING_PREPARATION -> I started dumping everything in the group mapping mode section

Feb 27 '25 13:02 jfy133

Replace Bowtie2 custom modules with official nf-core ones

PR checklist

nf-core pipelines lint overall result: Passed :white_check_mark: :warning:

:heavy_exclamation_mark: Test warnings:

:grey_question: Tests ignored:

:white_check_mark: Tests passed:

Run details

To invesigate before merge

`nf-core pipelines lint` overall result: Passed :white_check_mark: :warning: