circrna icon indicating copy to clipboard operation
circrna copied to clipboard

Published 20 hours ago • verified publisher icon nf-core

[Do not merge!] Pseudo PR for first release

Open jfy133 opened this issue 2 years ago • 6 comments

Do not merge! This is a PR of dev compared to first release for whole-pipeline reviewing purposes.

Comments can be copied over to #66 for safety if necessary and changes made there. Changes should be made to dev and this PR should not be merged into first-commit-for-pseudo-pr!

jfy133 avatar Jan 23 '23 07:01 jfy133

Python linting (black) is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks. To fix this CI test, please run:

  • Install black: pip install black
  • Fix formatting errors in your pipeline: black .

Once you push these changes the test should pass, and you can hide this comment :+1:

We highly recommend setting up Black in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!

Thanks again for your contribution!

github-actions[bot] avatar Feb 01 '23 10:02 github-actions[bot]

Hi @jfy133 & @FriederikeHanssen

I've managed to implement most of the suggestions, I'll address what's not going so smoothly below.

These two things are going to cause linting to fail, the second is really hindering the bam qc, multiqc config suggestions you provided.

Pretty confident I can do everything else out once these are knocked on the head! 🙏🏼

1. Python black

(nf-core) barry@YT-1300:/data/github/circrna$ black ./
All done! ✨ 🍰 ✨
3 files left unchanged.

It still complains about the bin/check_samplesheet.py file?

2. Ill formatted HEREDOC (versions.yml)

Genuinely cannot find the root cause of these improperly formatted versions.yml outputs, I've wasted a decent amount of time on this 🤢

nextflow pull -r dev nf-core/circrna
nextflow run -r dev nf-core/circrna -profile test,docker --module 'circrna_discovery,differentail_expression'

The MultiQC modules fails because the versions.yml file from NFCORE_CIRCRNA:CIRCRNA:DIFFERENTIAL_EXPRESSION:STRINGTIE_PREPDE is not properly formatted:

    "NFCORE_CIRCRNA:CIRCRNA:DIFFERENTIAL_EXPRESSION:STRINGTIE_PREPDE":
        stringtie: 2.2.1
    END_VERSIONS

Process code 👇🏼

process STRINGTIE_PREPDE {
    label 'process_low'

    conda "bioconda::stringtie=2.2.1"
    container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
        'https://depot.galaxyproject.org/singularity/stringtie:2.2.1--hecb563c_2' :
        'quay.io/biocontainers/stringtie:2.2.1--hecb563c_2' }"

    input:
    path gtf

    output:
    path "transcript_count_matrix.csv" , emit: transcript_matrix
    path "gene_count_matrix.csv"       , emit: gene_matrix
    path "versions.yml"                , emit: versions

    when:
    task.ext.when == null || task.ext.when

    script:
    """
    for file in \$(ls *.gtf); do sample_id=\${file%".transcripts.gtf"}; touch samples.txt; printf "\$sample_id\t\$file\n" >> samples.txt ; done

    prepDE.py -i samples.txt

    cat <<-END_VERSIONS > versions.yml
    "${task.process}":
        stringtie: \$(stringtie --version 2>&1)
    END_VERSIONS
    """
}

I'm running into this same issue for the CIRIquant module I'm adding to nf-core modules:

git clone [email protected]:nf-core/modules.git
cd modules
git checkout ciriquant
export PROFILE='docker'
nextflow run ./tests/modules/nf-core/ciriquant -entry test_ciriquant -c ./tests/config/nextflow.config -c ./tests/modules/nf-core/ciriquant/nextflow.config --outdir ciriquant_test/
cat ciriquant_test/ciriquant/versions.yml

    "test_ciriquant:CIRIQUANT":
        ciriquant: 1.1.2
    END_VERSIONS

I've tried defining $VERSION to no avail.

Process code 👇🏼

process CIRIQUANT {
    tag "$meta.id"
    label 'process_high'

    conda "bioconda::ciriquant=1.1.2"
    container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
        'https://depot.galaxyproject.org/singularity/ciriquant:1.1.2--pyhdfd78af_2' :
        'quay.io/biocontainers/ciriquant:1.1.2--pyhdfd78af_2' }"

    input:
    tuple val(meta), path(reads)
    path gtf
    path fasta
    path bwa
    path hisat2

    output:
    tuple val(meta), path("${prefix}/${prefix}.gtf"), emit: gtf
    path("${prefix}")                               , optional: true, emit: intermediates_directory
    path "versions.yml"                             , emit: versions

    when:
    task.ext.when == null || task.ext.when

    script:
    def args = task.ext.args ?: ''
    prefix = task.ext.prefix ?: "${meta.id}"
    bwa_prefix = fasta.toString() == 'genome.fa' ? fasta.toString() : fasta.toString() - ~/.(fa|fasta)$/
    hisat2_prefix = fasta.toString() - ~/.(fa|fasta)$/
    fasta_path = fasta.toRealPath()
    gtf_path = gtf.toRealPath()
    bwa_path = bwa.toRealPath()
    hisat2_path = hisat2.toRealPath()
    """
    BWA=`which bwa`
    HISAT2=`which hisat2`
    STRINGTIE=`which stringtie`
    SAMTOOLS=`which samtools`
    touch travis.yml
    printf "name: ciriquant\ntools:\n  bwa: \$BWA\n  hisat2: \$HISAT2\n  stringtie: \$STRINGTIE\n  samtools: \$SAMTOOLS\n\nreference:\n  fasta: ${fasta_path}\n  gtf: ${gtf_path}\n  bwa_index: ${bwa_path}/${bwa_prefix}\n  hisat_index: ${hisat2_path}/${hisat2_prefix}" >> travis.yml
    CIRIquant \\
        -t ${task.cpus} \\
        -1 ${reads[0]} \\
        -2 ${reads[1]} \\
        --config travis.yml \\
        -o ${prefix} \\
        -p ${prefix} \\
        $args
    cat <<-END_VERSIONS > versions.yml
    "${task.process}":
        ciriquant: \$(echo \$(CIRIquant --version 2>&1) | sed 's/CIRIquant //g' )
    END_VERSIONS
    """
}

This is slowing me down w.r.t implementing some BAM QC modules and fixing up my final MultiQC file.

BarryDigby avatar Feb 08 '23 09:02 BarryDigby

@BarryDigby awk version not printing in ANNOTATION (busybox container)

sh-5.0# awk
BusyBox v1.32.1 (2021-04-13 11:15:36 UTC) multi-call binary.

Usage: awk [OPTIONS] [AWK_PROGRAM] [FILE]...

        -v VAR=VAL      Set variable
        -F SEP          Use SEP as field separator
        -f FILE         Read program from FILE
        -e AWK_PROGRAM

BarryDigby avatar Feb 08 '23 09:02 BarryDigby

Re. busybox unix tool versions: https://stackoverflow.com/questions/75383910/how-to-print-unix-tool-version-in-busybox-container/75385994#75385994

Is this kosher?

BarryDigby avatar Feb 09 '23 14:02 BarryDigby

Re. busybox unix tool versions: https://stackoverflow.com/questions/75383910/how-to-print-unix-tool-version-in-busybox-container/75385994#75385994

Is this kosher?

Good quesiotn, not sure. Maybe one to ask on slack...

jfy133 avatar Feb 16 '23 07:02 jfy133

nf-core lint overall result: Passed :white_check_mark: :warning:

Posted for pipeline commit f96729c

+| ✅ 249 tests passed       |+
!| ❗   1 tests had warnings |!

:heavy_exclamation_mark: Test warnings:

  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).

:white_check_mark: Tests passed:

Run details

  • nf-core/tools version 2.14.1
  • Run at 2024-09-26 13:33:27

github-actions[bot] avatar Jan 13 '24 15:01 github-actions[bot]