Uncorrect retrieval of stage02 PLOT_TSTART and PLOT_STOP parameters

[cosimolupo]

The current syntax in stage02 Snakefile is suspected of not properly inferring the actual values of PLOT_TSTART and PLOT_TSTOP indicated in the corresponding config file, hence causing a blocking error in the execution of Cobrawap.

In more detail, when invoking the command for stage02 execution

cobrawap run_stage --stage 2 --profile PROFILE_NAME

one gets an error, as documented by the following log:

INFO:root:executing Cobrawap stage
[...]
INFO:root:Executing `snakemake -c1 --config PROFILE=PROFILE_NAME STAGE_INPUT=/PATH/TO/OUTPUT/FOLDER/PROFILE_NAME/stage01_data_entry/data.nix --configfile /PATH/TO/CONFIG/FOLDER/stage02_processing/configs/config.yaml`
Config file configs/config_template.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
MissingInputException in rule all in file /PATH/TO/COBRAWAP/SOURCE/CODE/cobrawap/cobrawap/pipeline/stage02_processing/Snakefile, line 26:
Missing input files for rule all:
    output: /PATH/TO/OUTPUT/FOLDER/PROFILE_NAME/stage02_processing/processed_data.nix
    affected files:
        /PATH/TO/OUTPUT/FOLDER/PROFILE_NAME/stage02_processing/processed_traces_0.5-Nones

Changing the current syntax of plot_processed_traces rule in Snakefile from:

use rule template as plot_processed_traces with:
    input:
        data = input_file,
        script = SCRIPTS / 'plot_processed_trace.py'
    params:
        params(channels=config.PLOT_CHANNELS,
               img_name="processed_trace_channel0." + config.PLOT_FORMAT,
               original_data=config.STAGE_INPUT)
    output:
        img_dir = directory(OUTPUT_DIR /
                            str("processed_traces_{t_start}-{t_stop}s"))

to:

use rule template as plot_processed_traces with:
    input:
        data = input_file,
        script = SCRIPTS / 'plot_processed_trace.py'
    params:
        params(channels=config.PLOT_CHANNELS,
               t_start=config.PLOT_TSTART,
               t_stop=config.PLOT_TSTOP,
               img_name="processed_trace_channel0." + config.PLOT_FORMAT,
               original_data=config.STAGE_INPUT)
    output:
        img_dir = directory(OUTPUT_DIR /
                            str("processed_traces_%s-%ss" % (config.PLOT_TSTART,config.PLOT_TSTOP)))

solves the issue, correctly retrieving the parameters values indicated in the config file.

[gulpgiulia]

After a dedicated discussion, we came to the proposition that a first step in tackling this issue should be documenting in a "Developer's manual" how Snakemake is using config parameters in rules, with attention payed to how default values are defined/recalled and when they are overwritten, with the goal of giving out guidelines to be kept common along the pipeline. Indeed, the lack of a clear and coherent description could also prevent developers to contribute to the pipeline with new blocks (rules). I add this comment as a reminder for us concerning the need to take this action