nextflow
nextflow copied to clipboard
Provide a unified start timestamp for scripts and config files
The nf-core pipeline template uses a custom timestamp in the various report file names:
def trace_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss')
timeline {
enabled = true
file = "${params.outdir}/pipeline_info/execution_timeline_${trace_timestamp}.html"
}
report {
enabled = true
file = "${params.outdir}/pipeline_info/execution_report_${trace_timestamp}.html"
}
// ...
This is problematic because the new config parser does not support top-level variable declarations like this.
The quick solution is to copy the timestamp expression into each report filename. But then there is a small chance that some report filenames might differ by one second since the timestamp is computed multiple times.
The deeper problem I think is that there is not a single start timestamp that can be used across all scripts and configs.
There is workflow.start
: https://github.com/nextflow-io/nextflow/blob/12ea4d7c1d9b35d0fc0a8a57836cb33649339b6d/modules/nextflow/src/main/groovy/nextflow/script/WorkflowMetadata.groovy#L256
But the default report file names use a different timestamp: https://github.com/nextflow-io/nextflow/blob/12ea4d7c1d9b35d0fc0a8a57836cb33649339b6d/modules/nextflow/src/main/groovy/nextflow/trace/ReportObserver.groovy#L43
It should be possible to define a single timestamp that can be exposed to config files and also used by workflow.start
. It would be another implicit config variable such as launchTime
.
We could also try to expose the entire workflow
variable, but it isn't resolved until later on, so we would have to do some config parser magic to lazy-evaluate expressions that use the workflow
variable.
cc @ewels @pditommaso for your four cents