nextflow

nextflow copied to clipboard

Surface AWS Batch time errors

1

When AWS Batch tasks time out, the error from AWS should be floated up to the Nextflow error log. Currently it just says "Killed by external process" or similar. 'cc...

ewels

Allow AWS Secrets to be used within IncludeConfig context

1

## Bug report As discussed with Ben, I tested this [PR](https://github.com/nextflow-io/nextflow/commit/00c9f226b201c964f67d520d0404342bc33cf61d) on Platform but the AWS Secrets are not being detected. Using slurm and local nextflow CLI, the secrets/pipeline secrets...

markpanganiban

Ability to pipe stdout -> stdin between processes

15

This suggestion / request has come up several times, so wanted to collect the [latest thread](https://nextflow.slack.com/archives/C02T98A23U7/p1716565533096219) into a GitHub issue ('cc @nh13 @kmhernan @mahesh-panchal @adamrtalbot @muffato ) Unix pipes are...

ewels

It would be helpful to include docs on how to run a subprocess in the completion handler. For instance: ``` workflow.onComplete{ def proc = ['./bin/final_checks.sh'].execute() proc.waitForProcessOutput() } ``` ...or another...

nick-youngblut

## Bug report The following errors are not being captured by Nextflow: [Errors 50002-50006](https://cloud.google.com/batch/docs/troubleshooting#vm_reporting_timeout_50002) No space left on device error:  ### Expected behavior and actual behavior Errors 50002-50006, apart...

VasLem

Hi all, I would like to store all .command.* files generated by each process into a central log folder. This is nice so that we can keep track of all...

trvietdung

The nxf_unstage function unnecessarily copies input files that match output glob pattern.

2

## Bug report Many users will use the `scratch true` directive, in part to minimize the size of the shared work directory - to ensure that the files saved to...

robsyme

Output the theoretical `.command.sh` file when using stub

1

It would be great if there was a way to run a workflow with `-stub` but also get the result of what *would* have run in the script section. Currently...

awgymer

AWS Batch: use custom log group name when requesting task logs, if it is defined

1

Nextflow allows the `aws.batch.logsGroup` parameter to be defined to override the log group name. The `getTaskLogStream` method is only able to use the default log group `/aws/batch/job`. Update this method...

nathanthorpe

Declarative DSL for piping high-volume data between tasks/processes on the same node

4

## Usage scenario ### Problem Description Highly tuned workflows may contain a sequence of processes that are connect by named-pipes, UNIX-pipes or similar (sockets?) to prevent the repeated I/O of...

vinjana