nextflow

nextflow copied to clipboard

See https://nextflow.slack.com/archives/C02T98A23U7/p1715351870737659 When a workflow publishes many files using S3-to-S3 copy, virtual threads are needed to maximize request throughput, since each publish task is simply waiting on an HTTP request...

bentsherman

## Bug report When tagging a commit with `[ci fast]` integration tests should be skipped, however it looks like they run in any case https://github.com/nextflow-io/nextflow/blob/115a39e9df654be835b4c20ede4f6594b5608d47/.github/workflows/build.yml#L111-L111

pditommaso

AWS: Switch from `containerProperties ` to `ecsProperties`

1

According to @delagoya [we need to change this](https://x.com/delagoya/status/1806036909623329209) ASAP: > I don't know who needs to hear this, but: > > If you use AWS Batch, you should default to...

ewels

Allow configuration of additional Azure retry options for connection to blob storage in nf-azure

1

## New feature When using Azure batch in Nextflow, there currently exists the options `azure.retryPolicy.*` for configuring the behaviour of Azure API request retries: https://www.nextflow.io/docs/latest/config.html#config-azure These options are applied within...

apetkau

Add retry options to Azure Blob client

1

This PR allows configuring the Azure Blob client retry options using `azure.retryPolicy` configuration settings. Find more details [here](https://www.nextflow.io/docs/latest/config.html#scope-azure). Closes #5097. Fixes #5067.

pditommaso

[Azure Batch]: TimeoutException when Nextflow attempts to read files following task completion for large-scale analyses in Azure

8

## Bug report We are using Nextflow with Azure batch to process collections of microbial genomes (whole-genome sequence data). We have began testing out processing larger collections of genomes and...

apetkau

## Bug report `executor.retry.reason` is mentioned in the [documentation](https://www.nextflow.io/docs/latest/config.html#scope-executor), but the code expects `executor.submit.retry.reason`. ### Expected behavior and actual behavior `executor.submit.retry.reason` is used in [this file](https://github.com/nextflow-io/nextflow/blob/0a8a484e082d0be4a41eec9c4ae9c298f9822f19/modules/nextflow/src/main/groovy/nextflow/executor/GridTaskHandler.groovy#L149). Since the `excutor` retry...

thalassemia

hyperqueue (>=0.17.0) and Nextflow interpret memory resource differently

4

## Bug report ### Expected behavior and actual behavior Expected behavior: hyperqueue and nextflow should have same interpretation of memory resources. Actual behavior: When set `memory 2.GB` in Nextflow, the...

bguo068

Using `array` fills my home with many files like "~/nf-PROC_NAME_(6294).o6823940.88"

4

## Bug report version 24.04.2 as in https://nextflow.slack.com/archives/C02T98A23U7/p1719476348728329 I started using the `array` keyword, but i see an accumulation of files in my home ``` ~/nf-PROC_NAME_(6294).o6823940.66 ~/nf-PROC_NAME_(6294).o6823940.67 ~/nf-PROC_NAME_(6294).o6823940.68 ~/nf-PROC_NAME_(6294).o6823940.69 ~/nf-PROC_NAME_(6294).o6823940.7...

lindenb

[ nextflow version 24.04.2.5914 ] Consider the following workflow, ``` process foo { errorStrategy { task.exitStatus == 2 && task.attempt local (3) [2c/02581e] foo [100%] 2 of 2, failed: 2,...

muniheart