nextflow
nextflow copied to clipboard
nextflow-io
Include retries in git repo requests
This PR include retries when performing request in git repositories.
The request errors are managed at RepositoryProvider.checkResponse, where some AbortException or RateLimitExceededExceptions are thrown for some HTTP error codes, or by IOExceptions thrown by the URLConnection methods. They include SocketExceptions and 50X error codes. I have made all IOExceptions retried by default, but this could be modified by including other codes in the checkResponse method.
Deploy Preview for nextflow-docs-staging canceled.
| Name | Link |
|---|---|
| Latest commit | 02fc98b99f275b15d6e7ccdb4cd46838d54ff7a9 |
| Latest deploy log | https://app.netlify.com/projects/nextflow-docs-staging/deploys/6852d34462679c00080e0825 |
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
I have added nextflow.retryPolicy and used it for the following operations:
- git repository provider
- ~~k8s client (replaces
k8s.client.retryPolicy)~~ - file publishing (replaces
workflow.output.retryPolicy)
There are a few other places that use retry settings, namely the grid and cloud executors. In a future iteration I think we could consolidate them as well, perhaps using the $<name> syntax in the executor scope to apply different settings to different contexts
This is the PR implementing the use of Java HttpClient https://github.com/nextflow-io/nextflow/pull/6188
Since there are several overlapping changes I've created a new PR. Closing this in favour of https://github.com/nextflow-io/nextflow/pull/6195