nextflow
nextflow copied to clipboard
Endpoint url error
Bug report
Nextflow file main.nf
#!/usr/bin/env nextflow
params.values = Channel.from(1)
process echoValue {
publishDir "${params.outdir}/echoValue/", mode: 'copy'
input:
val value
output:
path "*_echoValue.txt"
script:
"""
echo "Value: $value" > ${value}_echoValue.txt
"""
}
workflow {
echoValue(params.values)
}
nextflow.conf
aws {
accessKey = "***"
secretKey = "***"
client {
endpoint = 'https://s3-hcm-r1.s3cloud.vn'
s3PathStyleAccess = true
}
}
command
nextflow run main.nf --outdir s3://project_1
Versions: nextflow: 23.10.1 nf-amazon:2.1.4
Expected behavior and actual behavior
I expected that the file will be uploaded to the s3 bucket similar to the minio s3 server that I tested successfully on minio image with endpoint="http://localhost:9000".
I tested with the aws cli command that showed I have permission to put objects. The error showed that it failed to parse the endpoint with no additional information.
com.amazonaws.SdkClientException: Unable to execute HTTP request: s3.hcm.amazonaws.com
I think it is failed because the nextflow plugin nf-amazon 2.1.4 or its dependencies SDK provided by the AWS failed to parsing the endpoint.
Steps to reproduce the problem
I can not provide the endpoint url and its credentials with specific patterns like above.
Program output
N E X T F L O W ~ version 23.10.1
Launching `tmp/main.nf` [jovial_boltzmann] DSL2 - revision: 7325aaaf63
executor > local (1)
[98/0715e2] process > echoValue (1) [ 0%] 0 of 1
ERROR ~ Error executing process > 'echoValue (1)'
Caused by:
s3.hcm.amazonaws.com
-- Check '.nextflow.log' file for details
Environment
- Nextflow version: 23.10.1
- Java version: 11.0.13 2021-10-19
- Operating system: macOS
- Bash version: zsh 5.9 (x86_64-apple-darwin23.0)
Additional context
Is there any documents for recompiling the plugin nf-amazon ? I tried to compile the nextflow after modification of the nf-amazon plugin but it created a build folder structure that is quite different from the [email protected] downloaded by nextflow. I cloned the nextflow repo using tag v23.10.1, then ran
make compile