nextflow icon indicating copy to clipboard operation
nextflow copied to clipboard

Endpoint url error

Open nttg8100 opened this issue 3 months ago • 5 comments

Bug report

Nextflow file main.nf

#!/usr/bin/env nextflow
params.values = Channel.from(1)
process echoValue {
        publishDir "${params.outdir}/echoValue/", mode: 'copy'
        input:
        val value 
        output:
        path "*_echoValue.txt"
        script:
        """
        echo "Value: $value" > ${value}_echoValue.txt
        """
    }
workflow {
    echoValue(params.values)
}

nextflow.conf

aws {
    accessKey = "***"
    secretKey = "***"
    client {
        endpoint = 'https://s3-hcm-r1.s3cloud.vn'
        s3PathStyleAccess = true
    }
}

command

nextflow run main.nf --outdir s3://project_1

Versions: nextflow: 23.10.1 nf-amazon:2.1.4

Expected behavior and actual behavior

I expected that the file will be uploaded to the s3 bucket similar to the minio s3 server that I tested successfully on minio image with endpoint="http://localhost:9000".

I tested with the aws cli command that showed I have permission to put objects. The error showed that it failed to parse the endpoint with no additional information.

com.amazonaws.SdkClientException: Unable to execute HTTP request: s3.hcm.amazonaws.com

I think it is failed because the nextflow plugin nf-amazon 2.1.4 or its dependencies SDK provided by the AWS failed to parsing the endpoint.

Steps to reproduce the problem

I can not provide the endpoint url and its credentials with specific patterns like above.

Program output

N E X T F L O W  ~  version 23.10.1
Launching `tmp/main.nf` [jovial_boltzmann] DSL2 - revision: 7325aaaf63
executor >  local (1)
[98/0715e2] process > echoValue (1) [  0%] 0 of 1
ERROR ~ Error executing process > 'echoValue (1)'

Caused by:
  s3.hcm.amazonaws.com


 -- Check '.nextflow.log' file for details

Environment

  • Nextflow version: 23.10.1
  • Java version: 11.0.13 2021-10-19
  • Operating system: macOS
  • Bash version: zsh 5.9 (x86_64-apple-darwin23.0)

Additional context

Is there any documents for recompiling the plugin nf-amazon ? I tried to compile the nextflow after modification of the nf-amazon plugin but it created a build folder structure that is quite different from the [email protected] downloaded by nextflow. I cloned the nextflow repo using tag v23.10.1, then ran

make compile

nttg8100 avatar Mar 12 '24 10:03 nttg8100