website
website copied to clipboard
Pipeline conflicts with user AWS configuration
I’ve got a default AWS configuration at $HOME/.aws/config
which includes the following:
[default]
region = us-east-1
This causes Sarek to fail when it’s trying to download the iGenomes data. The error message is:
The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: EFD0047C3BC01410; S3 Extended Request ID: MMiKH3WnN3XQ8ZdtQN48D6lriG36JRv7cbMm5yuNv8cNqHpwhfB5XFUJvZt4pY6qX1ZsOQoPGMg=)
-- Check script '/home/konrad/.nextflow/assets/nf-core/sarek/main.nf' at line: 268 or see '.nextflow.log' file for more details
Removing the default AWS profile fixes the issue, but should not be necessary: Sarek should explicitly set its own AWS profile (including a region) to avoid conflicts with system profiles.
(This is using Sarek at rev b952fe2b3fcd3541237c9f4a9f27e1852f537967 with no user configuration, launched as nextflow run nf-core/sarek --input samples
.)
Hi @klmr, I transferred your issue here. It has been reported with nf-core, and we're working on addressing it. We talked about that on our slack yesterday (https://nfcore.slack.com/archives/CE7FBAMRP/p1588157967036100) and we are thinking of a way to make it clearer for the users.
Note that I'm being hassled by AWS people to mirror iGenomes to other regions, so that should help when I finally get to it.