By default Ice fails to use S3 on eu-central-1

[multani]

I configured all the things in AWS as explained in the README of Ice and as we are mostly (99%) using AWS from eu-central-1, I setup the S3 bucket to hold the billing reports on this datacenter.

After starting up and download the instances prices and everything, it successfully manages to download some things from this bucket on eu-central-1 but eventually fails with the following error:

ice_1  | 2016-05-30 13:41:04,902 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor  - trying to download 123456789-aws-billing-detailed-line-items-2016-04.csv.zip...
ice_1  | | Error 2016-05-30 13:41:05,068 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error polling
ice_1  | Message: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: XXXXXXXXXXXXX)
ice_1  |     Line | Method
ice_1  | ->> 1077 | handleErrorResponse        in com.amazonaws.http.AmazonHttpClient
ice_1  | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
ice_1  | |    725 | executeOneRequest          in     ''
ice_1  | |    460 | executeHelper . . . . . .  in     ''
ice_1  | |    295 | execute                    in     ''
ice_1  | |   3710 | invoke . . . . . . . . . . in com.amazonaws.services.s3.AmazonS3Client
ice_1  | |   1010 | getObjectMetadata          in     ''
ice_1  | |    988 | getObjectMetadata . . . .  in     ''
ice_1  | |    275 | downloadFileIfChangedSince in com.netflix.ice.common.AwsUtils
ice_1  | |    175 | poll . . . . . . . . . . . in com.netflix.ice.processor.BillingFileProcessor
ice_1  | |     50 | doWork                     in com.netflix.ice.common.Poller
ice_1  | |     28 | access$000 . . . . . . . . in     ''
ice_1  | |     88 | run                        in com.netflix.ice.common.Poller$1
ice_1  | ^    745 | run . . . . . . . . . . .  in java.lang.Thread

(running from Docker Compose)

It was hard to find more information, but I managed to upgrade the log levels, saw the actual HTTP HEAD query made and tried to do the same with a GET instead and I got the following error message instead:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>InvalidRequest</Code>
<Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message>
</Error>

For some reason, this mechanism is not used directly and I guess that's why it fails on eu-central-1 which doesn't support other mechanism (as far as I understand).

It looks like #193 is here to address this problem although both the explanation and the fix are confusing.

After looking more closely at the code, it appears that if I set the EC2_REGION system property to eu-central-1 at startup it solves the problem, but:

1. This is not documented anywhere;
2. I'm pretty sure there's a way to get the authentication mechanism right, even it the user doesn't manually specify the S3 region to use (or that's what the AWS SDK promise seems to be?)

It would be cool if at least the first point could be addressed so users using other regions than the US ones are also able to start Ice, although I'm not sure exactly why this setting was introduced in the first place.

Probably a better fix would be to use the GetLocation method on the bucket to get where the location of the bucket is and set the correct endpoint then.

[snakelab]

hi, thanks for this information - but this does not fix my installation :-/

#!/bin/bash

AWS_PROFILE=dev AWS_DEFAULT_REGION=eu-central-1 EC2_REGION=eu-central-1 docker-compose -f ./docker-compose.yml up