Received 403 from AWSAuth, invalidating credentials for retrial. ES inside VPC with open access

[nikhilo]

I have set up the ES inside of a VPC. The access policy is set to open. I'm running this proxy on an instance inside the same VPC with public IP. And then trying to access the /_cat/indices endpoint. Getting a 403 on the curl and seeing following error in the proxy logs,

root@some-host:~# docker run --rm -p 9200:9200 abutaha/aws-es-proxy:v1.0 -endpoint https://vpc-mydomain-randomcharacters.us-east-1.es.amazonaws.com -listen 0.0.0.0:9200 -verbose
time="2020-08-04 09:33:15" level=info msg="Listening on 0.0.0.0:9200...\n"
time="2020-08-04 09:33:21" level=info msg="Generated fresh AWS Credentials object"
time="2020-08-04 09:33:24" level=error msg="Received 403 from AWSAuth, invalidating credentials for retrial"
2020/04/08 09:33:24  -> GET; 69.31.114.42:61611; /_cat/indices; ; 403; 3.268s

Not necessarily saying that there's a problem with the proxy. Just need help in identifying the problem here.

[abutaha]

Try passing '-no-sign-reqs' and see if that works.

[nikhilo]

Nope, that's a quick 401. Equivalent to me doing, curl https://vpc-mydomain-randomcharacters.us-east-1.es.amazonaws.com/_cat/indices from the ec2 instance.

[abutaha]

ok, is the policy open to public or it is open to specific IP?

[nikhilo]

The policy is allowing "*" to the IAM role of the instance, on which I'm running the proxy from

[DustinKLo]

I'm getting the same issue did you ever find a workaround @nikhilo ? thanks 👍

[DustinKLo]

i made a fix for in my org's fork of this repo: https://github.com/hysds/aws-es-proxy/commit/c656b87ba1d15e819b7bdeeee450e92eb37063a3

looks like the proxy parses the endpoint variable to get the values of region and service and region was being set to an empty string

the AWS Elasticsearch endpoint that we're using isn't what the code is expecting: https://vpce-########-######vpce-#####-######.us-west-#.vpce.amazonaws.com

so i removed that logic and added an additional flag region to set it explicitly

[spicoflorin]

@DustinKLo Thank you very much for the fix! Can you please merge your changes in this repository? @abutaha it will be very helpful for the community to include this fix into the releases. Thanks

[abutaha]

i made a fix for in my org's fork of this repo: hysds@c656b87

looks like the proxy parses the endpoint variable to get the values of region and service and region was being set to an empty string

the AWS Elasticsearch endpoint that we're using isn't what the code is expecting: https://vpce-########-######vpce-#####-######.us-west-#.vpce.amazonaws.com

so i removed that logic and added an additional flag region to set it explicitly

Thanks for pointing it out. Can you please submit a merge request?

[DustinKLo]

sure thing @abutaha https://github.com/abutaha/aws-es-proxy/pull/84