Getting error 403 forbidden using AWS Elasticsearch (version 7.4)

[harshit98]

Which version of Elastic are you using?

[x] elastic.v7 (for Elasticsearch 7.x) [ ] elastic.v6 (for Elasticsearch 6.x) [ ] elastic.v5 (for Elasticsearch 5.x) [ ] elastic.v3 (for Elasticsearch 2.x) [ ] elastic.v2 (for Elasticsearch 1.x)

Please describe the expected behavior

Elasticsearch client should connect with AWS Elasticsearch cluster successfully with all permissions.

Please describe the actual behavior

When I try to call client.IndexNames() after initializing elasticsearch client, it throws 403 forbidden.

When I try this: curl -X GET http://elasticsearch-host/_cat/indices?v', this works fine.

Code:

package main

import (
	"flag"
	"fmt"
	"github.com/aws/aws-sdk-go/aws/credentials"
	"github.com/olivere/elastic/v7"
	aws "github.com/olivere/elastic/v7/aws/v4"
        "github.com/spf13/viper"
	"log"
)

func main() {
	var (
		accessKey = viper.GetString("aws.access_key")
		secretKey = viper.GetString("aws.secret_key")
		url       = viper.GetString("elasticsearch.host")
		sniff     = flag.Bool("sniff", false, "Enable or disable sniffing")
		region    = flag.String("region", "ap-southeast-1", "AWS Region name")
	)

	if url == "" {
		log.Fatal("please specify a URL with -url")
	}
	if accessKey == "" {
		log.Fatal("missing -access-key or AWS_ACCESS_KEY environment variable")
	}
	if secretKey == "" {
		log.Fatal("missing -secret-key or AWS_SECRET_KEY environment variable")
	}
	if *region == "" {
		log.Fatal("please specify an AWS region with -region")
	}

	creds := credentials.NewStaticCredentials(accessKey, secretKey, "")
	_, err := creds.Get()

	if err != nil {
		log.Fatal("Wrong credentials: ", err)
	}

	signingClient := aws.NewV4SigningClient(creds, *region)

	// Create an Elasticsearch client
	client, err := elastic.NewClient(
		elastic.SetURL(url),
		elastic.SetSniff(*sniff),
		elastic.SetHealthcheck(*sniff),
		elastic.SetHttpClient(signingClient),
	)

	if err != nil {
		log.Fatal(err)
	}

        // This part gives 403 forbidden error
       client.IndexNames()

	_ = client

	// Just a status message
	fmt.Println("Connection succeeded")
}

AWS Elasticsearch Policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:<region>:111111111111:domain/prod-elasticsearch/*"
    }
  ]
}

Any steps to reproduce the behavior?

To reproduce the behaviour, above code can be used as reference.

[Goondrious]

Ran into this as well, FYI, I tried this example (which I think is similar to here?) and it returns 403 as well

[harshit98]

@Goondrious I have resolved this error from my end. Let me know if you need any help.

[Goondrious]

@harshit98 yeah that'd be great!

I found a workaround using opensearch-go and didn't scrutinize it much because it worked:

opensearch "github.com/opensearch-project/opensearch-go"
// ...
client, err := opensearch.NewClient(opensearch.Config{
  Transport: &http.Transport{
	  TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
  },
  Addresses: []string{*url},
  Username:  "...", // For testing only. Don't store credentials in code.
  Password:  "...",
})

How'd you do it?