yet-another-cloudwatch-exporter

yet-another-cloudwatch-exporter copied to clipboard

Is there an existing issue for this?

• [X] I have searched the existing issues

Feature description

To include the option to include LinkedAccounts when using a monitoring account.

I would probably say to have it default on, as I would imagine majority of people would want to enabled vs disabled.

What might the configuration look like?

apiVersion: v1alpha1
sts-region: ap-southeast-2
discovery:
  jobs:
    - type: AWS/EC2
      regions:
        - ap-southeast-2
      linkedaccounts: true
      period: 300
      length: 300
      metrics:
        #EC2
        - name: CPUUtilization
          statistics:
            - Average

Anything else?

Quick config based off the metrics AWS SDK and stored token

package main

import (
	"flag"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/cloudwatch"

	"fmt"
)

var (
	metric                = flag.String("metric", "", "Metric to query")
	namespace             = flag.String("namespace", "", "Namespace to query")
	dimension             = flag.String("dimension", "", "Dimension Set")
	includelinkedaccounts = flag.Bool("includelinkedaccounts", true, "View Attached Acounts")
)

func main() {

	flag.Parse()

	metric := *metric
	namespace := *namespace
	dimension := *dimension
	setincludelinkedaccounts := *includelinkedaccounts

	// Initialize a session that the SDK uses to load
	// credentials from the shared credentials file ~/.aws/credentials
	// and configuration from the shared configuration file ~/.aws/config.
	sess := session.Must(session.NewSessionWithOptions(session.Options{
		SharedConfigState: session.SharedConfigEnable,
	}))

	// Create CloudWatch client
	svc := cloudwatch.New(sess)

	// Get the list of metrics matching your criteria
	result, err := svc.ListMetrics(&cloudwatch.ListMetricsInput{
		MetricName:            aws.String(metric),
		Namespace:             aws.String(namespace),
		IncludeLinkedAccounts: aws.Bool(setincludelinkedaccounts),
		Dimensions: []*cloudwatch.DimensionFilter{
			&cloudwatch.DimensionFilter{
				Name: aws.String(dimension),
			},
		},
	})
	if err != nil {
		fmt.Println("Error", err)
		return
	}

	fmt.Println("Metrics", result.Metrics)
}

then running the following command outputs data .\aws-metrics --namespace AWS/EC2 --metric CPUUtilization --dimension InstanceId --includelinkedaccounts=true OR .\aws-metrics --namespace AWS/EC2 --metric CPUUtilization --dimension InstanceId

returns values from IAM role account and all linkedaccounts from IAM role account

.\aws-metrics --namespace AWS/EC2 --metric CPUUtilization --dimension InstanceId --includelinkedaccounts=false

only returns values from IAM role account

@erroltuparker thanks for the detailed issue. I haven't used monitoring accounts before. Would this allow to fetch metrics from all linked accounts within one single request? This seems to be limited to a single region though, correct?

@cristiangreco yes, that's what I've seen from testing.

In my environment, the account that my role is created in is in the monitoring account, which has 8 EC2 instances in it. Viewing Cloudwatch metrics within the Console, I can see 16 EC2 instances across my linked accounts. Currently, YACE can only detect those 8 EC2 instances (which are the ones deployed within the monitoring account), but using the above method, I was able to pull metrics from all 16 EC2 instances.

Regarding the regions, I'm not sure about that. My assumption would be yes because generally how AWS seems to operate, but at the moment I only have access to resources deployed within a single region, so I can't confirm or deny it for you.

We (@hugomcfonseca @marinarcgomes) have reached the same conclusion. Observability Access Manager(OAM) does allow one to fetch metrics from all linked accounts with one single request of ListMetrics, just like @erroltuparker mentioned.

Our context is that we have a multi-account and multi-region setup, and are therefore trying to use OAM to sink and link all accounts that hold metrics into one single monitoring account where we are running yace.

However, since discovery is bound to the resources of the calling account (i.e. the monitoring account using the OAM terminology), the metrics corresponding to resources from linked accounts are skipped.

Would it make sense to not skip the found metrics that don't relate with any resource within the monitoring account? By indicating that we should includeLinkedAccounts: true in the config we are already within the context of a monitoring account so skipping could be avoided altogether.

Happy to open a PR for this btw.

This would also be a huge win for us as we are currently running YACE in over 60 AWS accounts, but also using a single monitoring account. I would love to be able to run YACE in just our monitoring account, even if it was multiple instances that are exporting different metrics (for performance reasons). Managing YACE in a single spot would be so much easier.

Voting for this too.

What I found i that SDKs usage details are:

• IncludeLinkedAccounts in listing metrics
• AccountId in getting metric data.

Discovery/listing metrics would also require something added.

But this looks like a very valuable feature over all AWS metric namespaces.

Also a duplicate in #1009

This would be a nice feature to be added on,

Do we have an existing branch on this, can't see to find it

Has anyone figured out if in this case it is possible to get resource tags for linked accounts via the monitoring account? It seems that only metric data itself when linked is obtainable via monitoring account but not resource tags, which means there's no easy way to associate resources from linked accounts.

Well there is a go function for retrieving it via oam, but I'm not very strong in go to determine how this could be implemented into the currently design.

https://docs.aws.amazon.com/sdk-for-go/api/service/oam/#OAM.ListTagsForResource

Based on the link provided in function's "see also" https://docs.aws.amazon.com/OAM/latest/APIReference/API_ListTagsForResource.html

I understood that this request only allows retrieving tags of OAM links and sinks and not all resources.

Maybe anyone has tested this one?

Can we get a bump on this? Would be very useful.

we have a in-house patch for this internally and would like to upstream this if possible.

@gtn3010 can put a PR up for review and we can go from there.

the fork: https://github.com/gtn3010/yet-another-cloudwatch-exporter

@tuananh It'd be awesome to see that PR get opened! I'd be happy to try and test it at work too :)

@tuananh It'd be awesome to see that PR get opened! I'd be happy to try and test it at work too :)

you can build from the repo i link above. in there we added linkaccount feature along with multiarch docker images as we need image for arm64 (aws graviton).

we will create a separated PR for this feature. and maybe putting multi-arch image feature in another PR.