opencost icon indicating copy to clipboard operation
opencost copied to clipboard

handle aws generated tags

Open Sean-Holcomb opened this issue 6 months ago • 1 comments

What does this PR change?

This PR updates AWS CUR integration to ingest AWS generated tags in addition to user generated tags. To do this columns with the AWS label prefix are gathered in a separate slice. With this separate list of columns the prefix can be properly handled and saved to cloud cost labels. This PR includes this addition for both Athena and S3 integrations.

Does this PR relate to any other PRs?

How will this PR impact users?

  • Users will now be able to see AWS generated labels in Cloud Costs Screenshot 2024-08-29 at 1 08 42 PM

Does this PR address any GitHub or Zendesk issues?

  • Closes ENG-150

How was this PR tested?

  • This PR was tested by running cloud cost and aggregator locally and ensuring that AWS label aws_eks_cluster_name was being populated in cloud cost. curl --location --globoff 'http://localhost:9004/cloudCost?window=yesterday&filter=label[aws_eks_cluster_name]%3A%22dev-1%22&aggregate=label%3Aaws_eks_cluster_name'
      "cloudCosts": {
                    "dev-1": {
                        "properties": {
                            "labels": {
                                "aws_created_by": "AssumedRole:AROAUKXXWPGDYLPOEB2GE:AutoScaling",
                                "aws_eks_cluster_name": "dev-1",
                                "eks_cluster_name": "dev-1"
                            }
                        },
                        "window": {
                            "start": "2024-08-27T00:00:00Z",
                            "end": "2024-08-28T00:00:00Z"
                        },
                        "listCost": {
                            "cost": 28.5624505521,
                            "kubernetesPercent": 1
                        },
                        "netCost": {
                            "cost": 28.5624505521,
                            "kubernetesPercent": 1
                        },
                        "amortizedNetCost": {
                            "cost": 28.5624505521,
                            "kubernetesPercent": 1
                        },
                        "invoicedCost": {
                            "cost": 28.5624505521,
                            "kubernetesPercent": 1
                        },
                        "amortizedCost": {
                            "cost": 28.5624505521,
                            "kubernetesPercent": 1
                        }
                    }
                },
                "window": {
                    "start": "2024-08-27T00:00:00Z",
                    "end": "2024-08-28T00:00:00Z"
                },
                "aggregationProperties": [
                    "label:aws_eks_cluster_name"
                ]
            }

Because this PR deals with integrations it will need to by pass unit testing gates

Does this PR require changes to documentation?

Have you labeled this PR and its corresponding Issue as "next release" if it should be part of the next OpenCost release? If not, why not?

Sean-Holcomb avatar Aug 29 '24 20:08 Sean-Holcomb