Provide "estimate" and "cost" of a cluster

[richardcase]

trafficstars

/kind feature /area clusterawsadm /priority important-longterm /triage accepted

Describe the solution you'd like

As a user of CAPA i would like to:

• get an estimate of the cost for a cluster before i create it based on the generated manifests. This is termed an estimate as it won't take into account all the resources....just the big ticket items such as EC2 instances.
• get the cost of an already created cluster. As all the resources are created we can get a more accurate figure and this is deemed the cost of the cluster.

We need to implement this with the following in mind:

• We should be able to change/choose the way that prices for a particular resource type are retrieved. Initially, lets use the AWS Pricing API and Infracost. But in the future it could be a file on disk.
• We should design with integration in mind:
  • Integration of the CLI into a pipeline, with options on the output format of the price (i.e. text, table, yaml, json)
  • Integration as a package by being imported into another application

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[richardcase]

/assign /lifecycle active

[Skarlso]

As a note ( because I was looking at my recent AWS bills :D ) the most spender was actually the NAT Gateway and network traffic. EC2 instances were all zero costs basically, because they were still in the range of my free tier.

Something to consider is we can't count network traffic. Even though the loadbalancer and gateways and such will probably take the majority of the cost. Also, we probably can't take free tier into the account either. Also, probably don't bother with spot instances. :D

~~Lucky for us though, AWS does provide a price API which should be used described here:~~ ~~https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/price-changes.html~~ Nevermind, you already wrote about this :P.

[richardcase]

Also some thinking by @bigkevmcd:

https://gist.github.com/bigkevmcd/6f6409b8acbfc54f9e37e51c0dc1d45f

[nikimanoledaki]

I agree with @Skarlso that services other than EC2 instances tend to be the costliest and should be accounted for, especially NAT Gateways, which are usually created by default for all public EKS clusters.

Note also that the Price List API only supports the following 2 regions. This should be okay for a rough estimate.

https://api.pricing.us-east-1.amazonaws.com
https://api.pricing.ap-south-1.amazonaws.com

[bigkevmcd]

@nikimanoledaki I'm not sure that's as true as you think it is...

[nikimanoledaki]

@bigkevmcd I may be wrong but they are the only regions that I could access info for through the API.

For example, for eu-north-1:

aws pricing describe-services --service-code AmazonEC2 --region eu-north-1

Could not connect to the endpoint URL: "https://api.pricing.eu-north-1.amazonaws.com/"

us-west-2:

aws pricing describe-services --service-code AmazonEC2 --region us-west-2

Could not connect to the endpoint URL: "https://api.pricing.us-west-2.amazonaws.com/"

It looks like support for other regions is available only through the console.

[Skarlso]

@bigkevmcd Do you have a picture of the NAT gateways prices in there / networking? I would be really interested for a comparison.

[bigkevmcd]

$ aws pricing get-products --service-code AmazonEKS --filters=Type=TERM_MATCH,Field=regionCode,Value=eu-west-1
{
    "PriceList": [
        "{\"product\":{\"productFamily\":\"Compute\",\"attributes\":{\"tiertype\":\"HAStandard\",\"regionCode\":\"eu-west-1\",\"servicecode\":\"AmazonEKS\",\"usagetype\":\"EU-AmazonEKS-Hours:perCluster\",\"locationType\":\"AWS Region\",\"location\":\"EU (Ireland)\",\"servicename\":\"Amazon Elastic Container Service for Kubernetes\",\"operation\":\"CreateOperation\"},\"sku\":\"HYMNGN7MKVJ8Y4HA\"},\"serviceCode\":\"AmazonEKS\",\"terms\":{\"OnDemand\":{\"HYMNGN7MKVJ8Y4HA.JRTCKXETXF\":{\"priceDimensions\":{\"HYMNGN7MKVJ8Y4HA.JRTCKXETXF.6YS6EN2CT7\":{\"unit\":\"Hours\",\"endRange\":\"Inf\",\"description\":\"Amazon EKS cluster usage in EU (Ireland)\",\"appliesTo\":[],\"rateCode\":\"HYMNGN7MKVJ8Y4HA.JRTCKXETXF.6YS6EN2CT7\",\"beginRange\":\"0\",\"pricePerUnit\":{\"USD\":\"0.1000000000\"}}},\"sku\":\"HYMNGN7MKVJ8Y4HA\",\"effectiveDate\":\"2022-10-01T00:00:00Z\",\"offerTermCode\":\"JRTCKXETXF\",\"termAttributes\":{}}}},\"version\":\"20221006173100\",\"publicationDate\":\"2022-10-06T17:31:00Z\"}",
        "{\"product\":{\"productFamily\":\"Compute\",\"attributes\":{\"regionCode\":\"eu-west-1\",\"servicecode\":\"AmazonEKS\",\"tenancy\":\"Shared\",\"usagetype\":\"EU-Fargate-vCPU-Hours:perCPU\",\"locationType\":\"AWS Region\",\"location\":\"EU (Ireland)\",\"servicename\":\"Amazon Elastic Container Service for Kubernetes\",\"cputype\":\"perCPU\",\"operation\":\"\"},\"sku\":\"WSQVUF527AQ8ZBPA\"},\"serviceCode\":\"AmazonEKS\",\"terms\":{\"OnDemand\":{\"WSQVUF527AQ8ZBPA.JRTCKXETXF\":{\"priceDimensions\":{\"WSQVUF527AQ8ZBPA.JRTCKXETXF.6YS6EN2CT7\":{\"unit\":\"hours\",\"endRange\":\"Inf\",\"description\":\"AWS Fargate - vCPU - EU (Ireland)\",\"appliesTo\":[],\"rateCode\":\"WSQVUF527AQ8ZBPA.JRTCKXETXF.6YS6EN2CT7\",\"beginRange\":\"0\",\"pricePerUnit\":{\"USD\":\"0.0404800000\"}}},\"sku\":\"WSQVUF527AQ8ZBPA\",\"effectiveDate\":\"2022-10-01T00:00:00Z\",\"offerTermCode\":\"JRTCKXETXF\",\"termAttributes\":{}}}},\"version\":\"20221006173100\",\"publicationDate\":\"2022-10-06T17:31:00Z\"}",
        "{\"product\":{\"productFamily\":\"Compute\",\"attributes\":{\"regionCode\":\"eu-west-1\",\"servicecode\":\"AmazonEKS\",\"tenancy\":\"Shared\",\"usagetype\":\"EU-Fargate-GB-Hours\",\"locationType\":\"AWS Region\",\"location\":\"EU (Ireland)\",\"servicename\":\"Amazon Elastic Container Service for Kubernetes\",\"memorytype\":\"perGB\",\"operation\":\"\"},\"sku\":\"YBF4D7Q5BJ7FVVBE\"},\"serviceCode\":\"AmazonEKS\",\"terms\":{\"OnDemand\":{\"YBF4D7Q5BJ7FVVBE.JRTCKXETXF\":{\"priceDimensions\":{\"YBF4D7Q5BJ7FVVBE.JRTCKXETXF.6YS6EN2CT7\":{\"unit\":\"hours\",\"endRange\":\"Inf\",\"description\":\"AWS Fargate - Memory  - EU (Ireland)\",\"appliesTo\":[],\"rateCode\":\"YBF4D7Q5BJ7FVVBE.JRTCKXETXF.6YS6EN2CT7\",\"beginRange\":\"0\",\"pricePerUnit\":{\"USD\":\"0.0044450000\"}}},\"sku\":\"YBF4D7Q5BJ7FVVBE\",\"effectiveDate\":\"2022-10-01T00:00:00Z\",\"offerTermCode\":\"JRTCKXETXF\",\"termAttributes\":{}}}},\"version\":\"20221006173100\",\"publicationDate\":\"2022-10-06T17:31:00Z\"}"
    ],
    "FormatVersion": "aws_v1"
}

[bigkevmcd]

@bigkevmcd Do you have a picture of the NAT gateways prices in there / networking? I would be really interested for a comparison.

@Skarlso Not quite sure what you're looking for here?

[Skarlso]

The price on the networking side. If you don't have NAT gateways, we don't have a comparison. :)

What I saw is that EC2 instances can be at a really low price, meanwhile, networking and NAT gateways ramp up a lot.

[bigkevmcd]

@Skarlso That's coming from the AWS calculator? https://calculator.aws?

[Skarlso]

Oh, sorry, I thought they are actual values. :/

[nikimanoledaki]

@bigkevmcd ah great, --filters=Type=TERM_MATCH,Field=regionCode,Value=eu-west-1 provides the per region capability. Thanks for clarifying this 😄

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[richardcase]

I think this will still be a good feature to add, so:

/remove-lifecycle stale

[richardcase]

/unassign /help

[k8s-ci-robot]

@richardcase: This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to this:

/unassign /help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.