Skip to main content

Understand EKS costs

DoiT Cloud Navigator: Enhanced and above

Overview

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service to run Kubernetes on AWS. DoiT Cloud Analytics provides an EKS cost monitoring solution that uses the OpenTelemetry Collector to collect metrics generated by kube-state-metrics (KSM) of EKS clusters running on Amazon EC2 nodes. The metrics are further integrated with AWS billing data, allowing you to break down EKS cost by Kubernetes resources such as pods, nodes, namespaces and other dimensions supported in the DoiT Platform.

Kubernetes compatibility

The DoiT EKS cost monitoring solution supports the latest release of kube-state-metrics, which interacts with Kubernetes clusters via client-go. See compatibility matrix for kube-state-metrics and client-go and compatibility matrix for client-go and Kubernetes cluster for details.

Note

The DoiT EKS cost monitoring solution is not available if you're on a DoiT consolidated billing account (payer account ID 561602220360, 017920819041, and 279843869311).

EKS clusters in the DoiT console

View EKS clusters

The DoiT Platform performs a daily scan on AWS billing data to uncover EKS clusters in your account. It assumes that the AWS-generated eks:cluster-name tag are applied to your EKS EC2 instances.

To view the EKS clusters:

  1. Sign in to the DoiT console, select Dashboards from the top navigation bar.

  2. In the Quick Links section, select Connect your EKS clusters.

    Identified clusters are listed on the EKS clusters page. (There might be a delay for newly created clusters to appear in the DoiT console.)

    Cost By Service report widget

    You can perform different tasks in accordance with the status of a cluster:

    • Active: The cluster is already connected to the DoiT Platform. You can download its deployment configuration file by selecting Download yaml from the three dots menu at the rightmost end of the cluster entry row.

    • Not started: The cluster is not connected to the DoiT Platform yet. You can select Connect to start the connecting process.

    • Deleted: The CloudFormation stack that allows DoiT to access the cluster has been removed. You can select Fix to re-connect the cluster or download the deployment configuration file via the three dots menu at the rightmost end of the cluster entry row.

Connect an EKS cluster

To analyze the usage and cost of resources in an EKS cluster, you need to connect the cluster to the DoiT Platform. You can choose either Terraform or CloudFormation (with kubectl or Helm) for the deployment. Be aware that:

  • Clusters in the same account or region must be deployed using the same deployment method.

  • It may take up to 24 hours after connecting for your data to be available in DoiT Cloud Analytics.

Required permissions

To connect an EKS cluster to the DoiT Platform, you need the following permissions:

See also

Terraform deployment

  1. In the DoiT console, select Terraform as the deployment method.

  2. Clone the DoiT terraform-eks-lens repository for the account/region.

    git clone https://github.com/doitintl/terraform-eks-lens.git eks-lens-{account}-{region}

    cd eks-lens-{account}-{region}
  3. Sign in to the Amazon EKS console, select your cluster on the Clusters page.

  4. In the Details section on the Overview tab, copy the value of the OpenID Connect provider URL and then paste it in the DoiT console to download the Terraform configuration file, {clustername}.tf, for your cluster. Save the downloaded file in the current Terraform directory.

  5. Create a new file named {clustername}_provider.tf, copy the code snippet in the DoiT console and modify it to set up your Terraform Kubernetes provider.

  6. Copy the code snippet in the DoiT console and modify it to set up your AWS provider in the aws_provider.tf file.

  7. Run the following Terraform commands in sequence:

    • The terraform init command initializes a working directory containing Terraform configuration files.
    • The terraform plan command creates an execution plan that allows you to preview the changes that Terraform plans to make to your infrastructure.
    • The terraform apply command executes the actions proposed in the Terraform plan.
  8. In the DoiT console, select Finish to complete the deployment.

If successful, the status of cluster shows Active on the EKS clusters page.

Troubleshooting

If you've successfully executed the Terraform commands but the state of your cluster still shows Not started, try the following:

  1. Open your Terraform configuration file {clustername}.tf, find the curl command in the section of null_resource and deploy_cluster.

  2. Run the curl command to send a request to https://console.doit.com/webhooks/v1/eks-metrics/terraform-validate with the correct parameters.

CloudFormation deployment

The CloudFormation deployment process consists of two steps.

Step 1: Add permission

In this step, you create a CloudFormation stack using the DoiT EKS onboarding template.

  1. In the DoiT console, select CloudFormation as the deployment method, click Next, and then select Open CloudFormation Stack.

  2. In the AWS CloudFormation console, review the pre-populated fields, and then create a stack using the DoiT template.

  3. Select the checkbox at the bottom of the page to acknowledge that AWS CloudFormation might create IAM resources with custom names.

  4. Create the stack.

  5. Once the stack is created, navigate back to the DoiT console. You should see a confirmation message that says Permission successfully added. Select Next to proceed.

Step 2: Connect and validate

In this step, you use kubectl or Helm to install the required components on your Kubernetes clusters using an auto-generated Kubernetes Deployment file or an EKS Lens Helm chart.

Using kubectl

  1. Download the deployment YAML file, if you haven't done so in the previous step.

  2. Open AWS CloudShell in the AWS Management Console. Upload the deployment YAML file.

  3. In the DoiT console, copy the command kubectl apply -f {deployment_yaml_file}. It is used to update the cluster configuration.

  4. Paste the command in the AWS CloudShell and run it to update the cluster configuration.

    The deployment file creates two service accounts in the namespace doit-eks-metrics:

  5. In the DoiT console, select Check to validate the connection. If successful, the status of cluster shows Active on the EKS clusters page.

Using Helm

  1. Install Helm on your local system.

  2. Copy the Helm commands shown in the DoiT console and run them in sequence.

    • The helm repo add command adds the chart repository to your local helm installation.
    • The helm template command renders chart template locally.
    • The helm upgrade --install command installs the doit-eks-lens chart with its specific kube-state-metrics deployment.
  3. In the DoiT console, select Check to validate the connection. If successful, the status of cluster shows Active on the EKS clusters page.

Multiple EKS clusters

If you have multiple EKS clusters, you need to create a new CloudFormation stack for each cluster as some AWS resources are deployed at the cluster level.

The S3 bucket should be created only once when onboarding the first cluster in each account/region. If the stack creation for a latter cluster fails because of the existing S3 bucket, you can modify the CreateBucket parameter to false; you should not change the bucket name.

Remove EKS clusters

To remove (decommission) an EKS cluster from the DoiT Platform:

  1. Delete the CloudFormation stack of the cluster from your AWS account. See Deleting a stack.

  2. Delete the agent (OpenTelemetry Collector) configuration.

    You can run the command kubectl delete -f {deployment_yaml_file} from the AWS CloudShell to delete the agent.

To remove multiple clusters, repeat the two steps above for each one.

Analyze EKS cost

DoiT Cloud Analytics integrates EKS cluster metrics with AWS billing data. It allows you to break down EKS cost by Kubernetes resources and other Dimensions and labels supported in the DoiT Platform.

To build and run Cloud Analytics reports, you need the Cloud Analytics permission in the DoiT console.

EKS labels

EKS labels are assigned at the Pods level. You can use EKS labels to track Kubernetes resources usage, analyze project cost, and identify cost optimization opportunities.

In the DoiT Cloud Analytics Reports, EKS labels are grouped under the EKS Labels section, which consists of two categories:

  • Kubernetes metadata: Data that helps uniquely identify a Kubernetes object, including EKS billing_source, EKS cluster name, EKS deployment_kind EKS pod, EKS uid, EKS pod_owner_name, and EKS nodegroup name.

  • Custom labels: Labels defined and assigned from your side. These labels appear with a prefix label_ in the DoiT console, for example, label_k8s_app.

DoiT Cloud Analytics integrates EKS metrics with AWS billing data, which means you can combine EKS labels with AWS cost allocation tags assigned to other AWS resources in the same report.

Example reports

The DoiT console provides a pre-built EKS Lens dashboard with report widgets that highlight various aspects of your Amazon EKS spend. You can also build your own reports using EKS labels and other dimensions. Below are some examples.

Cost breakdown by namespace and pod owner

This example breaks down the cost of the EKS cluster eks-cluster-1 in the last month by namepace and pod owner.

Cost By Service report widget

Cost breakdown by cost type and SKU

This example further breaks down the EKS cluster cost by cost type and SKU.

Cost By Service report widget

Cost attribution across services

In this example, the organization uses the label Team to distinguish AWS resources requested by different entities. To display the costs of EKS and other AWS services in the same report, you can include both the standard label Team and the EKS custom label EKS label_team. (Alternatively, you can create an attribution that contains the standard label and its corresponding EKS custom label, and then use attributions in your report.)

Cost By Service report widget

FAQ

What's the difference between EKS Lens, Kubecost, and OpenCost?

While Kubecost and OpenCost provide valuable Kubernetes cost management capabilities, DoiT EKS Lens is specifically optimized for AWS environments, offering a more integrated approach to managing EKS costs and performance:

  • EKS Lens is designed with a deep understanding of AWS services and their integration with EKS, providing insights and analytics that are closely aligned with AWS billing and management practices.

  • EKS Lens focuses on EKS-specific metrics, offering detailed insights into cluster performance, cost allocation, and optimization opportunities within the AWS ecosystem.

  • EKS Lens leverages AWS-native services for enhanced security, compliance, and performance monitoring, ensuring a cohesive and secure environment for your EKS clusters.

  • With EKS Lens, you can easily customize dashboards and reports in the DoiT console, enabling effective cost management and optimization strategies.

The table below lists the main differences between DoiT EKS Lens, Amazon EKS optimized Kubecost custom bundle, and OpenCost.

FeatureDoiT EKS LensAmazon EKS optimized Kubecost custom bundleOpenCost
DescriptionA DoiT Cloud Analytics feature that provides comprehensive visibility into EKS spend.A customized version of Kubecost by AWS, including a subset of commercial features.A vendor-neutral open source project for measuring and allocating cloud infrastructure and container costs in real time.
Number of clustersUnlimited (unified multi-cluster view)Unlimited (unified multi-cluster view)Unlimited (no unified view)
Number of labels for podsUnlimitedUnlimitedUnlimited
EKS on FargateLimited support (for Amazon EKS on Fargate, DoiT Cloud Analytics uses metrics from AWS billing data but not metrics generated by kube-state-metrics).Kubecost tracks costs for Amazon EKS on Fargate with lower accuracy than with EKS on EC2.No support for Amazon EKS on Fargate.
Deployment
  • Terraform deployment: using DoiT terraform-eks-lens repository; bundled with OpenTelemetry Collector and kube-state-metrics dependencies.
  • CloudFormation deployment: using CloudFormation stack and kubectl or DoiT EKS Lens Helm chart. Bundled with OpenTelemetry Collector and kube-state-metrics dependencies.
Deployed using Helm. Bundled with Prometheus and Grafana dependencies.Deployed as a pod. Prometheus and kube-state-metrics dependencies are managed separately.
Also provides a community-supported Helm chart.
Metrics retentionSubject to your contract with DoiT.Unlimited historical data for database retention; 15 days for the Kubecost API retention (ETL).Limited by Prometheus environment.
Cloud billing integrationCustom pricing support for AWS (including multiple clusters and multiple accounts).Core billing integrations with AWS EKS.
SupportSubject to your contract with DoiT.Subject to your AWS support agreements.Built and supported by community users.
See also

Can I use existing OpenTelemetry Collector and kube-state-metrics on my clusters for DoiT EKS Lens?

Currently, no. You need to deploy the two components following the instructions to Connect an EKS cluster. The components will be deployed in the namespace doit-eks-metrics.

The DoiT Platform uses the AWS-generated eks:cluster-name tag to identify EKS EC2 instances. Make sure your clusters are tagged correctly in the AWS billing data. See AWS cost allocation tags for how the DoiT Platform handles the tags.

In addition, the DoiT Platform scans the billing data on a daily basis. If your clusters are newly created (within 72 hours) or have no spend, then the DoiT Platform will not be able to detect them.

Interactive demo

Try out our interactive demo for a hands-on walk-through experience.

If the demo doesn't display properly, try expanding your browser window or opening the demo in a new tab.