Skip to main content

Understand EKS costs

Overview​

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service to run Kubernetes on AWS. DoiT provides an EKS cost monitoring solution that uses the OpenTelemetry Collector to collect metrics generated by kube-state-metrics (KSM) of EKS clusters running on Amazon EC2 nodes. The metrics are further integrated with AWS billing data, allowing you to break down EKS cost by Kubernetes resources such as pods, nodes, namespaces and other dimensions supported in the DoiT platform.

Kubernetes compatibility​

The OpenTelemetry agent of the DoiT EKS cost monitoring solution supports the latest release of kube-state-metrics, which interacts with Kubernetes clusters via client-go. See compatibility matrix for kube-state-metrics and client-go and compatibility matrix for client-go and Kubernetes cluster for details.

Before you begin​

  • The DoiT EKS cost monitoring solution is not available if you're on a DoiT consolidated billing account (payer account ID 561602220360, 017920819041, and 279843869311).

  • The OpenTelemetry agent relies on the AWS-generated cost allocation tag eks:cluster-name to identify EKS EC2 instances. Make sure the tag has been activated and your clusters are tagged correctly in the AWS billing data. See Activating AWS-generated tags cost allocation tags.

  • Currently, it's not possible to use existing OpenTelemetry Collector and kube-state-metrics on your clusters for the DoiT EKS Lens. You must deploy the agent following the instructions on this page. The components will be deployed in the doit-eks-metrics namespace.

EKS clusters in the DoiT console​

View EKS clusters​

DoiT performs a daily scan on AWS billing data to uncover EKS clusters in your account. It assumes that the AWS-generated cost allocation tag eks:cluster-name has been applied to your EKS EC2 instances.

The identified clusters are listed on the EKS clusters page, which is also where you check the EKS costs monitoring settings and take actions if necessary.

Cost By Service report widget

To access the EKS clusters page in the DoiT console:

You can perform different tasks in accordance with the status of a cluster:

  • Active: The cluster is already connected to the DoiT platform. You can download its deployment configuration file by selecting Download yaml from the three dots menu at the rightmost end of the cluster entry row.

  • Not started: The cluster is not connected to the DoiT platform yet. Select Connect to start the connecting process.

  • Deleted: The CloudFormation stack that allows DoiT to access the cluster has been removed. Select Fix to re-connect the cluster or download the deployment configuration file via the three dots menu at the rightmost end of the cluster entry row.

Connect an EKS cluster​

To analyze the usage and cost of resources in an EKS cluster, you need to connect the cluster to the DoiT platform. You can choose either Terraform or CloudFormation (with kubectl or Helm) for the deployment. Be aware that:

  • Clusters in the same account or region must be deployed using the same deployment method.

  • It may take up to 24 hours before your data appear in DoiT Cloud Analytics.

  • The DEPLOYMENT-ID variable in Terraform or Helm deployment is provided by DoiT. It's not possible to automate the DEPLOYMENT-ID connection in the current implementation.

Required permissions​

To connect an EKS cluster to the DoiT platform, you need the following permissions:

See also

Amazon EKS User Guide: Allowing users to access your cluster

Terraform deployment​

  1. In the DoiT console, select Terraform as the deployment method.

  2. Clone the DoiT terraform-eks-lens repository for the account/region.

    git clone https://github.com/doitintl/terraform-eks-lens.git eks-lens-ACCOUNT-REGION

    cd eks-lens-ACCOUNT-REGION

  3. Sign in to the Amazon EKS console, select your cluster on the Clusters page.

  4. In the Details section on the Overview tab, copy the value of the OpenID Connect provider URL and then paste it in the DoiT console to download the Terraform configuration file, CLUSTERNAME.tf, for your cluster. Save the downloaded file in the current Terraform directory.

  5. Create a new file named CLUSTERNAME_provider.tf, copy the code snippet in the DoiT console and modify it to set up your Terraform Kubernetes provider.

  6. Copy the code snippet in the DoiT console and modify it to set up your AWS provider in the aws_provider.tf file.

  7. Run the following Terraform commands in sequence:

    • The terraform init command initializes a working directory containing Terraform configuration files.
    • The terraform plan command creates an execution plan that allows you to preview the changes that Terraform plans to make to your infrastructure.
    • The terraform apply command executes the actions proposed in the Terraform plan.

  8. In the DoiT console, select Finish to complete the deployment.

If successful, the status of cluster shows Active on the EKS clusters page.

Troubleshooting​

If you've successfully executed the Terraform commands but the state of your cluster still shows Not started, try the following:

  1. Open your Terraform configuration file CLUSTERNAME.tf, find the curl command in the section of null_resource and deploy_cluster.

  2. Run the curl command to send a request to https://console.doit.com/webhooks/v1/eks-metrics/terraform-validate with the correct parameters.

CloudFormation deployment​

The CloudFormation deployment process consists of two steps.

Step 1: Add permission​

In this step, you create a CloudFormation stack using the DoiT EKS onboarding template.

  1. In the DoiT console, select CloudFormation as the deployment method, click Next, and then select Open CloudFormation Stack.

  2. In the AWS CloudFormation console, review the pre-populated fields, and then create a stack using the DoiT template.

  3. Select the checkbox at the bottom of the page to acknowledge that AWS CloudFormation might create IAM resources with custom names.

  4. Create the stack.

  5. Once the stack is created, navigate back to the DoiT console. You should see a confirmation message that says Permission successfully added. Select Next to proceed.

Step 2: Connect and validate​

In this step, you use an auto-generated Kubernetes Deployment file or an EKS Lens Helm chart to install the required components on your Kubernetes clusters.

Using kubectl​

  1. Download the deployment YAML file, if you haven't done so in the previous step.

  2. Open AWS CloudShell in the AWS Management Console. Upload the deployment YAML file.

  3. In the DoiT console, copy the command kubectl apply -f DEPLOYMENT_YAML_FILE. It is used to update the cluster configuration.

  4. Paste the command in the AWS CloudShell and run it to update the cluster configuration.

The deployment file creates two service accounts in the namespace doit-eks-metrics:

  1. In the DoiT console, select Check to validate the connection. If successful, the status of cluster shows Active on the EKS clusters page.

Using Helm​

  1. Install Helm on your local system.

  2. Copy the Helm commands shown in the DoiT console and run them in sequence.

    • The helm repo add command adds the chart repository to your local helm installation.
    • The helm template command renders chart template locally.
    • The helm upgrade --install command installs the doit-eks-lens chart with its specific kube-state-metrics deployment.

    Refer to EKS Lens Helm chart for detailed instructions.

  3. In the DoiT console, select Check to validate the connection. If successful, the status of cluster shows Active on the EKS clusters page.

Resource management for OpenTelemetry Collector Pods​

EKS Lens Helm chart version 1.0.3 introduced support for the OpenTelemetry Collector's Memory Limiter Processor to avoid issues with memory usage on the collector.

You'll find the configuration options in the values.yaml file in the chart. You can adjust the values in accordance with your usage.

  memory_limiter:
    check_interval: 1s
    limit_percentage: 70
    spike_limit_percentage: 30

Where:

  • check_interval: Time between measurements of memory usage.

  • limit_percentage: Maximum amount of total memory that can be allocated by the process heap. This option is supported on Linux systems with cgroups and it's intended for dynamic platforms like Docker.

  • spike_limit_percentage: Maximum spike expected between the measurements of memory usage. The value must be less than limit_percentage. This option is intended to be used only with limit_percentage.

See also Resource Management for Pods and Containers.

Multiple EKS clusters​

If you have multiple EKS clusters, you must create a new CloudFormation stack for each cluster because some AWS resources are deployed at the cluster level.

The S3 bucket should be created only once when onboarding the first cluster in each account/region. If the stack creation for a later cluster fails because of the existing S3 bucket, set the CreateBucket parameter to false; you should not change the bucket name.

Analyzing EKS cost​

DoiT Cloud Analytics integrates EKS cluster metrics with AWS billing data. It allows you to break down EKS cost by Kubernetes resources and other Dimensions and labels supported in the DoiT platform.

To build and run Cloud Analytics reports, you need the Cloud Analytics permission in the DoiT console.

EKS labels​

EKS labels are assigned at the Pods level. You can use EKS labels to track Kubernetes resources usage, analyze project cost, and identify cost optimization opportunities.

In the DoiT Cloud Analytics Reports, EKS labels are grouped under the EKS Labels section, which consists of two categories:

  • Kubernetes metadata: Data that helps uniquely identify a Kubernetes object, including EKS billing_source, EKS cluster name, EKS deployment_kind EKS pod, EKS uid, EKS pod_owner_name, and EKS nodegroup name.

  • Custom labels: Labels defined and assigned from your side. These labels appear with a prefix label_ in the DoiT console, for example, label_k8s_app.

DoiT Cloud Analytics integrates EKS metrics with AWS billing data, which means you can combine EKS labels with AWS cost allocation tags assigned to other AWS resources in the same report.

Example reports​

The DoiT console provides a pre-built EKS Lens dashboard with report widgets that highlight various aspects of your Amazon EKS spend. You can also build your own reports using EKS labels and other dimensions. Below are some examples.

Cost breakdown by namespace and pod owner​

This example breaks down the cost of the EKS cluster eks-cluster-1 in the last month by namepace and pod owner.

Cost By Service report widget

Cost breakdown by cost type and SKU​

This example further breaks down the EKS cluster cost by cost type and SKU.

Cost By Service report widget

Cost attribution across services​

In this example, the organization uses the label Team to distinguish AWS resources requested by different entities. To display the costs of EKS and other AWS services in the same report, you can include both the standard label Team and the EKS custom label EKS label_team. (Alternatively, you can create an attribution that contains the standard label and its corresponding EKS custom label, and then use attributions in your report.)

Cost By Service report widget

EKS clusters offboarding​

To offboard an EKS cluster from the DoiT platform:

  • Cluster connected via Terraform: Run the terraform destroy command to destroy the full stack based on your CLUSTERNAME.tf file, or use the -target option to destroy single resources, for example, terraform destroy -target RESOURCE_TYPE.NAME.

  • Cluster connected via CloudFormation with helm:

    1. Delete the CloudFormation stack of the cluster from your AWS account. See Deleting a stack.

    2. Run the helm uninstall doit-eks-lens command to delete the agent (OpenTelemetry Collector) from Kubernetes.

  • Clusters connected via CloudFormation with kubectl:

    1. Delete the CloudFormation stack of the cluster from your AWS account. See Deleting a stack.

    2. Run the kubectl delete -f DEPLOYMENT_YAML_FILE command from the AWS CloudShell to delete the agent (OpenTelemetry Collector) configuration.

To remove multiple clusters, repeat the steps above for each one.

FAQ​

What's the difference between EKS Lens, Kubecost, and OpenCost?​

While Kubecost and OpenCost provide valuable Kubernetes cost management capabilities, DoiT EKS Lens is specifically optimized for AWS environments, offering a more integrated approach to managing EKS costs and performance:

  • EKS Lens is designed with a deep understanding of AWS services and their integration with EKS, providing insights and analytics that are closely aligned with AWS billing and management practices.

  • EKS Lens focuses on EKS-specific metrics, offering detailed insights into cluster performance, cost allocation, and optimization opportunities within the AWS ecosystem.

  • EKS Lens leverages AWS-native services for enhanced security, compliance, and performance monitoring, ensuring a cohesive and secure environment for your EKS clusters.

  • With EKS Lens, you can easily customize dashboards and reports in the DoiT console, enabling effective cost management and optimization strategies.

The table below lists the main differences between DoiT EKS Lens, Amazon EKS optimized Kubecost custom bundle, and OpenCost.

FeatureDoiT EKS LensAmazon EKS optimized Kubecost custom bundleOpenCost
DescriptionA DoiT Cloud Analytics feature that provides comprehensive visibility into EKS spend.A customized version of Kubecost by AWS, including a subset of commercial features.A vendor-neutral open source project for measuring and allocating cloud infrastructure and container costs in real time.
Number of clustersUnlimited (unified multi-cluster view)Unlimited (unified multi-cluster view)Unlimited (no unified view)
Number of labels for podsUnlimitedUnlimitedUnlimited
EKS on FargateLimited support (for Amazon EKS on Fargate, DoiT Cloud Analytics uses metrics from AWS billing data but not metrics generated by kube-state-metrics).Kubecost tracks costs for Amazon EKS on Fargate with lower accuracy than with EKS on EC2.No support for Amazon EKS on Fargate.
Deployment
  • Terraform deployment: using DoiT terraform-eks-lens repository; bundled with OpenTelemetry Collector and kube-state-metrics dependencies.
  • CloudFormation deployment: using CloudFormation stack and kubectl or DoiT EKS Lens Helm chart. Bundled with OpenTelemetry Collector and kube-state-metrics dependencies.
Deployed using Helm. Bundled with Prometheus and Grafana dependencies.Deployed as a pod. Prometheus and kube-state-metrics dependencies are managed separately.
Also provides a community-supported Helm chart.
Metrics retentionSubject to your contract with DoiT.Unlimited historical data for database retention; 15 days for the Kubecost API retention (ETL).Limited by Prometheus environment.
Cloud billing integrationCustom pricing support for AWS (including multiple clusters and multiple accounts).Core billing integrations with AWS EKS.
SupportSubject to your contract with DoiT.Subject to your AWS support agreements.Built and supported by community users.
See also

DoiT uses the AWS-generated cost allocation tag eks:cluster-name to identify EKS EC2 instances. Make sure your clusters are tagged correctly in the AWS billing data. See Activating AWS-generated tags cost allocation tags and AWS cost allocation tags in DoiT platform.

In addition, the DoiT platform scans the billing data on a daily basis. If your clusters are newly created (within 72 hours) or have no spend, then the DoiT platform will not be able to detect them.

Interactive demo​

Try out our interactive demo for a hands-on walk-through experience.

If the demo doesn't display properly, try expanding your browser window or opening the demo in a new tab.

Last updated on