Skip to main content

Connect EKS clusters

To analyze the usage and cost of resources in an EKS cluster, you need to connect the cluster to the DoiT platform. You can choose either Terraform or CloudFormation (with kubectl or Helm) for the deployment. Be aware that:

  • Clusters in the same account or region must be deployed using the same deployment method.

  • It may take up to 24 hours before your data appear in DoiT Cloud Analytics.

  • The DEPLOYMENT-ID variable in Terraform and Helm deployment is provided by DoiT. It's not possible to automate the DEPLOYMENT-ID connection in the current implementation.

Required permissions​

To connect an EKS cluster to the DoiT platform, you need the following permissions:

See also

Amazon EKS User Guide: Allowing users to access your cluster

Terraform deployment​

  1. In the DoiT console, select Terraform as the deployment method.

  2. Clone the DoiT terraform-eks-lens repository for the account/region.

    git clone https://github.com/doitintl/terraform-eks-lens.git eks-lens-ACCOUNT-REGION

    cd eks-lens-ACCOUNT-REGION

  3. Sign in to the Amazon EKS console, select your cluster on the Clusters page.

  4. In the Details section on the Overview tab, copy the value of the OpenID Connect provider URL and then paste it in the DoiT console to download the Terraform configuration file, CLUSTERNAME.tf, for your cluster. Save the downloaded file in the current Terraform directory.

  5. Create a new file named CLUSTERNAME_provider.tf, copy the code snippet in the DoiT console and modify it to set up your Terraform Kubernetes provider.

  6. Copy the code snippet in the DoiT console and modify it to set up your AWS provider in the aws_provider.tf file.

  7. Run the following Terraform commands in sequence:

    • terraform init: Initializes a working directory containing Terraform configuration files.
    • terraform plan: Creates an execution plan that allows you to preview the changes that Terraform plans to make to your infrastructure.
    • terraform apply: Executes the actions proposed in the Terraform plan.

  8. In the DoiT console, select Finish to complete the deployment.

If successful, the cluster status shows Active on the EKS clusters page.

Troubleshooting​

If you've successfully executed the Terraform commands but the state of your cluster still shows Not started, try the following:

  1. Open your Terraform configuration file CLUSTERNAME.tf, find the curl command in the section of null_resource and deploy_cluster.

  2. Run the curl command to send a request to https://console.doit.com/webhooks/v1/eks-metrics/terraform-validate with the correct parameters.

CloudFormation deployment​

The CloudFormation deployment process consists of two steps.

Step 1: Add permission​

In this step, you create a CloudFormation stack using the DoiT EKS onboarding template.

  1. In the DoiT console, select CloudFormation as the deployment method, click Next, and then select Open CloudFormation Stack.

  2. In the AWS CloudFormation console, review the pre-populated fields, and then create a stack using the DoiT template.

  3. Select the checkbox at the bottom of the page to acknowledge that AWS CloudFormation might create IAM resources with custom names.

  4. Create the stack.

  5. Once the stack is created, navigate back to the DoiT console. You should see a confirmation message that says Permission successfully added. Select Next to proceed.

Step 2: Connect and validate​

In this step, you install the required components on your Kubernetes clusters, using an auto-generated Kubernetes Deployment file or an EKS Lens Helm chart.

Using kubectl​

  1. Download the deployment YAML file if you haven't done so in the previous step.

  2. Open AWS CloudShell in the AWS Management Console. Upload the deployment YAML file.

  3. In the DoiT console, copy the command kubectl apply -f DEPLOYMENT_YAML_FILE.

  4. Paste the command in the AWS CloudShell and run it to update the cluster configuration.

The deployment file creates two service accounts in the namespace doit-eks-metrics:

  1. In the DoiT console, select Check to validate the connection. If successful, the status of cluster shows Active on the EKS clusters page.

Using Helm​

  1. Install Helm on your local system.

  2. Copy the Helm commands provided in the DoiT console and run them in sequence.

    • The helm repo add command adds the chart repository to your local helm installation.
    • The helm template command renders the chart template locally.
    • The helm upgrade --install command installs the doit-eks-lens chart with its specific kube-state-metrics deployment.

    Refer to EKS Lens Helm chart for detailed instructions.

  3. In the DoiT console, select Check to validate the connection. If successful, the cluster's status will show Active on the EKS clusters page.

Multiple EKS clusters​

If you have multiple EKS clusters, you must create a new CloudFormation stack for each cluster because some AWS resources are deployed at the cluster level.

The S3 bucket is created per account/region. It should be created only once when onboarding the first cluster in each account/region; all other clusters in the same account/region will use the same S3 bucket.

If the stack creation for a later cluster fails because of the existing S3 bucket, set the CreateBucket parameter to false. You should not change the bucket name.

Resource management for OpenTelemetry Collector Pods​

DoiT EKS cost monitoring solution supports the OpenTelemetry Collector's Memory Limiter Processor to help you mitigate memory usage issues with the OpenTelemetry Collector.

Terraform EKS Lens​

With terraform-eks-lens, you can set environment variables for the OpenTelemetry Collector using the otel_env variable.


  # If you need to set environment variables for the OpenTelemetry Collector, you can do so by setting the `otel_env` variable:
  # otel_env = {
  #   "GOMEMLIMIT"  = "2750MiB" # set the memory limit for the OpenTelemetry Collector
  # }

  # We recommend to read the OpenTelemetry Collector documentation to understand the memory limiter processor configuration: https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md#best-practices
  # If you want to customize the memory limiter processor for the OpenTelemetry Collector, you can do so by setting the `otel_memory_limiter` variable:
  # otel_memory_limiter = {
  #   check_interval         = "1s"
  #   limit_percentage       = 70
  #   spike_limit_percentage = 30
  # }

  # If you want to customize the resources for the OpenTelemetry Collector container, you can do so by setting the `otel_resources` variable:
  # otel_resources = {
  #   requests = {
  #     cpu    = "100m"
  #     memory = "256Mi"
  #   }
  #   limits = {
  #     cpu    = "100m"
  #     memory = "256Mi"
  #   }
  # }

EKS Lens Helm chart​

With EKS Lens Helm chart, you can adjust memory limiter settings using variables in the values.yaml file.


collector:
  otelcol:
    replicas: 1
    image:
      repository: otel/opentelemetry-collector-contrib
      tag: 0.83.0
    kubeStateMetrics:
      endpoint: "kube-state-metrics:8080"
    env:
  # - name: "GOMEMLIMIT"
  #   value: "2750MiB"
  ## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
  tolerations: []
  ## Ref: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
  resources: {}
  ## Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md
  memory_limiter:
    check_interval: 1s
    limit_percentage: 70
    spike_limit_percentage: 30

Where:

  • check_interval: Time between measurements of memory usage.

  • limit_percentage: Maximum amount of total memory that can be allocated by the process heap. This option is supported on Linux systems with cgroups and it's intended for dynamic platforms like Docker.

  • spike_limit_percentage: Maximum spike expected between the memory usage measurements. The value must be less than limit_percentage. This option is intended to be used only with limit_percentage.

See also Resource Management for Pods and Containers.

EKS clusters offboarding​

To offboard an EKS cluster from the DoiT platform:

  • Cluster connected via Terraform: Run the terraform destroy command to destroy the full stack based on your CLUSTERNAME.tf file, or use the -target option to destroy single resources, for example, terraform destroy -target RESOURCE_TYPE.NAME.

  • Cluster connected via CloudFormation with Helm:

    1. Delete the CloudFormation stack of the cluster from your AWS account. See Deleting a stack.

    2. Run the helm uninstall doit-eks-lens command to delete the agent (OpenTelemetry Collector) from Kubernetes.

  • Clusters connected via CloudFormation with kubectl:

    1. Delete the CloudFormation stack of the cluster from your AWS account. See Deleting a stack.

    2. Run the kubectl delete -f DEPLOYMENT_YAML_FILE command from the AWS CloudShell to delete the agent (OpenTelemetry Collector) configuration.

To remove multiple clusters, repeat the steps above for each one.

Interactive demo​

Try out our interactive demo for a hands-on walk-through experience.

If the demo doesn't display properly, try expanding your browser window or opening the demo in a new tab.

Last updated on