Skip to main content

GKE usage metering (deprecated)

Warning

GKE usage metering is deprecated. We strongly recommend using GKE cost allocation to analyze your GKE costs. See FAQ for the difference between GKE usage metering and GKE cost allocation, as well as other commonly asked questions regarding switching. If you need further assistance, please open a support ticket.

Set up GKE usage metering​

For the GKE usage metering to function properly in the DoiT Platform, you need to complete all the steps below.

Note

It's no longer possible to enable GKE usage metering if you do not have it already.

Step 1 — Enable GKE Cost Analytics​

To enable GKE Cost Analytics, you need to connect your Google Cloud Organization and grant the required permissions to the DoiT Platform.

  1. Check the status of your service account.

    Log in to the DoiT Console, select the Settings icon (a gear) from the top menu bar, and then select Google Cloud.

  2. Select the GKE Cost Analytics checkbox to enable the feature.

    A screenshot highlighting the GKE Cost Analytics feature

  3. Select Update role.

    A slide-out will appear containing the gcloud commands you need to run to update your service account's role.

Step 2 — Enable the Kubernetes Engine API​

  1. Log in to the Google Cloud console, go to the project selector page, and select the project where you created the service account.

  2. Enable the Google Kubernetes Engine APIs.

See Google Cloud Docs for details.

Step 3 — Enable GKE usage metering​

In this step, you enable GKE usage metering for your GKE clusters:

  1. Create a BigQuery dataset. You can use Google Cloud console or Terraform.

  2. Configure your clusters to export their resource usage to the newly created BigQuery dataset by enabling GKE usage metering on new or existing clusters.

    You can use the following methods:

    • Google Cloud console: See Google Cloud Docs.

    • gcloud command: Run the following command to create a cluster with GKE usage metering enabled (see Google Cloud Docs for more information).

      gcloud container clusters create $CLUSTER_NAME \
      --resource-usage-bigquery-dataset $RESOURCE_USAGE_DATASET

    • Terraform Kubernetes Engine Module: Set enable_resource_consumption_export = true to enable resource consumption metering on the cluster. When enabled, a table will be created in the resource export BigQuery dataset to store resource consumption data.

Note

  • It's possible to have multiple clusters in one project writing to the same BigQuery dataset.

  • Clusters can export usage data only to the BigQuery datasets in the same project.

Step 4 — Grant the BigQuery Data Viewer role​

To import the GKE usage metering data into the DoiT Platform, the service account must have the BigQuery Data Viewer role for all the GKE usage datasets.

Datasets with missing permissions are listed in the Features widget.

You can Grant the BigQuery Data Viewer role using the Google Cloud console or the bq command-line tool.

Datasets with missing permissions

See also

For the generic information about granting access to a dataset, see Google Cloud Docs.

Analyze GKE cost​

Once you've set up GKE Cost Analytics, you'll see the following dimensions in the DoiT Cloud Analytics Reports.

  • Google Kubernetes Engine dimensions:

    • GKE Cluster: A cluster is the foundation of Google Kubernetes Engine (GKE). The Kubernetes objects that represent your containerized applications all run on top of a cluster.

    • GKE Namespace: The Kubernetes namespace where the usage is generated.

    • GKE Region: The Google Cloud region where the GKE nodes reside.

    • GKE Resource: The GKE resource in use, for example, cpu, memory, networkEgress, and storage.

  • GKE Labels: Only labels from Pod and PersistentVolumeClaim objects are tracked (see Google Cloud Docs).

    Caution

    You can't use GKE Labels enabled by GKE usage metering together with other Google Cloud labels in the same report.

You can drag a GKE chip or a GKE label chip into the Group by or Dimensions section to start understanding the real costs of running Kubernetes-based workloads on GKE. See Dimensions and labels for more information.

Limitations​

  • GKE reports only show data from the day when all the steps were completed. If you need to load historical data (i.e. backfill), please contact DoiT support team to file a backfill request.

  • If you never enabled GKE usage metering in the Google Cloud Console before setting up GKE Cost Analytics, we will not be able to show historical data in GKE reports.

  • After you complete all the steps, it usually takes around four hours for you to be able to use GKE reports in Cloud Analytics.

  • GKE reports are not compatible with some of the dimensions. For example, you will not be able to create a report that is based on the GKE Cluster field and the Zone field. You also cannot combine Google Cloud Labels and GKE Labels in the same report.

  • Expect a delay of one or two days before a new GKE label becomes available for GKE reports.

  • GKE usage metering is not applicable to Autopilot clusters (see Google Cloud Docs).

FAQ​

What's the difference between GKE usage metering and GKE cost allocation?​

GKE cost allocation provides an alternative way for getting cluster information. It is intended by Google to replace GKE usage metering.

The main difference between the two features is: GKE usage metering data is separate from the cost data in your Cloud Billing account, while GKE cost allocation data is part of it, which means, GKE cost allocation allows you to calculate cluster costs in your Cloud Billing account (see also GKE cost allocation and cluster usage metering comparison).

You may also notice differences between the data in GKE usage metering and GKE cost allocation due to the latter's higher sampling frequency.

Can I use GKE usage metering and GKE cost allocation simultaneously?​

Yes, but with constraints.

If you enable GKE cost allocation for your clusters when GKE usage metering is already enabled, in the Cloud Analytics reports:

  • Your GKE cost allocation data will be under the GKE Labels goog-k8s-cluster-name and k8s-namespace.

  • The two Google Kubernetes Engine dimensions GKE Cluster and GKE Namespace only carry GKE usage metering data.

Warning

We strongly recommend not to use the two features simultaneously. Once we stop supporting GKE usage metering in the DoiT Platform, all the reports, attributions, metrics and budgets built with GKE usage metering data will be lost. In addition, the historical data will not be available in the DoiT Platform anymore.

Do I need to pay extra for GKE cost allocation?​

While DoiT does not charge extra for GKE cost allocation, enabling GKE cost allocation can increase your storage costs in BigQuery. See Potential data volume increase.

What to expect if I switch to GKE cost allocation?​

Once you switch to GKE cost allocation:

  • Your GKE clusters cost data will be added to the cost and usage data in your Cloud Billing account.

  • The Google Kubernetes Engine dimensions GKE Namespace and GKE Cluster will carry the GKE cost allocation data.

  • The dimensions GKE Region and GRE Resource will be included in the standard dimensions Region and Resource.

  • You will lose access to the historical GKE usage metering data.

  • You will lose all the reports, attributions, metrics and budgets built with GKE usage metering data.

Last updated on