Skip to main content

GKE Cost Analytics

GKE Cost Analytics in the CMP helps you demystify your Google Kubernetes Engine (GKE) spend.

Overview

With real-time visibility into GKE spend, you'll be able to:

  • Accurately track the total cost of ownership (TCO) of Kubernetes-based workloads across teams or product lines.

  • Plan better with realistic Kubernetes-based budgets.

  • Forecast and identify trends across Kubernetes primitives in one click.

info

GKE Cost Analytics is not applicable to Autopilot clusters, which do not support usage metering (see Google Cloud Docs).

Set up GKE Cost Analytics

note

For the GKE Cost Analytics feature to function properly, you need to complete all the steps below.

Step 1 Enable GKE Cost Analytics

To enable GKE Cost Analytics, you need to connect your Google Cloud Organization and grant the required permissions to the CMP.

  1. To check the status of your service account, log in to the CMP, select the Settings icon (a gear) from the top menu bar, and then select Google Cloud.

  2. Select the GKE Cost Analytics checkbox to enable the feature.

  3. Select Update role.

    A slide-out will appear containing the gcloud commands you need to run to update your service account's role.

Step 2 Enable the Kubernetes Engine API

  1. In the Google Cloud console, on the project selector page, select the project where you created the service account.

  2. Enable the Google Kubernetes Engine APIs.

See Google Cloud Docs for details.

Step 3 Enable GKE usage metering

In this step, you enable GKE usage metering for your GKE clusters:

  1. Create a BigQuery dataset. You can use Google Cloud console or Terraform.

  2. Configure your clusters to export their resource usage to the newly created BigQuery dataset by enabling GKE usage metering on new or existing clusters.

    You can use the following methods:

    • Google Cloud console: See Google Cloud Docs.

    • gcloud command: Run the following command to create a cluster with GKE usage metering enabled (see Google Cloud Docs for more information).

      gcloud container clusters create $CLUSTER_NAME \
      --resource-usage-bigquery-dataset $RESOURCE_USAGE_DATASET
    • Terraform Kubernetes Engine Module: Set enable_resource_consumption_export = true to enable resource consumption metering on the cluster. When enabled, a table will be created in the resource export BigQuery dataset to store resource consumption data.

note
  • It's possible to have multiple clusters in one project writing to the same BigQuery dataset.

  • Clusters can export usage data only to the BigQuery datasets in the same project.

Step 4 Grant the BigQuery Data Viewer role

You can grant the BigQuery Data Viewer role to the service account for each GKE usage dataset by using the Google Cloud console or the Google Cloud Shell.

Use the Google Cloud console

  1. Navigate to the BigQuery console and locate the dataset created in Step 3.

  2. Select Share Dataset.

  3. In the Add members field, enter the email address of the service account configured in Step 1.

  4. Grant the service account the BigQuery Data Viewer role, select Add, and then select Done to apply the changes.

Use the Google Cloud Shell

Copy the command code snippets below and run them in sequence in the Google Cloud Shell.

gcloud config set project $customer_billing_project_id

bq show --format=prettyjson $DATASET_ID |
jq '.access = .access + [{"role": "READER", "userByEmail": "$customer_service_account' \
>permissions.json

bq update --source permissions.json $DATASET_ID

Verify the access

Check that the access to the BigQuery Data Viewer role has been granted.

Datasets with missing permissions are listed in the Features widget. They are not available in the GKE Cost Analytics feature.

Analyze GKE cost

Once you've set up GKE Cost Analytics, additional Google Kubernetes Engine dimensions and GKE Labels will be available in Cloud Analytics Reports.

You can drag a GKE chip or a GKE label chip into the Group by or Dimensions section to start understanding the real costs of running Kubernetes-based workloads on GKE. See Dimensions, Group by, and Filters for more information.

Limitations

  • GKE reports only show data from the day when all the steps were completed. If you need to load historical data (i.e. backfill), please contact DoiT support team to file a backfill request.

  • If you never enabled GKE usage metering in the Google Cloud Console before setting up GKE Cost Analytics, we will not be able to show historical data in GKE reports.

  • After you complete all the steps, it usually takes around four hours for you to be able to use GKE reports in Cloud Analytics.

  • GKE reports are not compatible with some of the dimensions. For example, you will not be able to create a report that is based on the GKE Cluster field and the Zone field. You also cannot combine Google Cloud Labels and GKE Labels in the same report.

  • Expect a delay of one or two days before a new label becomes available for GKE reports .