Import historical data

Overview

This feature enables you to upload all your historical Google Cloud billing data into the DoiT Platform to immediately analyze your cloud spend and promptly identify optimization opportunities through Cloud Analytics, BigQuery Lens, and other DoiT features.

To get started, first, you need to ensure that you have a Service Account that connects your Google Cloud Organization to the DoiT Platform. Then, you'll have to grant two additional required permissions.

Note

We import your historical Cloud Billing data from Google BigQuery. To start collecting your Google Cloud Billing data, you must enable Cloud Billing data export to BigQuery.

You can only import as much historical Cloud Billing data as you already have available in BigQuery.

Suppose you enable Cloud Billing data export to BigQuery for the first time while following the steps in this document. In that case, you will only have around one month of historical Cloud Billing data to import into the DoiT Platform.

Identify the service account

Within the DoiT Console, select the gear icon (Settings) from the top navigation bar, then select Google Cloud under Cloud settings.

If you do not have a service account configured yet, you will need to create a service account before you can continue.

Select your desired service account, copy the service account email, and proceed to grant the required permissions.

Grant the required Permissions

Note

You need to have the appropriate IAM role/permission in GCP to grant the required permissions to the service account.

BigQuery Data Viewer role

1. From GCP, navigate to the BigQuery Console and open the dataset where the billing data is stored, then click "Share dataset"
2. In the Dataset Permissions tab, click "Add Member" and paste the service account email that was linked to the DoiT Platform as identified in Step 1.
3. Click "Select a Role" highlight BigQuery then choose BigQuery Data Viewer
4. Click "Add"
5. Click "Done"

Grant the bigquery.jobs.create permission

Depending on the features initially enabled from the process Connect Google Cloud Service Account, you may already have this permission granted to the role. Follow the steps below to verify / add the permission to the project.

1. From the GCP console, select your organization from the dropdown options
2. On the navigation menu, go to "IAM & Admin" then select "Roles"
3. Click on the role "DoiT Platform Service Account"
4. Check if bigquery.jobs.create is included in the assigned permissions, if yes, no further action is needed since these permissions will be inherited by the projects
5. If bigquery.jobs.create permission is not yet added, click on "Edit Role"
6. Click "Add Permissions"
7. Filter by "Permission: bigquery.jobs.create" and check the box for this permission
8. Click "Add"

Link your Google Cloud Direct Billing Account

Note

Required Permission: Assets Manager

Within the DoiT Console, select Billing from the top navigation bar and then select Assets.

From the Assets page, select Google Cloud from the left-hand navigation:

From the Google Cloud assets page, select Link Direct Account.

To import your historical billing data into the DoiT Platform, enter the following:

• Billing Account ID — For example 01CF51-9FDB87-7894FD
• Source Project — Enter the Project ID
• Source Dataset — Enter the Dataset ID

When you are done, select Import to continue.

A new card will be added to your Google Cloud assets page with import status information.