Connect Google Cloud resources

Google Cloud resources are organized hierarchically. You can use Google Cloud's Identity and Access Management (IAM) system to assign granular access to specific resources and prevents unwanted access to other resources. The IAM policy hierarchy follows the same path as the Google Cloud resource hierarchy.

To offer the full capabilities of the DoiT platform, we require permissions at different levels to access the Google API and obtain information for monitoring or alerts, and help you act on recommendations if applicable. See Security and data access policy for details about the required permissions.

Required permissions

To connect Google Cloud resources with the DoiT console:

• Your DoiT account must have the Manage Settings permission.

• To connect a Google Cloud Organization, you must have been granted the Organization Role Administrator (roles/iam.organizationRoleAdmin) IAM role on the organization. You need this role to create and attach custom roles to service accounts under the organization.

• To connect a Google Cloud project that is not part of an already connected organization, you must have been granted the Role Administrator (roles/iam.RoleAdmin) IAM role on the project. You need this role to create and attach custom roles to service accounts under the project.

• To connect a BigQuery dataset for insights and recommendations from GCP recommender, you must have been granted the Role Administrator (roles/iam.RoleAdmin) IAM role on the project that hosts the dataset. You need this role to create and attach custom roles to service accounts under the project.

Note that at the moment, most of the DoiT features require permissions at the organization level. Only a small set of feature and core functionality permissions can be granted at the project level.

Service account authentication

Google Cloud supports using service accounts as identities for workloads. Starting June 2023, the preferred way to authenticate the service account for DoiT console workloads changes to service account impersonation, which mitigates the security risks associated with service account keys.

Service account impersonation involves two principals:

• The privilege-bearing service account that has the required permissions to access the target resource.

• The caller that needs to access the target resource but lacks permissions.

Configure identities for DoiT workloads

When connecting the DoiT console, you mainly need to perform two tasks to configure the identities for DoiT workloads:

• Task 1: Create a privilege-bearing service account with permissions to access the target resources.

• Task 2: Grant the DoiT-owned service account [email protected] (caller) the ServiceAccountTokenCreator role, allowing it to impersonate the privilege-bearing service account.

  Note

  If you use GCP organization policy to restrict identities by domain, make sure to whitelist the DoiT caller service account using its Organization Resource ID C03rw2ty2.

Below are the detailed configuration steps:

1. Sign in to the DoiT console, select the gear icon () from the top navigation bar, and then select Google Cloud in the Cloud settings section.

   

2. Continue with the section specific to your target resource type:

   • Connect an organization
   • Connect a project
   • Connect a dataset

Connect an organization

1. From the Connect drop-down, select Organization.

2. Select the features you want to enable for the organization. You can expand features to view the required permissions.

   

3. Select Generate gcloud commands. The generated gcloud commands you need to run will be displayed in the DoiT console.

   

4. Select a location to store your BigQuery Lens dataset.

5. Launch a Google Cloud Shell session.

6. Select a Google Cloud project that meets the following conditions:

   • The project belongs to your organization.

   • The project is connected to a Google Cloud Billing account (see Verify the billing status of your projects).

   • You intend to keep the project for the long term and restrict its access only to trusted people.

7. Configure access control for the organization by running the commands provided in the DoiT console sequentially:

   1. Create a new service account in the current project.

   2. Enable service usage API and resource manager API in the current project.

   3. Get your Google IAM organization ID.

   4. Create a custom role named doit_cmp_role with the permissions required by your chosen features, then add a policy binding to attach the role to the new service account under the organization. Permissions are granted at the organization level.

   5. Add an IAM policy binding to attach the role roles/iam.serviceAccountTokenCreator to the DoiT caller service account, allowing it to impersonate the privilege-bearing service account.

8. Sign in to the Google Cloud console, go to the Roles page and verify that you have created a doit_cmp_role linked to the DoiT service account in your organization.

Connect a project

Note

This section applies only if you want to connect a Google Cloud project that is not part of an already connected organization.

To connect an project:

1. From the Connect drop-down, select Project.

2. Select the features you want to enable for your project. You can expand features to view the required permissions.

   

3. Select Generate gcloud commands. The generated gcloud commands you need to run will be displayed in the DoiT console.

   

4. Get your Google IAM Project ID. See Identifying projects.

5. Launch a Google Cloud Shell session and select the project you want to connect.

6. Configure access control for the project by running the commands provided in the DoiT console sequentially:

   1. Set the project ID.

   2. Create a new service account in the project.

   3. Enable service usage API and resource manager API in the project.

   4. Create a custom role named doit_cmp_role with the permissions required by your chosen features, then add a policy binding to attach the role to the new service account. Permissions are granted at the project level.

   5. Add an IAM policy binding to attach the role roles/iam.serviceAccountTokenCreator to the DoiT caller service account, allowing it to impersonate the privilege-bearing service account.

7. Sign in to the Google Cloud console, go to the Roles page and verify that you have created a doit_cmp_role linked to the DoiT Service Account in your project.

Connect a dataset

To display insights from GCP Recommender BigQuery export in DoiT Insights, you must connect the BigQuery dataset that contains insights and recommendations:

1. From the Connect drop-down, select Dataset.

2. Select the checkbox of Insights from GCP Recommender BigQuery export. You can expand the feature to view the required permissions.

   

3. Select Generate gcloud commands to display the instructions and gcloud commands you need to run.

   

4. Provide the dataset name and an identifier to form the new service account. Make sure to follow the naming conventions displayed in the DoiT console.

5. Get your Google IAM Project ID. See Identifying projects.

6. Launch a Google Cloud Shell session and select the project that hosts the dataset.

7. Configure access control for the BigQuery dataset resource by running the commands provided in the DoiT console sequentially:

   1. Set the project ID.

   2. Create a new service account for the dataset access.

   3. Create a custom role named doit_cmp_dataset_project_access_role with project-level permissions, then Add a policy binding to attach the role to the service account.

   4. Create a custom role named doit_cmp_dataset_access_role with dataset-level permissions, then update the dataset IAM policy to attach the role to the service account.

   5. Add an IAM policy binding to attach the role roles/iam.serviceAccountTokenCreator to the DoiT caller service account, allowing it to impersonate the privilege-bearing service account.

8. Download the service account details to a file named doit_cmp_dataset_sa_details.json, then select Upload file in the DoiT console to upload the file.

9. Sign in to the Google Cloud console, go to the Roles page and verify that you have created two custom roles (doit_cmp_dataset_project_access_role and doit_cmp_dataset_access_role) linked to the DoiT Service Account in your project.

Update role

To add or remove features for an organization or project:

1. Select the gear icon () from the top navigation bar, and then select Google Cloud.

2. Select the three-dot menu next to the organization or project you want to update, and then select Edit.

3. Modify the selection of features as needed.

4. Select Update role.

   You'll be asked to run gcloud commands to update your service account's role.

Connect multiple resources

• If you have multiple Google Cloud organizations or projects, connect each separately to regulate which organization or project has access to specific features.

  Note that BigQuery Lens doesn't support multiple service accounts at the moment. See Set up BigQuery Lens.

• If you connect multiple BigQuery datasets for insights and recommendations, DoiT Insights will generate insights based on data in all the connected datasets.

Remove a connection

To remove a connected organization, project, or dataset.

1. Select the three-dot menu next to the resource you want to disconnect.

2. Select Delete connection.

Remove service accounts

In case you need to remove a service account under a connected organization from the DoiT console and then create a new one, be aware that:

• Removing a service account from the DoiT console also removes the related datasets and sinks created by BigQuery Lens.

• Once the BigQuery Lens permissions are granted to the new service account, we'll re-create the sink and trigger historical jobs backfill, which may lead to increased costs during your service account setup.