Skip to main content

Set up BigQuery Intelligence

Before you begin​

BigQuery Intelligence needs specific permissions at the organization level. See BigQuery Intelligence permissions for details.

Set up BigQuery Intelligence​

To set up BigQuery Intelligence:

  1. Connect your Google Cloud Organization to the DoiT platform. Make sure to allow service account impersonation when configuring identities for DoiT workloads.

  2. Update your service account to grant the permissions required by BigQuery Intelligence. If you have workloads on BigQuery editions, also grant the permissions for BigQuery Intelligence Editions.

    BigQuery Intelligence Insights permissions allow DoiT Insights to generate actionable recommendations about your BigQuery usage. They do not directly relate to the BigQuery Intelligence setup and can be enabled later.

    Note

    In the current implementation, BigQuery Intelligence works with a single service account. If you grant permissions to multiple service accounts, BigQuery Intelligence works only for the first one.

Once set up, the BigQuery Intelligence will backfill historical data for the last 30 days and start gathering information about your Google Cloud BigQuery usage patterns. It can take up to 24 hours to fully populate the BigQuery Intelligence dashboard with statistics and recommendations.

VPC Service Controls​

If you deploy VPC Service Controls in your environment, you need to configure it at organization level to allow context-aware access from the BigQuery Intelligence:

  1. Create an access policy.

    1. In the Google Cloud console, navigate to Access Context Manager. If prompted, select your organization.

    2. Create a basic access level with the following settings:

      1. In the Access level title field, enter DoiT Console Access.

      2. Select Basic mode.

      3. For the When condition is met, return option, select TRUE.

      4. In the Conditions section, select Add attribute, IP Subnetworks, and then in the IP Subnetworks box, select Private IP.

      5. Choose Select VPC networks, in the Import options list, select Manually enter VPC network address, and then enter //compute.googleapis.com/projects/me-doit-intl-com/global/networks/doit-vpc-ca4b552.

      6. Save the configuration.

  2. Create a service perimeter to allow access to the following APIs:

    • BigQuery Reservation API
    • BigQuery API
    • BigQuery Data Policy API
    • BigQuery Data Transfer API
    • BigQuery Migration API
    • Cloud Logging API

  3. In the Access level pane, add the DoiT Console Access access policy created in step 1.

  4. In the Ingress policy pane, add a new rule for the log sink writer identity:

    1. FROM: Select Identities & groups, and then select the identity of the org log sink writer (update the ORG_ID): [email protected].

    2. TO: Select Projects, and then select the project of your DoiT console serice account (aeo-ipt-svc-accounts).

    3. For the Selected service BigQuery API, select all methods.

Last updated on