BigQuery Lens as data source
The DoiT Console provides a pre-built dashboard BigQuery Lens that sets up an audit log sink for all your BigQuery jobs. The audit logs record activities in BigQuery such as creating or deleting a table, purchasing slots, or running a load job.
You can use the BigQuery Lens audit log sink as data source for your own analysis about BigQuery activities.
- Cloud Analytics
Build reports using BigQuery Lens data
To build a report using data from the BigQuery Lens audit log sink:
Make sure you have already set up BigQuery Lens.
Open a report. Select Update at the top of the left-hand sidebar.
Select BigQuery Lens as the data source.
You can choose to Create a new report or Update the current one. Note that updating the report will lose all its current configurations.
Metrics and dimensions
Cloud Analytics reports use different sets of metrics and dimensions for billing data and BigQuery Lens data.
The basic metrics and extended metrics for BigQuery Lens data utilize fields of the jobStatistics object in BigQuery AuditData.
Cost: Processed bytes (
totalBilledBytes) converted to TiB multiplied by the on-demand scan price.
Usage: Processed bytes (
totalBilledBytes) converted to TiB, adjusted by the job's CPU usage.
Slots Used: The total number of slot-ms consumed divided by the query duration:
Total Slots Ms: The total number of slot-ms consumed by the query job. It maps to
Total Load Output Bytes: Total bytes loaded for an import job. It maps to
Total Tables Processed: Total number of unique tables referenced in the query. It maps to
Total Billed Bytes: Processed bytes, adjusted by the job's CPU usage. It maps to
Total Processed Bytes: Total bytes processed for a job. It maps to
Below are the available standard dimensions when the report's data source is BigQuery Lens:
Event name: Name of the event. It maps to
jobCompletedEvent.eventNamein BigQuery AuditData.
Job Status: State of a job: PENDING, RUNNING, or DONE. It maps to
jobStatus.statein BigQuery AuditData.
Query Priority: Priority given to the query: QUERY_INTERACTIVE or QUERY_BATCH. It maps to
jobConfiguration.query.queryPriorityin BigQuery AuditData.
Region: Location of the completed job. It maps to
jobName.locationin BigQuery AuditData.
Resource: URI for the referenced resource. For example, a table created by using an insert job reports the resource URI of the table. It maps to
protoPayload.resourceNamein BigQueryAuditMetadata messages.
Statement Type: Type of the statement. Example: SELECT, INSERT, CREATE_TABLE, CREATE_MODEL. It maps to
jobConfiguration.query.statementTypein BigQuery AuditData.
Caller IP: IP address of the caller. It maps to
requestMetadata.callerIpin audit logs.
User: The email address of the authenticated user (or service account on behalf of third party principal) making the request. It maps to
authenticationInfo.principalEmailin audit logs.