Skip to main content

Introduction

The Databricks Data Intelligence Platform enables your organization to leverage your data and AI, simplifying the complexities of working with large datasets and advanced analytics. Databricks cost structure is a combination of:

  • Your Databricks platform fees (based on your usage of Databricks units (DBUs)). DBUs usage and cost varies based on your workload types, such as, clusters, warehouses, and models. To prevent costs from spiraling, you need to understand the DBUs you are consuming so you can optimize Databricks for your needs.

  • The Cloud resources that Databricks uses. In addition to platform fees, you are also charged separately by your chosen cloud provider for the underlying infrastructure that Databricks uses. For example, VMs, storage, and networking.

Integrating Databricks with DoiT enables you to integrate Databricks cost and usage data, providing granular visibility into your Databricks spend, and attributing costs directly to projects and teams. This helps improve your forecasting capabilities, cost efficiency, and accountability across your data science and engineering initiatives.

Benefits​

  • Unified cost structure. DoiT enables you to integrate Databricks cost and usage data alongside data from your other cloud providers, giving you a comprehensive view of your cloud expenditures and facilitating better financial decision-making.

  • Enhanced visibility. Once you've imported your Databricks data into the DoiT platform, you can start using Cloud Analytics and other DoiT features such as Budgets, Allocations, Forecasting and Anomaly detection to analyze and monitor your Databricks cost and usage.

Key features​

  • Databricks Lens. The preset Databricks Lens dashaboard helps you understand the true cost of your data warehouses, shows DBU usage, and resource costs. Use the Lens dashboard to decide how and when to optimize costs, monitor performance, and more.

  • Databrick unit (DBU) consumption and cloud resource costs. Monitor the total usage and spend on DBUs and the cloud cost from your cloud provider.

  • Cluster utilization. Track average cluster utilization, CPU, and memory usage. This enables you to monitor the provisioning of your workloads.

  • Workload type. Monitor your DBU usage by workload type. This enables you to see trends for your different workload types.

  • Unit cost metrics. Evaluate costs per job, query, or user to track your engineering efficiency and understand the actual cost of providing a service or feature to your customers.

Last updated on