Databricks uses a pay-as-you-go model where the Databricks Unit (DBU) serves as the fundamental measure of compute power. Users pay for the duration and intensity of resource usage. For a job running at 1 DBU per hour for 10 hours at $0.55 per DBU, the total cost equals $5.50. System tables in Unity Catalog provide detailed usage logs, including DBU consumption, job runtimes, cluster usage, and model serving metrics. Cost tagging by project, agent, or environment allows users to identify which features drive the highest costs. These logs provide the data needed to see where spend goes across different workloads.
Primary cost factors in AI agent development
Compute resources constitute the most substantial expenditure in the development of AI agents. All-purpose compute supports interactive data science work and collaborative development. Jobs compute handles scheduled batch processing tasks, such as data preparation pipelines. SQL compute provides the analytics and business intelligence that feed into AI applications. Serverless compute manages infrastructure through automatic resource allocation. These four resource types determine the hourly DBU rate for a user.
Storage costs for data and models originate from cloud providers like AWS S3, Azure Data Lake, or Google Cloud Storage. Databricks does not charge for creating Delta tables or using Unity Catalog. Managed services like Predictive Optimization for Managed Tables run on top of this storage and incur additional Databricks costs.
The nature of operations influences the compute profile. A GPU-accelerated training task incurs higher hourly costs but may result in faster completion. Training or fine-tuning large language models (LLMs) remains the most resource-intensive stage of the AI development lifecycle. These workloads often require GPU-enabled or high-memory clusters. Cost drivers include the size and complexity of the training dataset, the choice of hardware, the duration of the training process, and the degree of parallelism.
Mosaic AI Gateway and model serving costs
Mosaic AI Gateway acts as a control plane for routing, monitoring, and governing AI model requests. It serves as a proxy between application logic and model endpoints. The gateway consumes DBU resources for request processing, observability, and governance checks. Every request passing through the gateway consumes compute time on a Model Serving endpoint.
When routing to external providers such as OpenAI or Anthropic, users pay provider token fees directly. Databricks also charges for gateway features, including routing, tracking, and logging, through DBUs. Because a single AI request often involves multiple DBU-consuming events, such as gateway routing, guardrail execution, and log ingestion into Delta Tables, engineering teams face significant difficulties when they attempt to forecast their monthly cloud spend with any degree of certainty.
Internal model serving involves two modes. The pay-per-token mode works for developmental testing or intermittent workloads. The provisioned throughput mode serves production performance. Provisioned throughput requires a minimum concurrency commitment and ensures availability, but it can result in idle capacity costs if traffic fluctuates.
| Component | Pricing Basis | Notes |
|---|---|---|
| Foundation Model Serving | DBU per 1M tokens | Prices vary by model |
| Provisioned Throughput | $0.07 per DBU minimum | Requires reserved capacity |
| Serverless SQL | $0.70 per DBU | Used for log analysis |
| T4-class Model Serving | $10.48 per hour | Hourly DBU rate |
| A100 80GB Model Serving | 628 DBUs per hour | For 8 GPUs |
| AI Guardrails | $1.50 per million tokens | Text filtering/PII masking |
Ecosystem dependencies and hidden costs
The Mosaic AI Gateway relies on Unity Catalog for governance. This reliance adds compute costs for permission checks during every model invocation. Inference logs stored in Delta Tables create storage costs through cloud providers and processing costs for background jobs. Querying these logs for audit or billing requires Databricks SQL, where serverless operations cost $0.70 per DBU.
Enabling AI Guardrails for tasks like PII masking or toxicity filtering uses Model Serving compute and additional DBUs. Internal benchmarks indicate that guardrail complexity can increase P95 latency by 50ms to 200ms.
High-concurrency environments may require provisioned capacity for external routing to prevent scale-to-zero issues. Using the gateway ties logging and governance to the Databricks architecture. You know that managing cloud costs is a constant struggle, so you must understand the nuances of the DBU.
How can organizations accurately predict the cost of a single agent run when every interaction triggers a different combination of DBU-consuming events?
Agent orchestration and evaluation workloads
Many AI agents rely on orchestration frameworks like MLflow Pipelines, LangChain, or custom scheduling logic. These tasks may not be compute-heavy, but they contribute to total DBU consumption if they run on inefficient infrastructure or last for long durations.
Evaluating the performance of AI agents is a critical phase. Mosaic AI Agent Evaluation assesses RAG systems and complex chains. Running evaluations on large datasets is resource-intensive. Using LLM judges to assess correctness and groundedness involves significant computational power. Evaluation processes also generate substantial data, including logs, metrics, and traces, which require storage.
An agent run involves a cycle where the system reasons, retrieves data, invokes tools, and returns a result. A single request might involve multiple DBU-consuming events, including gateway routing, guardrail execution, and log ingestion into Delta Tables.
Customer support agent cost example
A company deploying a customer support agent with 1M queries and 500K inferences of 100 tokens each faces several distinct costs. The LLM via Model Serving for 500K requests costs between $500 and $1000.
The agent uses Delta Lake for structured reads. 1M structured queries on Silver or Gold tables cost between $300 and $500 using Photon SQL compute.
Mosaic AI Evaluation for 50K offline evaluations per month costs 1,000 DBUs at a rate of $1.20 to $1.80 per DBU. The 100K live evaluations cost 2,000 DBUs at a rate of $0.50 to $0.75 per DBU.
The AI Gateway requires two active endpoints running 720 hours per month at 1 DBU per hour, costing between $720 and $1,080. Payload logging for 500K requests of 100 tokens each costs between $100 and $150. AI Guardrails for 50M tokens cost $75.
The ETL and orchestration tasks using 500 DBUs at a rate of $1.00 to $1.40 per DBU cost between $500 and $700. Storage for 2 TB of data in Delta Lake costs $40. The agent also uses 500GB of external output.
Optimizing agentic AI expenditures
Complexity in AI agents leads to escalating costs. Organizations can manage these costs by streamlining orchestration. Evaluating if an agent truly requires a full orchestration framework like LangChain can reduce overhead. For simpler agents, custom logic provides a lightweight approach. Using job compute clusters with short auto-termination windows minimizes idle time. For compute-heavy tasks, running them as separate jobs provides better control over resource allocation and autoscaling.
Vector retrieval optimization can reduce costs. Users should leverage Delta Table filters to narrow the search space. Using Approximate Nearest Neighbors (ANN) search libraries can accelerate Vector Search.
Structured data access can be more efficient through column pruning and precise WHERE clauses. Selecting only necessary columns minimizes data transfer.
Alternatives to Databricks Mosaic AI
Some engineering teams evaluate unbundled alternatives to achieve clearer unit economics. TrueFoundry provides a Kubernetes-native architecture that deploys directly into a customer’s cloud account. It does not add a management DBU on top of raw instance costs. TrueFoundry does not charge a per-token markup for external routing and supports Spot instances for inference.
Amazon Bedrock provides managed access to foundation models for teams already using AWS. Baseten handles autoscaling inference for users who own their models. Fireworks AI and Together AI focus on fast, low-cost inference for open models. Replicate and Modal provide per-second compute pricing for running custom code or community models on GPUs.
Databricks makes the product easier to adopt technically faster than it makes the workload easier to price commercially.
Scaling and deployment requirements
The Mosaic AI Agent Framework reached general availability on 7 March 2025. By mid-2026, the product includes MLflow 3, Databricks Apps, Model Serving, AI Search, and multi-agent orchestration. Users can build with OpenAI Agents SDK, LangGraph, or LangChain.
Scaling an agent requires managing the balance between latency and cost. High request volumes require scaled infrastructure. GPU-based inference improves latency but increases costs.
The DBU model provides flexibility but complicates forecasting. A single request can trigger various DBU rates depending on the model, the tool, and the governance requirements. Organizations must manage the relationship between throughput and provisioned capacity to avoid excessive spend.
