AI agents run on data pipelines rather than prompts. Every agent that makes a decision in production reads from a data pipeline, queries a data store, and relies on data quality to produce reliable outputs. Data engineering provides the foundation through clean pipelines, governed data stores, and semantic layers. The major platforms see this convergence clearly. Snowflake describes its platform as a control plane that connects AI, data, and enterprise systems. Databricks released Agent Memory for Agent Bricks in Beta on October 20, 2026. This feature uses Lakebase to retain business preferences, previously referenced data, revised analysis logic, and frequently used business procedures. This capability reduces unnecessary re-searches and tool calls. Snowflake also evolves through the Cortex Agent SDK, which entered Public Preview in October 2026. The Cortex AI Gateway provides monitoring for AI usage, security, and cost management. You should monitor how these tools manage context and execution. Data engineers who understand agent architectures become more valuable. Organizations that invest in both disciplines simultaneously see real production results. This convergence relies on tools like dbt, Airflow, and Prefect to handle ETL, ELT, and orchestration. The industry’s shift toward AI agents requires a move from passive analytics to active, AI-assisted operations. AI agents must reason over business terminology and context through semantic layers to avoid inconsistent results.
Platform competition and revenue growth
Databricks exceeded a $7 billion annualized revenue run rate on August 13, 2026. The company reached a $190 billion valuation following a $5 billion funding round. Databricks reported over 80% year-over-year revenue growth in the second quarter of 2026. This growth follows a $4.8 billion run rate in December 2025 and a $5.4 billion run rate in February 2026. Databricks acquired MosaicML for $1.3 billion to combine its technology with a language-model platform. In India, Databricks launched infrastructure on Google Cloud in Mumbai to meet demand, following 80% annualized growth in the country over the past two financial years.
Snowflake competes for these workloads by expanding into AI agents and real-time processing. Snowflake’s Online Feature Store reached General Availability in October 2026. The company also provides enhanced Iceberg interoperability for its Horizon Catalog. Snowflake also shipped SnowConvert AI to automate Redshift migration, removing friction for organizations using competing platforms. Snowflake maintains a mature data sharing ecosystem with over 2,000 listings in its Marketplace. While Snowflake builds a distribution machine through ecosystem leverage and partner integrations, Databricks targets technical buyers who require flexibility for custom AI applications and deep machine learning workflows that require specialized, highly technical, and scalable tooling. Databricks leads the competition in machine learning integration while Snowflake wins on SQL simplicity. Snowflake provides more explicit control over compute isolation through virtual warehouses, whereas Databricks uses clusters that run Spark, SQL, or ML workloads.
| Capability | Databricks | Snowflake |
|---|---|---|
| AI Agents | Agent Bricks / Agent Memory | Cortex Agents / AI Gateway |
| Semantic Management | Metric Views / Genie | Semantic Studio |
| Open Catalog | Unity Catalog | Horizon Catalog |
| Real-time AI | Zerobus / Lakebase | Online Feature Store |
Governance and adoption trends
Industry leaders expect AI adoption to deliver efficiency gains. A report by MIT Technology Review Insights says 81% of survey respondents expect AI to boost efficiency in their industry by at least 25% in the next two years. Nearly half of organizations expect budget increases for AI and data infrastructure to exceed 25%. However, 39% of respondents say they must invest in talent and upskilling the workforce to improve their data strategy. An even larger share, 72%, say innovation will be very important to attract and retain talent. Most organizations, specifically 58%, use a hybrid approach by using vendor large language models for some tasks and building their own for others. Nearly 74% of lakehouse adopters say the benefits are significant.
Governance creates new risks during deployment. A Snowflake Cortex AI sandbox escape allowed a prompt injection attack chain to execute malware on a SOC 2 certified platform. This incident reveals that platform-level security controls do not cover AI workload-level behaviors. Databricks released DASF v3.0 to include agentic AI-specific risks and controls. Organizations also look to ISO 42001 to manage AI system risks, including lifecycle management and operational controls. Sixty percent of respondents say a single governance model for data and AI is very important. In financial services, AI will be nearly ubiquitous by the end of 2026. Every major bank, insurer, and asset manager will have a pilot or deployment. Will the integration of real-time processing and FinOps reduce the total cost of business decisions?




