Databricks Co-Founder Wins ACM Prize, Declares AGI Already Here

Matei Zaharia's pioneering work on Apache Spark and data infrastructure has enabled global-scale AI, but he argues the field is misunderstanding what current systems can do.

Apr. 10, 2026 at 6:09am

Got story updates? Submit your updates here. ›

A highly detailed, glowing 3D illustration of a complex data processing pipeline or AI agent architecture, with recognizable physical tech elements like circuit boards and cooling systems illuminated by vibrant neon lights, conceptually representing the powerful data and AI infrastructure that has enabled the rapid growth of enterprise AI.Zaharia's work on the data and AI infrastructure powering enterprise AI has enabled rapid commercial adoption, challenging the field to reframe how it measures progress.Berkeley Today

Matei Zaharia, the co-founder of Databricks and creator of Apache Spark, has won the prestigious 2026 ACM Prize in Computing for his foundational contributions to distributed data systems and AI infrastructure. In an interview, Zaharia argued that artificial general intelligence (AGI) has already arrived, 'it's just not in a form that we appreciate,' and that the field should stop benchmarking AI against human cognition.

Why it matters

Zaharia's work on Apache Spark, Delta Lake, and MLflow has become the backbone of modern data and AI infrastructure, powering the rapid growth of enterprise AI adoption. His provocative claim about the current state of AGI challenges the conventional definition and measurement of AI progress, which could have significant implications for the competitive dynamics in the AI research landscape.

The details

Zaharia began developing Apache Spark as a PhD student at UC Berkeley in 2009, creating a faster alternative to Hadoop MapReduce for large-scale distributed data processing. Spark's in-memory computing approach cut processing times from hours to minutes or seconds, effectively superseding MapReduce. Spark became the seed of Zaharia's co-founded company, Databricks, which reached a $134 billion valuation in 2025. Zaharia later co-developed Delta Lake to bring ACID transactional semantics to cloud data lakes, enabling the 'data lakehouse' architecture, and MLflow to manage the operational chaos of machine learning in production. His recent research has focused on systems to make AI agents more reliable and capable, including the open-source DSPy framework for prompt engineering and the GEPA project for improving multi-step AI workflows.

  • Zaharia began building Apache Spark as a doctoral student at UC Berkeley in 2009.
  • Zaharia's doctoral dissertation on Spark won the ACM Doctoral Dissertation Award in 2014.
  • Zaharia co-founded Databricks in 2013 with six Berkeley colleagues.
  • Databricks reached a $134 billion valuation in December 2025 following its Series L funding round.
  • Databricks disclosed a revenue run rate of $5.4 billion in February 2026, growing at more than 65% year on year.

The players

Matei Zaharia

The co-founder of Databricks and creator of Apache Spark, who has won the 2026 ACM Prize in Computing for his foundational contributions to distributed data systems and AI infrastructure.

Databricks

The data and AI company co-founded by Zaharia in 2013, which has become a leading provider of enterprise data and AI infrastructure.

Apache Spark

The open-source distributed data processing framework created by Zaharia, which has become the default framework for AI model and tool releases that aim for broad commercial adoption.

Delta Lake

The open-source project co-developed by Zaharia to bring ACID transactional semantics to cloud data lakes, enabling the 'data lakehouse' architecture.

MLflow

The open-source project created by Zaharia to address the operational chaos of machine learning in production, becoming one of the leading platforms for operationalizing AI at scale.

Got photos? Submit your photos here. ›

What they’re saying

“AGI is here already, it's just not in a form that we appreciate.”

— Matei Zaharia, Co-founder, Databricks

“We should stop trying to apply human standards to these AI models.”

— Matei Zaharia, Co-founder, Databricks

What’s next

Zaharia's recent research has shifted to systems that make AI agents more reliable and capable, including the open-source DSPy framework for prompt engineering and the GEPA project for improving multi-step AI workflows. His vision is for AI to enable autonomous scientific investigation at a scale and speed that no human team could replicate.

The takeaway

Zaharia's work has been instrumental in building the data and AI infrastructure that has enabled the rapid growth of enterprise AI adoption. His provocative claim about the current state of AGI challenges the field's conventional definitions and measurement of AI progress, potentially reshaping the competitive dynamics in AI research.