Ali Ghodsi, co-founder and chief executive of Databricks Inc., believes that data and artificial intelligence estates are fragmented, costly and much too complex. Today, he explained how his company’s flurry of announcements will address these issues and more.

“It’s actually really hard still to succeed with data and AI,” Ghodsi (pictured) said during his keynote remarks today at the company’s Data+AI Summit in San Francisco. “It’s a complexity nightmare of high costs and proprietary lock-in. It’s slowing down the organizations. We think this is the biggest problem.”

Part of the solution will involve AI agents and an ability to simplify their development for enterprise use. Today Databricks unveiled Agent Bricks, a unified workspace that automates agent building and optimization using customers’ enterprise data and synthetic equivalents.

A key part of the release involves the use of large language model automated “judges” that generate questions and expected answers to assess model performance. This is presumably to resolve situations such as the one described by Ghodsi where one automaker expressed concern to him about an agent that was recommending a competitor’s cars.

“This maps to the business problems you have,” Ghodsi said. “You get an agent that can basically evaluate. We will optimize an AI system that does really well on the LLM judge benchmark.”

Leveraging Postgres for the AI era

The other major announcement from Databricks today involved the database world. The company’s release of Lakebase, a managed Postgres database built for AI, added an operational layer to the firm’s Data Intelligence Platform.

Databricks’ participation in the operational database or OLTP world is designed to address what the company’s believes to be an outmoded, inflexible model. “If you look at these databases, they were really built for a different era,” Ghodsi said. “There’s just this massive lock-in of data in traditional databases. It’s such a sticky thing.”

Lakebase builds on the company’s acquisition of Neon Inc., which was announced in May. Neon’s serverless PostgreSQL platform allows developers to add support for data structures where AI models keep information.

“We think this is going to be the future of all databases,” Ghodsi said. “Postgres has won. You really should properly separate compute and storage in that database using Postgres. It should be built for the AI era.”

Databricks CEO Ali Ghodsi (left) spoke with JPMorganChase CEO Jamie Dimon (right) at the Data+AI Summit

Today’s announcements were designed to address what Databricks has been hearing about AI deployment from its extensive enterprise customer base. Cost remains in issue, and data management is a significant part of the equation. This was echoed during an appearance by the chief executive of financial giant JPMorgan Chase and Co., Jamie Dimon, who told attendees that his company now spends about $2 billion per year on AI.

“The hardest part is the data, it isn’t the AI,” Dimon said. “Getting the data in the form that it can be usable is the hard part.”

New paradigm for software

Making data usable will require automation, based on the sheer size and complexity of the information involved. Among the many announcements from Databricks was an offering called Lakeflow Designer, a no-code capability that allows users to author production data pipelines by using a drag-and-drop interface and a natural language generative AI assistant. It’s the latest entry in the field of “vibe coding,” a recently coined description where users give into the “vibes” of AI and express their intentions without needing to know how to write a single line of code.

“It’s kind of like vibe coding if you are not coding at all, we call it vibe designing,” Ghodsi said during a briefing for the media following his keynote address. “It’s a new market for us, a new market opportunity.”

Tools such as Lakeflow Designer foreshadow a major sea change in how software will be designed and deployed for enterprise applications. During a brief appearance at the Databricks keynote session, Neon’s Chief Executive Nikita Shamgunov described how AI agents were creating four times more databases than humans on his company’s platform.

“I think we are at the dawn of the AI software revolution,” Shamgunov said.

The exhibit hall at Databricks’ Data+AI Summit

An example of this trend can be seen in Goose, an interoperable open-source AI agent framework developed at the digital finance firm Block Inc. Goose not only automates code generation, it can enable non-programmers to prototype new apps for features.

It was developed by Block using Anthropic Inc.’s Claude model, employed by developers for coding and tool use. In his remarks at the Databricks conference, Anthropic co-founder and CEO Dario Amodei acknowledged that virtual collaborators will soon become commonplace.

“Coding is moving the fastest,” Amodei said. “It’s a foreshadowing of what’s going to happen across all applications.”

A byproduct of this could be that the notion of devices as disposable, replaceable resources, often characterized as “cattle, not pets,” could soon apply to enterprise software as well. Software that once ran enterprise tasks for years could suddenly be replaced by a model where key new pieces are added and then removed within a matter of hours.

“There are going to be new paradigms for how software is produced and delivered,” Matt Dugan, vice president of data platforms for AT&T Inc., said in an interview with SiliconANGLE. “These are disposable. I wonder if we’re heading into a spot where we start to think of software assets the same way. It’s a change.”

Through tools such as Agent Bricks and Lakebase, Databricks is building the infrastructure to support this change in how software is created and deployed. It’s a game plan that has proven to work well for the 12-year-old company. Ghodsi noted that despite raising $10 billion late last year on a valuation of $62 billion, the privately held Databricks is not dependent on external capital, and it has plans to hire 3,000 employees this year.

“What’s our secret sauce?” Ghodsi said. “We have the customers, and we have the data. That’s the differentiator.”

Photos: Robert Hof/SiliconANGLE

VIDEO:  TheCUBE Databricks Data+AI Keynote Analysis with John Furrier, SiliconANGLE cofounder and George Gilbert, analyst theCUBE Research.