Unified data intelligence is the foundation of modern artificial intelligence, but unlocking its full potential requires a platform that makes data accessible, scalable, and easy to use.
Databricks Inc. has positioned itself at the center of this movement, offering a unified data intelligence platform that empowers developers, analysts and business users alike. A key part of the company’s strategy is its deep integration with Google LLC’s Google Cloud Marketplace, where 98% of Databricks customer journeys begin. The partnership — now in its fourth year — enables businesses to start small and expand without limits while leveraging unified data intelligence, according to Shanku Niyogi (pictured, left), vice president of product management at Databricks.
“At Databricks, our mission is to democratize data and AI,” he said. “Our data intelligence platform … brings together data analytics and AI with a unified way to govern … so that really anyone can get value out of data. I think the key value of integrating that with Google is to … use the power of the Google Cloud platform to be able to do that at scale in a way that’s really industry-leading. A key part of that for us has been this integration with the Google Cloud Marketplace.”
Niyogi spoke with theCUBE Research’s John Furrier for the Google Cloud Marketplace Marvels interview series, during an exclusive broadcast on theCUBE, News Media’s livestreaming studio. He was joined by Stephen Orban (right), vice president of migrations, ISVs and marketplace at Google Cloud. They discussed how their organizations are collaborating to provide a unified data intelligence platform that scales AI and analytics, helping ensure efficiency and ease of use for organizations. (* Disclosure below.)
Making AI accessible: The power of unified data intelligence on Google Cloud
Databricks’ availability on Google Cloud allows businesses to operationalize AI and analytics more efficiently, according to Orban. With Databricks on Google Cloud Marketplace, companies can adopt AI and analytics solutions without heavy infrastructure overhead. The platform’s ease of use enables analysts and developers to quickly transform raw data into insights, accelerating time to value.
“I think Databricks and Google … share a lot of common approaches to this,” Orban said. “We believe in the value that data brings … in the power of AI … in cloud-native technologies, to be able to scale these kinds of platforms to get business value. We both believe in open architectures and open source between things like Kubernetes at Google and Apache Spark, which kind of really started Databricks’ journey.”
Databricks has evolved from its origins in Apache Spark to introduce structured query language warehousing, its open lakehouse approach and Unity Catalog, which centralizes governance at scale as part of its unified data intelligence strategy. Its integration with Google Cloud ensures a seamless connection to Google’s data ecosystem, according to Niyogi.
“We brought in Unity Catalog … to help unlock the data at scale across the company in a way that was secure and governed in a single way and bringing in with Google’s data estate together in a way that’s deeply integrated and governed,” he said. “Then, adding this concept of data intelligence where we can put AI models to work on top of that.”
Optimizing AI and analytics: The Databricks and Google Cloud synergy
By launching its first fully containerized deployment on Google Cloud, Databricks provided organizations with a streamlined way to manage large-scale Spark workloads. This approach optimized resource allocation and improved cost efficiencies, according to Niyogi.
“When we launched that integration … we deeply integrated with Google’s kind of containerized platform,” he said. “This was actually the first fully containerized way to run Databricks, and that meant that you could take open-source Spark workloads and run them with [digital transformation] faster on Google Cloud, on Databricks, which is immediately kind of unlocking an immense amount of value and scalability for customers.”
Google Cloud’s technical architecture enhances Databricks’ platform by ensuring interoperability with tools such as BigQuery, Vertex AI and MLflow, reinforcing its unified data intelligence approach, according to Orban. These integrations allow customers to blend Google’s analytics and AI services with Databricks’ unified environment, improving workflow efficiency and reducing infrastructure overhead.
“I think it’s a huge testament to the technical strength of this partnership,” Orban said. “Databricks took the approach of actually taking advantage of all of the unique capabilities that Google Cloud has so that Databricks is actually highly optimized to run really, really well on Google Cloud.”
From data to action: How companies are leveraging Databricks
Uplight Inc., a company focused on energy optimization, needed a scalable data infrastructure to process real-time insights from 60 million endpoints, including thermostats and electric vehicle chargers, according to Niyogi. By adopting Databricks through Google Cloud Marketplace, Uplight gained the ability to process massive datasets in real time, applying unified data intelligence to optimize energy use at scale.
“They were able to use Google Cloud Marketplace to get started with Databricks on Google Cloud,” Niyogi said. “They now have 30,000-plus … Spark jobs. They’re adding new applications and new insights every day, and it’s already shown immense results.”
In 2022, Uplight’s AI-driven insights contributed to 1,200 megawatts of energy savings during a severe California heat wave, helping prevent widespread power outages. Faster application deployment also allowed the company to iterate on new services rapidly, ensuring they could meet shifting energy demands in real time, according to Niyogi.
“As a lifetime developer, one of my favorite stories out of that is using serverless just because they were able to iterate on their value quickly because they had things like quicker startup times,” he said. “They were able to get new applications to production 50% faster. That’s amazing because in a fast-moving landscape, that ability to turn something from idea to something that’s getting results in production is so valuable.”
Beyond Uplight, organizations worldwide are leveraging Databricks for critical applications, such as fraud detection, trade settlement and capital market analysis, applying unified data intelligence to these essential use cases, according to Orban. Financial institutions in New York, along with companies across the globe, are adopting the platform for these essential use cases.
“Just in the last year alone, I visited Latin America, the U.K., France, Germany, Singapore [and] Japan,” he said. “In every single one of those theaters, I’ve talked to customers locally in the region who are either evaluating or using Databricks in a material way to power some of those capabilities.”
Here’s the complete video interview, part of News’s and theCUBE Research’s coverage of the Google Cloud Marketplace Marvels interview series:
(* Disclosure: TheCUBE is a paid media partner for the Google Cloud Marketplace Marvels interview series. Neither Google LLC, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or News.)
Photo: News
Your vote of support is important to us and it helps us keep the content FREE.
One click below supports our mission to provide free, deep, and relevant content.
Join our community on YouTube
Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.
THANK YOU
