Apache Druid reinforces its leadership position as the real-time analytics database for modern analytics applications.

Imply, the company founded by the original creators of Apache Druid®, today unveiled at a virtual event the second milestone in Project Shapeshift, the 12-month initiative designed to solve the most pressing issues developers face when building analytics applications. This milestone introduces the following:

• A multi-stage query engine for Apache Druid, representing the largest expansion of Druid’s architecture since inception.
• Imply’s Total Value Guarantee, a global guarantee for Apache Druid users that shows how the Imply subscription is effectively free for qualified participants.
• Continued momentum for Imply Polaris, the cloud database service built from Apache Druid.

Imply is uniquely built for the intersection of analytics and applications. Increasingly, companies are building modern analytics applications that combine analytics on big data with application traits including sub-second response, high concurrency and operational use. The developers at leading companies, including Atlassian, Reddit, and TrueCar, are building these applications to underpin their digital businesses, and Apache Druid with Imply is their database-of-choice.

“Snowflake is perfect for BI reporting analytics, and MongoDB is a great transactional database for applications,” stated FJ Yang, CEO and co-founder at Imply. “But where analytics meets applications requires a new type of database. That’s where Apache Druid with Imply comes in.”

“There’s a big market opportunity for database companies at the intersection of analytics and applications,” said Eric Kavanagh, CEO and Co-Founder, The Bloor Group. “What I’m excited about with Imply is their keen focus on rewriting the end-to-end process of discovery, design and deployment of next-gen analytic apps.”

After introducing Project Shapeshift at Druid Summit 2021, Imply launched Polaris at the first milestone event in March. Today for the second milestone, Imply introduces three major announcements that helps make Druid even more capable, accelerates developer success, and enhances the Imply Polaris cloud service.

Breakthrough Apache Druid expansion

A source of Apache Druid’s performance for interactive analytics at scale lies within its highly efficient and optimized single-stage query engine. A second engine—a multi-stage query engine, optimized for more complex data flow—was introduced in a private preview by Imply in March. The new engine was conceived to dramatically transform data ingestion and expand querying capabilities.

“We always thought of Druid as a shapeshifter when we originally built it to support analytics apps of any scale,” said Gian Merlino, CTO and co-founder of Imply and PMC chair for Apache Druid. “Now we’re excited to show the world just how nimble it can be with the addition of multi-stage queries and SQL-based ingestion.”

Today, Imply announces its contribution to the multi-stage query engine in Apache Druid

24.0. In this release, the multi-stage stage query engine enables the following:

• Simplified and accelerated batch ingestion – Like every database, data must be ingested before it can be Now, ingestion in Druid uses common SQL queries that benefit from the new extensions to Druid, making it both easier to ingest data and up to 65% faster. With many Druid deployments ingesting hundreds of terabytes daily, this saves both time and expense.
• SQL-based in-database transformation – Formerly, Druid supported a limited set of in-database table transformation capabilities. Now, Druid supports any in-database transformation without tuning or expertise using SQL, enabling data enhancement, data enrichment, easy experimentation with aggregates, approximations (including hyperloglogs and theta sketches), and more, with the same ease-of-use and performance enhancements as SQL-based ingestion.
• Foundation for expanded ecosystem support – With the addition of SQL-based ingestion and transformation, Druid has built a foundation for integration with a range of open source and commercial data tools, covering transformation (dbt), data integration (Informatica, FiveTran, Matillion, Nexla, io), data quality (Great Expectations, Monte Carlo, Bigeye) and others.

Imply’s Total Value Guarantee for Apache Druid users

Imply delivers a complete developer experience for Druid across its commercial distribution, cloud services and expertise. With Imply, developers are able to save time from managing the database and save money on infrastructure—where both time and money contribute to total cost of ownership (TCO). Paytm, for example, saved 12 additional engineering hours per week and reduced infrastructure costs by 50% with Imply.

Today, Imply is making it possible for Apache Druid users to get all the value of a partnership with Imply effectively for free—and lower their TCO for Druid. Imply is introducing the Total Value Guarantee for qualified participants that guarantees the total cost of ownership (TCO) to run Druid with Imply—measured across software, support, and infrastructure—will be less than the TCO when running Apache Druid on their own.

“At Imply, we love building and using open source software but having a partner can be helpful too,” said Vadim Ogievetsky, CXO and co-founder of Imply. “Now with Imply’s Total Value Guarantee, developers can get a partner for Druid that will help them get all the advantages of Imply’s products and services and be there in the middle of the night if needed— with Imply effectively for free. ”

Continued Imply Polaris momentum

Imply Polaris, a database-as-a-service built from Apache Druid, was introduced in March. This cloud database was built to do more than cloudify Druid; it also optimizes data operations and delivers an end-to-end service from stream ingestion to data visualization. In five months, Polaris has added over 250 accounts, representing companies across a wide range of industries.

“As a market leader in weather information and data solutions, and with a global audience of more than 50 million consumers, we’re committed to providing advertisers with the insights they need to optimize their campaigns on our platforms, while remaining committed to protecting the privacy of our users,” said Radu Nicolae, Technical Product Manager for Pelmorex’s DSP. “With Imply Polaris as an internal analytics powerhouse, we can track changes in traffic and enable our operations team to adjust strategies in real time. As a cloud database service, Polaris was the fastest, most affordable, and secure way to build our Apache Druid-powered service.”

Today, Imply announces a series of product updates to Polaris that enhance the developer experience, including:

• More flexible data ingestion – Polaris adds support for schemaless ingestion to accommodate nested columns, allowing for arbitrary nesting of typed data like JSON or DataSketches are now supported at ingestion for faster subsecond approximate queries.
• Simpler Polaris operations – Polaris adds performance monitoring alerts to ensure consistent performance for ultra-low latency queries and greater security with resource-based access control and row-level security. Finally, updates to Polaris’ built-in visualization enables faster slicing and dicing.
• More flexible pricing – Polaris adds new node types to flexibly meet any price/performance requirement at any New comprehensive consumption and billing metrics were added for instant usage visibliity.

Learn More

• Learn about the new enhancements in Apache Druid in this blog
• Read our blog and webpage to learn more about Imply’s Total Value Guarantee
• Sign up for a free trial of Imply Polaris and read this blog on the latest updates
• View the Apache Druid developer success ebook
• Watch this video to learn more about Imply
• Learn about the K2D (Kafka to Druid) stack for real-time analytics applications
• Watch the “How Druid Does It” technical lightboard series
• Hear from FJ Yang at the 2022 Confluent Current keynote

Availability:

• Apache Druid 0 is available for public download on druid.apache.org
• All of the Imply Polaris features included are now available

Subscribe to Industry Today