Big-data analytics startup creates petabyte-scale analytics solution with ultrafast query results
Date Updated: Tuesday, August 14, 2018
Founded in 2016, Kyligence is a cutting-edge data intelligence company that offers solutions for big data analytics. Kyligence products are powered by Apache Kylin, the leading open-source online analytical processing (OLAP) engine that’s built for interactive analytics of petabyte-scale data on Hadoop. (Apache Hadoop is the open-source software framework used for distributed storage and processing of big datasets.) We also offer Kyligence Robot, an online tuning and optimization service for Apache Kylin, and Kyligence Cloud, an elastic data analytics platform in the cloud.
"With Kyligence Analytics Platform in Azure, big data analytics that used to take days or weeks to process may now require only a few mouse clicks to get query results in seconds. That’s why our motto is ‘unleash big data productivity."
Luke Han: Cofounder and Chief Executive Officer
Big data, ultrafast analytics, no programming
Our flagship product is the Kyligence Analytics Platform (KAP), an enterprise OLAP solution on Hadoop that provides super-fast analysis results, known as subsecond query latency, on truly massive datasets. Users can access business intelligence (BI) capabilities on Hadoop with industry-standard methodologies for data warehouse and BI operations. As part of this, KAP simplifies analytics by providing self-service, seamless interoperability with popular BI tools—no programming is required.
Specifically, many of our customers use Microsoft Power BI and Excel as analytics tools. These tools are both comprehensive and easy to use; however, our customers recognize the value of the enhanced database querying capabilities that KAP offers. For example, with KAP, customers can explore massive data and make decisions quickly without repeated IT help. Plus, the corresponding short learning curve further boosts productivity and saves on the cost of specialized labor.
High performance from pre-calculation
For cloud implementations, KAP operates in Azure where most data is stored using Azure Blob storage or Azure Data Lake Store. From there, the information is loaded into Apache Hive as external tables. KAP builds cubes from the tables and then aggregates and analyzes them. (An OLAP cube is a collection of data stored in a multidimensional array.) At runtime, the user’s queries access the aggregated (that is, pre-calculated) cube data. Our cube pre-calculation approach is the magic ingredient for speed.
The pre-calculation might happen, say, before a business opens so that when people arrive at the office, a framework is already in place to process a variety of query types. In this way, KAP can provide consistently fast analytics results—or low query latency—often in the subsecond range, with high concurrency. Thousands of results per second are a real use case, not just a theoretical maximum. Or as we summarize it without all the jargon: “build once, query fast, serve more.”
In addition to aggregated query results, KAP supports detailed data queries that help users discover more from underlying datasets. By supporting Kafka data sources, KAP is able to consume streaming data, resulting in near real-time analytics, which is a key benefit in today’s world of increasing streaming activity.
Note that KAP is native on Hadoop, interacts with clusters through standard APIs, and fully supports Hadoop distributions on-premises and in the cloud. KAP connects with HDInsight Kafka clusters to ingest data from Apache Kafka. And KAP performance is 100 times faster than comparable Apache Hive querying.
Less time and money, higher productivity
Using KAP, customers almost immediately gain three practical benefits. First, because the solution is so fast, customers get the mission-critical analytics they need when they need it. With Kyligence Analytics Platform in Azure, big data analytics that used to take days or weeks to process may now require only a few mouse clicks to get query results in seconds. That’s why our motto is “unleash big data productivity.” Second, KAP accelerates customers’ business decision making. Faster data analysis corresponds to higher performance of customers’ big-data driven applications and more opportunities to examine BI from different perspectives. Third, our solution is cost-effective. By tapping into the effectively unlimited capacity of Azure, KAP helps customers take on big-data projects without more effort or bigger investments in IT.
A partner, not just a provider
Thanks to working with Microsoft, Kyligence is now part of an ecosystem of professionals who share our enthusiasm for both the Microsoft open-source roadmap and the open-source platforms that our products run on. Azure has become an essential element in the development and delivery of our open-source solutions—not just in terms of technology but also of a business relationship. Kyligence is a Microsoft re-seller, so we frequently work with an Azure sales team that connects us to customers and helps us provide them with better service. We are also a Microsoft Accelerator member, so we have access to expert Microsoft assistance and advice and to cost-effective Azure resources that reduce our IT costs.
Additionally, one of our cofounders, Luke Han, is the creator of Kylin and both a Microsoft Regional Director and Microsoft Most Valuable Professional (MVP) awardee. Consequently, our expertise is validated and exposed to a wide audience.
Early success, long-term growth
Based on the success we’ve had so far—we are one of the top 25 big-data companies in the Asia-Pacific region as recognized by APAC CIO Outlook magazine—Kyligence plans to make our products and solutions available globally. We have dual headquarters, one in Shanghai and the other in the heart of Silicon Valley, Santa Clara, California. We have a lot of on-premises business in China and the United States, but future worldwide operations will benefit from the scalability and availability of Azure. We can’t travel everywhere, but Azure reaches almost anywhere. It’s critical to our growth strategy.
"Kyligence is now part of an ecosystem of professionals who share our enthusiasm for both the Microsoft open-source roadmap and the open-source platforms that our products run on."
Luke Han: Cofounder and Chief Executive Officer
This post was originally featured on Microsoft Customer Stories.