{"id":9467,"date":"2020-10-15T10:13:56","date_gmt":"2020-10-15T17:13:56","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=9467"},"modified":"2025-06-13T18:21:22","modified_gmt":"2025-06-14T01:21:22","slug":"when-couchbase-meets-splunk-the-real-time-ai-driven-data-analytics-platform","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/when-couchbase-meets-splunk-the-real-time-ai-driven-data-analytics-platform\/","title":{"rendered":"Couchbase & Splunk: A Real-Time AI-Driven Analytics Platform"},"content":{"rendered":"

This article builds on top of the session \u201cWhen Couchbase meets Splunk, the real-time, AI-driven data analytics platform\u201d presented at Couchbase Connect Online 2020 by James Powenski and Andrea Vasco.<\/em><\/p>\n

The Wall of Confusion<\/h2>\n

Ever since I was a University student, I had always been fascinated with data science. Back in the day, it was not yet a thing one would brag about – but I still remember how I felt the first time I stumbled upon estimation theory.<\/p>\n

We now live in a Golden Age of Data, an era in which datasets grew exponentially, becoming publicly available; today, many great platforms provide streamlined ways to take advantage of machine learning and deep learning techniques to bring the decision-making process to a superhuman level.<\/p>\n

Multi-Dimensional Scaling<\/strong> (MDS) is one of my preferred Couchbase features, with a soft spot for Indexing, Querying, and Analytics workloads isolation: I spent several years mining data on relational databases looking for patterns and correlations across large datasets, and many times I found myself \u2013 needless to say – running complex queries that would generate friction with DBAs about locks, degradation of performance, and so on.<\/p>\n

I hit hard on the so-called Wall of Confusion<\/a>, I guess. This problem was (and is) too so dear to me that in 2013 we wrote a paper<\/a> about the demystification of Oracle workload characterization.<\/p>\n

Setting the scene: bringing Continuous Intelligence in ACME<\/h2>\n

In 2019, Gartner identified Continuous Intelligence<\/a> as a Top-10 Analytics trend<\/a>, estimating that by 2022 “more than 50% of all business initiatives will require continuous intelligence, leveraging streaming data to enhance real-time decision-making”. <\/em><\/p>\n

In this article \u2013 and series \u2013 we will walk you through a practical way to get yourself started with Continuous Intelligence with Couchbase, aiming at disrupting your business towards digital transformation without significantly impacting your daily operations; we will pretend to be a Site Reliability Engineer (SRE)<\/a> at ACME Inc., tasked with the mission of implementing Continuous Intelligence for ACME\u2019s online store.<\/p>\n

We will assume you have a basic familiarity with Couchbase Analytics<\/a>. As shown in the picture below, Couchbase Analytics allows to create, in real-time, shadow copies of data stored into the KV Engine within a Massive Parallel Processing (MPP) architecture, that can be used to either query the shadow data using a SQL-like language (SQL++) and expose large, pre-aggregated datasets to third party solutions for further processing.<\/p>\n

\"\"<\/p>\n

Couchbase provides the foundational technology needed to kickstart ACME\u2019s journey towards Continuous Intelligence: the flexibility of NoSQL built onto a platform capable of sub-millisecond operations, workload isolation, multi-dimensional linear scalability, and integration with third-party solutions, all combined in one elegant platform spanning from the Multi-cloud to the Edge.<\/p>\n

As SRE at ACME, we can easily understand how Couchbase can provide a world-class enterprise data plane for the next-gen online store. What about business logic, though?<\/p>\n

Well, it depends on what kind of business logic is needed from our business: Couchbase provides, out of the box, a complete set of capabilities required to either programmatically retrieve individual documents or run ad-hoc\/a priori queries (query or analyze<\/strong><\/a>). But for ACME\u2019s next-gen online store, we may want to elevate our game and scout synergies with the many enterprise solutions in today\u2019s market that ship the full analytics arsenal \u2013 data exploration, observability, data navigation, real-time dashboarding, Machine Learning, and AI.<\/p>\n

It goes without saying, Couchbase is designed to integrate with them, and in today\u2019s example, we will be using Couchbase Analytics REST APIs<\/strong><\/a> to integrate with Splunk<\/strong>.<\/p>\n

Why Splunk? Here are a few compelling reasons \u2013 without getting too much into detail:<\/p>\n