Home | Why NoSQL? | Use Cases

Post-relational data management for interactive software systems.

NoSQL Use Cases

Today's interactive software systems serve vast numbers of Web users and operate on a computing infrastructure that has changed radically in recent years. Couchbase NoSQL solutions uniquely fit the data management demands of many businesses that are defining this new computing environment. Check out the scenarios described in the tabs below to get a clearer picture of how we are solving real problems in these emerging Web and mobile applications use cases.

Couchbase Social Gaming Whitepaper

Social and Online Gaming

On a global basis, the social and online gaming market generates billions of US dollar equivalents in revenue annually and represents one of the fastest growing sectors of the economy. The explosion has been driven primarily by the relative ease of user acquisition (anyone with Internet connectivity can participate – and that represents an estimated 2 billion potential customers at this point) – and the viral awareness generation techniques, particularly via social media.

These software systems can go from zero to millions of users almost literally overnight – Zynga’s CityVille game reached 3 million daily active users within a week of its launch.  Supporting that kind of growth, while maintaining system performance and delivering a snappy gaming experience, presents an enormous challenge at every layer of the infrastructure stack, including the data tier.

Couchbase has become the de facto standard data management layer in the social and online gaming market, powering some of the largest and most successful games.

 

A variety of properties make Couchbase an ideal data management technology for social and interactive games:

  • Low, predictable latency. Interactive games must be responsive. Making a player wait for feedback is a first-class ticket to nowhere good. Countless studies over the years make it clear that humans hate to wait for software systems. If the experience is not snappy, users leave…and take their entertainment budget with them. Couchbase consistently delivers sub-millisecond random reads and writes of data, across the entire scaling spectrum (e.g. a 3k user game profile can be read in 800 microseconds, on average, with a very narrow 2 standard deviation band)
  • Dynamic elasticity. Couchbase supports the non-disruptive addition (or removal) of servers from a cluster. Game profitability is optimized by matching infrastructure costs to demand. And because changes can be made to a live cluster, there is never a need to take a game offline for maintenance, meaning continuous revenue generation.
  • Automated data caching. Not all gamers are online all the time. Couchbase is designed to hold in memory the data required to service online players. When offline, user data is automatically migrated to disk media, dramatically reducing infrastructure costs. When users log back in, data is automatically cached in main memory for the duration of the gaming session, ensuring maximum performance.
  • RightScale and Amazon relationships and experience. Many social games are launched on Amazon EC2 and a large percentage of those games run in RightScale templates. Couchbase is a standard component in the RightScale social gaming pack and is a proven strong performer in the Amazon AWS environment.
  • Proven in large-scale production game deployments. Couchbase has emerged as a key infrastructure component in some of the market’s most successful social and interactive games at organizations including Tribal Crossing, Zynga, Diversion (FameTown), DoubleDown Interactive, Come2Play, MeteorGames, Playdom/ESPN, Buffalo Studios, and many others.

Ad and Offer Targeting with Couchbase

Selecting an ad to display or an offer to present on a web page is a choice with direct revenue impact; and a choice that must be made quickly to minimize page load times. To make a revenue-maximizing decision, targeting logic must consider current behavioral, demographic and psychographic characteristics known, or inferred, about the targeted ad or offer recipient; and it must weigh those against the current status of running campaigns, contract commitments and goals.

AOL Advertising runs one of the largest online ad serving operations, serving billions of impressions each month to hundreds of millions of people. AOL faced three data management challenges when building their ad serving platform:

  • How to analyze billions of user-related events, presented as a mix of structured and unstructured data, to infer demographic, psychographic and behavioral characteristics that are encapsulated into hundreds of millions of user profiles
  • How to make hundreds of millions of user profiles available to their targeting platform with sub-millisecond latency on random reads
  • How to keep the user profiles fresh and current

The solution was to integrate two data management systems: one optimized for high-throughput data analysis (the “analytics” system), the other for low-latency random access (the “transactional” system). After evaluating alternatives, the final architecture paired Hadoop with Couchbase:

In this architecture, (1) click-stream data and other events are fed into Hadoop from a wide variety of sources (2) the data is analyzed using Hadoop MapReduce to generate hundreds of millions of user profiles; then, based on which ad campaigns are running, selected user profiles are loaded into Couchbase where (3) ad targeting logic can query Couchbase with sub-millisecond latency to get the data needed to make optimized decisions about real-time ad placement. The targeting system also writes to Couchbase after each decision is made – updating historical and statistical information that helps shape subsequent targeting decisions.

A number of intrinsic characteristics and capabilities of Couchbase make it a strong fit for the real-time data management needs of modern ad and offer targeting systems:

  • Production proven in large-scale ad and offer targeting systems. Couchbase has proven itself in large-scale production ad and offer targeting deployments with organizations including Chango, ShareThis, Context Web, Delta Project’s Ad Action, Media Mind, Ad Scale and AOL Advertising.
  • Schema-free data model. No need to define (or redefine) a database schema before inserting data. Targeting algorithms and approaches can change rapidly and often require changes in input data.
  • Elastic scaling. Effortlessly scales out to hold billions of data items for hundreds of millions of users, on commodity hardware or cloud computing instances
  • Consistent sub-millisecond random read and write latency. Consistently delivers sub-millisecond random read and write latency across the entire data set, supporting not only optimized decision making but enabling finely-targeted personalization of ad and offer content in tight decision time windows
  • Hadoop integration. Available Sqoop and Flume connectors provide “off the shelf” bi-directional connectivity between Hadoop and Couchbase
  • Built-in transparent caching. Stores data supporting active campaigns in main memory for deterministic sub-millisecond latency, while automatically migrating data items not currently required to disk for lower-cost storage

Cloud (XaaS) Data Services

Data elasticity is a fundamental cloud computing infrastructure requirement – whether building a public or private cloud computing environment. While the other three categories of cloud infrastructure software are fairly well understood and with myriad commercial and open source options (virtualization, automation, elastic application framework), elastic data management technology has been the most elusive component and the technology area where web application developers have been forced to innovate. Google invented the Bigtable distributed storage system for structured data. Amazon created the Dynamo highly available key-value store. LiveJournal invented the memcached distributed object caching system. These organizations were forced to invent solutions to their data management problems because there were no good “off-the-shelf” options. Solving the data elasticity problem is fundamental to enabling cloud computing.

Couchbase offers elastic data management technologies, ideally suited for cloud data management, based on widely deployed open source projects.  These systems are explicitly designed to scale “horizontally.” By simply adding more Couchbase data nodes, a PaaS cloud automation system can automatically expand the number of operations per second the data layer can sustain and the amount of data that can be stored, without requiring any change or disruption to running applications storing data in these systems.

In addition to scalability advantages, and an architectural “fit” with the cloud computing approach, these systems consistently perform 1 or 2 orders of magnitude faster than relational database management technology. This means far fewer servers are required to support a given workload.


 
Heroku is a leading cloud service provider with a PaaS offering hosting over 100,000 Ruby on Rails applications. Couchbase offers the most widely deployed add-on service for Heroku’s customers – delivering distributed caching services used by thousands of applications deployed to the Heroku cloud.

As shown in the architecture diagram below, Heroku customers (1) subscribe to a service appropriate for their application needs via the Heroku.com website (2) when a customer requests a Couchbase add-on service, the Heroku provisioning system communicates with a Couchbase provisioning system that (3) creates a new “bucket” (a secure Couchbase “slice” of the Heroku Couchbase cluster with memory and disk quotas) for the customer. The customer application (Best Buy, in this example) is then able to store data in its secure Couchbase container up to the enforced quota limits.

With a unique secure, resource-managed approach to multi-tenancy and its “no downtime” cluster elasticity, Couchbase is particularly well-suited to cloud computing deployments:

  • Secure multi-tenancy. Couchbase offers standards-based (SASL) secure application authentication and access control. Administrators need only manage a single cluster of servers, fairly and securely shared by many applications. In the Heroku environment, multiple thousands of applications securely share only a handful of servers. If every application required its own Couchbase Server, the solution would be orders of magnitude more expensive.
  • Quotas. In addition to security, resource management is critical in a cloud environment. No application should be able to monopolize system resources. Couchbase will enforce administrator-defined quotas which can be set and reset at any point either through a web administrator console, or programmatically through Membase RESTful management APIs.
  • Non-disruptive elasticity. Adding or removing servers from a Couchbase cluster can be done without disrupting any of the applications using the cluster. In a cloud computing environment users can come and go unpredictably. This capability allows appropriately matching resources to current customer load – eliminating costly over- or under-provisioning.
  • Upgrade a live system. There is never a need for planned system downtime with Couchbase – even software upgrades can be done on a live system without disrupting applications. In a cloud environment, many applications with widely varying tolerance for system downtime are hosted. Infrastructure components that cannot be in continuous service are a poor fit for this environment.
  • Proven in public and private cloud environments. Heroku (recently acquired by salesforce.com) and NHN (one of the largest enterprises in South Korea) use Couchbase technology in high-profile public and private (respectively) cloud deployments.

Mobile to Cloud Data Synchronization

Mobile computing is rapidly overtaking all other forms of consuming software. Apple is the obvious leader in the market; recently announcing over 10 billion iOS applications have been downloaded from the Apple App Store with over $2 billion paid out to developers of over 300,000 applications.

The most successful mobile applications are native applications – designed using the SDKs and libraries of a given device, and leveraging the unique capabilities of those platforms. By far, the most successful iOS applications are those specifically targeting the iOS platform. There are a number of reasons for this, but one of the most important is the transiently connected nature of these devices.

Not all iPhones, iPads or iPod Touch devices are online all the time, or even within range of Internet connectivity. But the devices, and software, must be useful whether online or offline over the middle of the Pacific Ocean at flight level 380.

In order to reach this burgeoning market, many of the most successful web application developers are building native mobile versions of their software.

Because these applications must be useful both on- and offline, data must be local to the device. But this requirement for local data availability presents at least two challenges to application developers. Because these devices are easily lost or damaged (certainly relative to data sitting in an Internet data center), provisions for data backup and restore must be made. The data must also often be refreshed, or synchronized with the state of the data behind the web version of the application

Couchbase offers a unique solution to these challenges. The Couchbase family of products includes mobile and data center varieties. A native iOS version of Couchbase was recently introduced and an Android version has been available for some time.

These mobile versions of Couchbase provide a powerful data management platform for native mobile applications. But the native synchronization capability of CouchDB technology, upon which Couchbase is built, enables transparent and automated database synchronization between the mobile device and a Couchbase deployment “in the cloud.”

Couchbase not only serves the data management needs of a Web application with millions of concurrent users, but also can maintain data synchronization between that platform and millions of mobile devices running a native version of the software system.