3 steps to determining your database needs
What is NoSQL and what is a NoSQL database?
NoSQL database technology stores information in JSON documents instead of columns and rows used by relational databases. To be clear, NoSQL stands for “not only SQL” rather than “no SQL” at all. This means a NoSQL JSON database can store and retrieve data using literally “no SQL.” Or you can combine the flexibility of JSON with the power of SQL for the best of both worlds. Consequently, NoSQL databases are built to be flexible, scalable, and capable of rapidly responding to the data management demands of modern businesses. The following defines the four most-popular types of NoSQL database:
- Document databases are primarily built for storing information as documents, including, but not limited to, JSON documents. These systems can also be used for storing XML documents, for example.
- Key-value stores group associated data in collections with records that are identified with unique keys for easy retrieval. Key-value stores have just enough structure to mirror the value of relational databases while still preserving the benefits of NoSQL.
- Wide-column databases use the tabular format of relational databases yet allow a wide variance in how data is named and formatted in each row, even in the same table. Like key-value stores, wide-column databases have some basic structure while also preserving a lot of flexibility.
- Graph databases use graph structures to define the relationships between stored data points. Graph databases are useful for identifying patterns in unstructured and semi-structured information.
Why use NoSQL?
Customer experience has quickly become the most important competitive differentiator and ushered the business world into an era of monumental change. As part of this revolution, enterprises are interacting digitally – not only with their customers, but also with their employees, partners, vendors, and even their products – at an unprecedented scale. This interaction is powered by the internet and other 21st century technologies – and at the heart of the revolution are a company’s cloud, mobile, social media, big data, and IoT applications.
How are these applications different from legacy enterprise applications like ERP, HR, and financial accounting? Today’s web, mobile, and IoT applications share one or more (if not all) of the following characteristics. They need to:
- Support large numbers of concurrent users (tens of thousands, perhaps millions)
- Deliver highly responsive experiences to a globally distributed base of users
- Be always available – no downtime
- Handle semi- and unstructured data
- Rapidly adapt to changing requirements with frequent updates and new features
Building and running these massively interactive applications has created a new set of technology requirements. The new enterprise technology architecture needs to be far more agile than ever before, and requires an approach to real-time data management that can accommodate unprecedented levels of scale, speed, and data variability. Relational databases are unable to meet these new requirements, and enterprises are therefore turning to NoSQL database technology.
Global 2000 enterprises are rapidly embracing NoSQL databases to power their mission-critical applications:
- Tesco, Europe’s No. 1 retailer, deploys NoSQL for e-commerce, product catalog, and other applications
- Ryanair, the world’s busiest airline, uses NoSQL to power its mobile app serving over 3 million users
- Marriott deploys NoSQL for its reservation system that books $38 billion annually
- Gannett, the No. 1 U.S. newspaper publisher, uses NoSQL for its proprietary content management system, Presto
- GE deploys NoSQL for its Predix platform to help manage the Industrial Internet
Five trends creating new technical challenges that NoSQL DBs address
These companies and hundreds more like them are turning to NoSQL because of five trends that present technical challenges that are too difficult for most relational databases.
Digital Economy trends
|1. More customers are going online||
• Scaling to support thousands, if not millions, of users
• Meeting UX requirements with consistent high performance
• Maintaining availability 24 hours a day, 7 days a week
|2. The internet is connecting everything||
• Supporting many different things with different data structures
• Supporting hardware/software updates, generating different data
• Supporting continuous streams of real-time data
|3. Big data is getting bigger||
• Storing customer generated semi-structured/unstructured data
• Storing different types of data from different sources, together
• Storing data generated by thousands/millions of customers/things
|4. Applications are moving to the cloud||
• Scaling on demand to support more customers, store more data
• Operating applications on a global scale – customers worldwide
• Minimizing infrastructure costs, achieving a faster time to market
|5. The world has gone mobile||
• Creating “offline first” apps – network connection not required
• Synchronizing mobile data with remote databases in the cloud
• Supporting multiple mobile platforms with a single backend
Why relational databases fall short
Relational DBMS (database management systems) were born in the era of mainframes and business applications – long before the internet, the cloud, big data, mobile, and today’s massively interactive enterprise. These databases were engineered to run on a single server – the bigger, the better. The only way to increase the capacity of these databases was to upgrade the servers – processors, memory, and storage – to scale up.
NoSQL databases emerged as a result of the exponential growth of the internet and the rise of web applications. Google released the Bigtable research paper in 2006, and Amazon released the Dynamo research paper in 2007. These databases were engineered to meet a new generation of enterprise requirements:
The need to develop with agility, to meet changing requirements, and to eliminate data transformation.
By contrast, a NoSQL database fully supports agile development and does not statically define how the data must be modeled. Instead, NoSQL defers to the applications and services, and thus to the developers as to how data should be modeled. With NoSQL, the data model is defined by the application model. Applications and services model data as objects.
Eliminate data transformation
Applications and services model data as objects (e.g., employee), multi-valued data as collections (e.g., roles), and related data as nested objects or collections (e.g., manager). However, relational databases model data as tables of rows and columns – related data as rows within different tables, and multi-valued data as rows within the same table.
The problem with relational databases is that data is read and written by disassembling, or “shredding,” and reassembling objects. This is the object-relational “impedance mismatch.” The workaround is transforming data via object-relational mapping frameworks, which are inefficient at best, and problematic at worst. NoSQL databases generally handle data as it is presented.
How does NoSQL work?
How does NoSQL compare? Let’s take a closer look. The following NoSQL tutorial illustrates an application used for managing resumes. It interacts with resumes as an object (i.e., the user object), contains an array for skills, and has a collection for positions. Alternatively, writing a resume to a relational database requires the application to “shred” the user object.
Storing this resume would require the application to insert six rows into three tables, as illustrated in Figure 3.
And, reading this profile would require the application to read six rows from three tables, as illustrated in Figure 4.
Operate at any scale
Databases that support web, mobile, and IoT applications must be able to operate at any scale. While it's possible to scale a relational database like Oracle (using, for example, Oracle RAC), doing so is typically complex, expensive, and not fully reliable. With Oracle, for example, scaling out using RAC technology requires numerous components and creates a single point of failure that jeopardizes availability. By contrast, a NoSQL distributed database – designed with a scale-out architecture and no single point of failure – provides compelling operational advantages.
Elasticity for performance at scale
Applications and services have to support an ever-increasing number of users and data – hundreds to thousands to millions of users, and gigabytes to terabytes of operational data. At the same time, they have to scale to maintain performance, and they have to do it efficiently.
The database has to be able to scale reads, writes, and storage.
This is a problem for relational databases that are limited to scaling up (i.e., adding more processors, memory, and storage to a single physical server). As a result, the ability to scale efficiently, and on demand, is a challenge. It becomes increasingly expensive because enterprises have to purchase bigger and bigger servers to accommodate more users and more data. In addition, it can result in downtime if the database has to be taken offline to perform hardware upgrades.
In contrast to relational technology, a distributed, NoSQL database partitions and distributes data to multiple database instances with no shared resources. In addition, the data can be replicated to one or more instances for high availability (intercluster replication). While relational databases like Oracle require separate software for replication (e.g., Oracle Active Data Guard), NoSQL databases do not – it’s built in and it’s automatic. In addition, automatic failover ensures that if a node fails, the database can continue to perform reads and writes by sending the requests to a different node.
As customer engagements move online, the need to be available in multiple countries and/or regions becomes critical. While deploying a database to multiple datacenters increases availability and helps with disaster recovery, it also has the benefit of increasing performance, because all reads and writes can be executed on the nearest datacenter, thereby reducing latency.
Ensuring global availability is difficult for relational databases in cases where the requirement of separate add-ons increases complexity (e.g., Oracle requires Oracle GoldenGate to move data between databases) – or when replication between multiple datacenters can only be used for failover because only one datacenter is active at a time. Also, when replicating between datacenters, applications built on relational databases can experience performance degradation or find that the data centers are severely out of sync.