Design decisions influenced by CAP Theorem
The CAP theorem states that a database cannot simultaneously provide all three of the following guarantees:
-
- Consistency (the latest information is always available everywhere)
- Availability (every read and write request receives a response)
- Partitioning Tolerance (think of it as a form of fault tolerance)
In the CAP theorem for databases, most databases need to persist the state and value of their data. Data needs a container and a host system. But running a single copy of data on a single host only works if the host has electricity. And the data can only be available on the host before and after an electrical outage if it is persisted to storage that can tolerate a lack of electricity. These two needs spawn the requirement to make copies of data and store them on multiple hosts and storage drives. Thus clusters are needed for Partition tolerance in a database (the P in the CAP theorem).
So if you must have partition tolerance, the tradeoffs for distributed databases are between Consistency and Availability. Consistency requires that writes to every node or data partition are consistent with one another across the cluster because data integrity is the most important thing. CockroachDB enjoys calling out important data examples like “the nuclear codes.”
Availability is the other attribute of CAP that a database should support for operational reliability–like Partition, the database is useless if you can’t find it. In many cases, it is important to always be “on” and never to miss a request to save or retrieve the data. For both people and countless applications, an unavailable database is a deal-breaker. So for AP systems, the experience is the most important thing.
CockroachDB is CP-based, Couchbase is Variable from CP to AP
CockroachDB is a CP-based database; every write to CockroachDB is transactional and replicated according to the consensus-based RAFT protocol to disk stores containing ranges of Key/Value data. Therefore, CockroachDB is an extremely strongly consistent database. Their great blog explains that they are “almost” able to support Jepsen’s “Strict Serializable” level of consistency in their database.
Couchbase is indeed a strongly consistent database too but offers variables to allow itself to modify consistency levels for availability, transforming it into an AP system. Consistency can still be achieved within milliseconds via asynchronous replication among a cluster due to its in-memory design. In trade, Couchbase focuses on delivering extremely high performance, tunability, and availability. Delivering low latency to applications is a primary design principle.
Cockroach’s strength is its focus on transactionality via a scalable cluster design. Couchbase’s strength is its focus on performance and flexibility at scale. Their primary concern for an application is data integrity, while ours is how the data is used (often by people). Data doesn’t tell you when it’s the wrong value, but people will tell you when your application is slow or unavailable.
ACID transactions
From a transactional standpoint, Couchbase is not following RAFT protocol, but the principles are the same. However, Couchbase does offer, and has recently received patent approval for, multi-document ACID transactions for JSON documents. Unlike CockroachDB, these features are tunable for durability and isolation levels. According to Jepsen, transactions can be isolated to support Monotonic Atomic Reads, which is the highest level of isolation afforded to always-available systems.
Scaling clusters via Data Location Maps
Both distributed databases scale well. CockroachDB scales its nodes by defining raft clusters (usually three) of key/value data ranges in a partition or data store. A CockroachDB node can host both leaders and replicas of the data ranges in its store, and the size of ranges is managed and partitioned automatically. The clever part of CockroachDB is that these raft-based ranges of KV data are small and easily replicable to other nodes. So an EC2 instance can host many data range sets of leader and two replicas.
The distribution layer of their architecture is what every node in CockroachDB uses to find the location of the application’s data using their “monolithic sorted map structure.” The tree structure of this map is called recursively and caches locations on the node. Then RPCs send batch query requests to the node that hosts the leader of the raft range.
CockroachDB is designed such that each node is homogeneous, requires little configuration and all nodes do the same kind of work. They characterize their availability design as “multi-active availability.”
Active-active cluster design
Couchbase supports active-active cluster design where any member of the cluster can take a write or a read. This flexibility is made possible by two design decisions. First, similar to CockroachDB, the data location of Couchbase data is managed by the cluster. But unlike CockroachDB, which requires every node to manage, query and cache the map, Couchbase delivers the cluster map to the application within the Couchbase SDK. This way, the application always knows where its data is, removing location inquiries that often sap the infrastructure’s resources. Couchbase reduces noisy traffic within the database so that it can cache real work. An explanation of the differences can be found in this cloud-based data processing presentation.
Couchbase replication and auto partitioning features are managed similarly to Cockroach–vBuckets break up Couchbase data into partitions for replication and storage management. The Couchbase cluster map tells the application where the writable active node for the vBucket exists and where its replicas are located. In the event of an active vBucket failure, a replica takes over and becomes the active, writable set of documents. Then a replacement replica is constructed and filled. All intra-cluster replication (and inter-cluster replication via XDCR) happens in-memory and is performance tunable.
In-Memory caching and performance tuning
The CockroachDB Blog explains their consumption of memory as it can simultaneously be used for many types of caching activities such as the RocksDB cache (containing the KV data for a data range), the map that locates nodes where data range leaders are hosted, session connections from clients, query channels for running SQL queries, SQL query operations, and other activities from each architectural layer. Of course, the faster the CPU of the host, the quicker some operations become. Still, there appears to be no way to performance-match resources to specific operations like Couchbase multidimensional scaling.
Like CockroachDB, at the core of Couchbase is a high-performance Key/Value data store (called Magma), which supports high-speed access to the data. Couchbase additionally draws from its memcached roots, offering in-memory processing of not only this Key/Value access, but also all internal operations, including intra-cluster replication for fault tolerance, inter-cluster replication (XDCR) for geographic locality, as well as query, search, analytics and eventing operations. But unlike CockroachDB, Couchbase is performance-configurable such that any of these Couchbase services can be performance-matched to its hosting infrastructure and application requirements. If the data access service needs more memory or CPU, give it more nodes, RAM, or CPU. You can do likewise with all the independent services of Couchbase. This feature is called multidimensional scaling, and is unique to Couchbase.
With Couchbase multidimensional scaling, data replication and access services can have their own set of dedicated node and memory allocations, as can each of the query, indexing, search, eventing and analytic services.
SQL query support
Both databases offer support for SQL. However, CockroachDB supports the PostgreSQL API, schema structures and ANSI SQL. Couchbase does not require schema structures, but they are available. Likewise, Couchbase does not require strong data typing, nor does it offer support for specific schema constraints within the database, leaving it to the application developer to implement them.
Couchbase uses JSON as its flexible, easily modified data access format and has extended it to support Full-Text Search and Time Series structures, among others. This extensibility gives Couchbase the advantage of being able to do the work of up to six different products. Couchbase also offers service engines for JavaScript and Python events, User-defined Functions, a Cost-based query optimizer for SQL++ (SQL for JSON, formerly called N1QL), analytic aggregations on active data and taking backups of the database. Interfaces to these services are intentionally intuitive to help rapid integration, e.g., using SQL++ as it operates identically to SQL for most users.
Replication and synchronization
CockroachDB does not support cross datacenter replication, but it can deploy cluster nodes in different geographic regions. Couchbase supports eventual consistency for XDCR and uses a CAS-based (Check and Swap) conflict resolution method when documents with the same HCL-stamped (hybrid-local clock) CAS value are encountered. In this case, either the most actively accessed document or the most recent can be specified as the winner. CockroachDB uses an HCL as its time stamp for consistency and conflict resolution too. Couchbase offers mobile synchronization features to its Couchbase Lite mobile database as well.
Reading stale data
The replication features, including those for maintaining write and read consistency (Serialized for CockroachDB, Monotonic Atomic View for Couchbase distributed transactions) and every other activity in CockroachDB is treated as a transaction with ACID guarantees. For reads, CockroachDB offers both:
-
- Strongly-consistent (“non-stale”) reads: The default and most common type of read, they go through the leaseholder where these reads see all writes performed by writers that committed before the reading transaction started. They always return data that is correct and up-to-date.
- Stale reads: Useful in situations where you can afford to read data that is slightly stale in exchange for faster reads. They can only be used in read-only transactions that use the AS OF SYSTEM TIME clause. They do not need to go through the leaseholder since they ensure consistency by reading from a local replica at a timestamp that is never higher than the closed timestamp.
Couchbase is a strongly consistent database whose isolation level should be considered Monotonic Atomic View for transactional reads. Couchbase could return stale data when not using ACID transactions, but this is controllable by the developer.
Which database to choose?
If durable transactions for 100% of your data are your primary (or only) concern, choose CockroachDB. But due to its inflexibility, because it is a relational design and monolithic installation, it is not easily tunable to the performance and scaling needs of your application except for its transaction and storage scaling. Because of this, CockroachDB will likely introduce unanticipated performance issues whose only solution is to scale vertically by adding RAM or CPU in addition to scaling horizontally for storage. Because it uses a schema-based design it will be more difficult to evolve and change once the database is deployed.
If performance, flexibility, application adaptability, user happiness, geographic scale, cost-effectiveness or mobility are at all a concern, use Couchbase. Couchbase offers numerous performance tuning and infrastructure optimization features that can maximize your environment’s efficiency. Couchbase will deliver impressively low latency timings that will not deteriorate under load or at scale. This ensures that Couchbase can do more work with less resources and also be more cost effective overall.
Couchbase’s JSON-based design allows the development team to control the data structures used by the application, and modifications to these structures can be made continuously, even after the application is in production. Couchbase offers distributed, multi-document ACID transactions but does not require them. This design choice allows developers to balance transactional guarantees against performance and availability needs.
Couchbase offers XDCR and mobile application services, including an embeddable database, Couchbase Lite. Ultimately, Couchbase delivers data closer to where an application will consume it and supports more data access design patterns (Key/Value, JSON, Search, relational, time series, analytic and eventing), giving developers more feature freedom without introducing more architectural complexity. Couchbase gives development teams more options and freedom to build what they need.
Both are great databases but for different reasons.
