For performance, Couchbase Server mainly stores and retrieves information for clients using RAM. At the same time, Couchbase Server will eventually store all data to disk to provide a higher level of reliability. If a node fails and you lose all data in the caching layer, you can still recover items from disk. We call this process of disk storage eventual persistence since the server does not block a client while it writes to disk, rather it writes data to the caching layer and puts the data into a disk write queue to be persisted to disk. Disk persistence enables you to perform backup and restore operations, and enables you to grow your datasets larger than the built-in caching layer. For more information, see Section 1.2.8, “Ejection, Eviction and Working Set Management”.
When the server identifies an item that needs to be loaded from disk because it is not in active memory, the process is handled by a background process that processes the load queue and reads the information back from disk and into memory. The client is made to wait until the data has been loaded back into memory before the information is returned.
Multiple Readers and Writers
As of Couchbase Server 2.1, we support multiple readers and writers to persist data onto disk. For earlier versions of Couchbase Server, each server instance had only single disk reader and writer threads. Disk speeds have now increased to the point where single read/write threads do not efficiently keep up with the speed of disk hardware. The other problem caused by single read/writes threads is that if you have a good portion of data on disk and not RAM, you can experience a high level of cache misses when you request this data. In order to utilize increased disk speeds and improve the read rate from disk, we now provide multi-threaded readers and writers so that multiple processes can simultaneously read and write data on disk:
This multi-threaded engine includes additional synchronization among threads that are access the same data cache to avoid conflicts. To maintain performance while avoiding conflicts over data we use a form of locking between threads as well as thread allocation among vBuckets with static partitioning. When Couchbase Server creates multiple reader and writer threads, the server assesses a range of vBuckets for each thread and assigns each thread exclusively to certain vBuckets. With this static thread coordination, the server schedules threads so that only a single reader and single writer thread that access the same vBucket at any given time. We show this in the image above with six pre-allocated threads and two data Buckets. Each thread has the range of vBuckets that is statically partitioned for read and write access.
For information about configuring this option, see Section 5.1, “Using Multi- Readers and Writers”.
Document Deletion from Disk
Couchbase Server will never delete entire items from disk unless a client explicitly deletes the item from the database or the expiration value for the item is reached. The ejection mechanism removes an item from RAM, while keeping a copy of the key and metadata for that document in RAM and also keeping copy of that document on disk. For more information about document expiration and deletion, see Couchbase Developer Guide, About Document Expiration .