For performance, Couchbase Server mainly stores and retrieves information for clients using RAM. However, this is not always possible or desirable in an application, particularly if you have a very large set of data but you only actively use part of the information. In order to handle this scenario while maintaining high performance, Couchbase Server manages a working set of your entire information; this set consists of the all data you most frequently access and is kept in RAM for high performance.
At the same time, Couchbase Server will eventually store all data to disk to provide a higher level of reliability. If a node fails and you lose all data in the caching layer, you can still recover items from disk. We call this process of disk storage eventual persistence since the server does not block a client while it writes to disk, rather it writes data to the caching layer and puts the data into a disk write queue to be persisted to disk. Disk persistence enables you to perform backup and restore operations, and enables you to grow your datasets larger than the built-in caching layer. For more information, see Section 1.2.8, “Ejection, Eviction and Working Set Management”.
Couchbase automatically moves data from RAM to disk asynchronously in the background in order to keep frequently used information in memory, and less frequently used data on disk. Couchbase constantly monitors the information accessed by clients, and decides how to keep the active data within the caching layer. Data is ejected to disk from memory in the background while the server continues to service active requests. During sequences of high writes to the database, clients will be notified that the server is temporarily out of memory until enough items have been ejected from memory to disk. The asynchronous nature and use of queues in this way enables reads and writes to be handled at a very fast rate, while removing the typical load and performance spikes that would otherwise cause a traditional RDBMS to produce erratic performance.
When the server identifies an item that needs to be loaded from disk because it is not in active memory, the process is handled by a background process that processes the load queue and reads the information back from disk and into memory. The client is made to wait until the data has been loaded back into memory before the information is returned.
When the server stores data on disk and a client requests the data, it sends an individual document ID then the server determines whether the information exists or not. Couchbase Server does this with metadata structures. The metadata holds information about each document in the database and this information is held in RAM. This means that the server can always return a 'document ID not found' response for an invalid document ID or it can immediately return the data from RAM, or return it after it fetches it from disk.
Document Deletion from Disk
Couchbase Server will never delete entire items from disk unless a client explicitly deletes the item from the database or the expiration value for the item is reached. The ejection mechanism removes an item from RAM, while keeping a copy of the key and metadata for that document in RAM and also keeping copy of that document on disk. For more information about document expiration and deletion, see Couchbase Developer Guide, About Document Expiration .