{"id":2270,"date":"2016-05-19T22:38:42","date_gmt":"2016-05-19T22:38:41","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=2270"},"modified":"2025-10-09T07:12:05","modified_gmt":"2025-10-09T14:12:05","slug":"efficient-io-with-circular-writes-on-global-indexes-in-couchbase-server-4-5","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/efficient-io-with-circular-writes-on-global-indexes-in-couchbase-server-4-5\/","title":{"rendered":"Faster Indexing and Query: Introducing Circular Writes with Standard Global Secondary Indexes for Efficient IO – PART III"},"content":{"rendered":"

In Part I<\/a> or the series, we have corevered the architecture behind global vs local Indexes and when to use a global (GSI) vs a local index (MapReduce View) index in Couchbase Server. In Part II<\/a> of the series, we have also talked about the new memory optimized global secondary indexes and how MOI improves the index maintenance performance with an in memory structure that is designed purely for high mutation rates and high scan rates. With\u00a0Part III, I’d like to tell you all about the how standard global secondary indexes improved in 4.5: There are a number of improvement in this area but the most important advancement is a new Write Mode called “Circular Writes”.<\/p>\n

Memory Optimized vs. Standard Global Secondary Indexes<\/h3>\n

Memory optimized indexes are added in 4.5 as an additional storage option for GSIs. Standard global secondary indexes have been there since version 4.0.\u00a0Administrators can configure GSI with either the standard GSI storage, which uses ForestDB underneath, for indexes that cannot fit in memory\u00a0or can pick the memory optimized GSI for faster in-memory\u00a0indexing and queries.\u00a0Even though memory optimized indexes, with in-memory index management, can provide the best index maintenance and scan performance, not everyone can efford to have all indexes in memory.\u00a0<\/span>Standard GSI can spill to disk when memory runs out, so efficient disk IO is critical to efficient indexing and scans.\u00a0<\/span><\/p>\n

Write Modes in Standard Global Secondary Indexes<\/h3>\n

Previously standard GSI only offered an apend-only write mode. Append Only writes write to the end of the file with every mutation to the index. However append-only writes require frequent compactions. With 4.5 Standard GSI comes with an additional write mode called “circular\u00a0writes”.<\/p>\n

When you enable \u201ccircular writes\u201d, as mutations arrive, instead of simply appending new pages to the end of the file, write operations look for reusing the orphaned space in the file. If there is not enough orphaned space available in the file that can accommodate the write, the operation may still do a write with append.<\/p>\n

<\/p>\n

With circular writes, full compaction still operates the same way. The compaction process reads the existing file and writes a new contiguous file that no longer contains the orphaned items, and is written as a contiguous file. However the number of compactions needed are drastically reduced.\u00a0Instead of compacting every few hours, it can be once a week and that is an amazing savings on the IO capacity (IOPS and\u00a0<\/span>MB\/sec).<\/span><\/p>\n

Configuring Write Mode and Compaction Trigger for Standard GSI<\/h3>\n

Standard GSI comes with 2 write modes.\u00a0The configuration for write mode and index fragmentation is under settings > auto-fragmentation in the w<\/span>eb console.\u00a0<\/span>(Note: F<\/span>ragmentation setting for index only applies when \u201cStandard Global Secondary Index\u201d storage option is selected for indexes.\u00a0<\/span>Write mode and compaction strategy does not apply to memory-optimized global secondary indexes.<\/span>)<\/span>\u00a0<\/span><\/p>\n