Huge amount of used disk space taken by a single node

Couchbase Server
1

I created a primary index on ~1.35 billion docs in a 3 node cluster.
Here is what the server reports as the size:

index name      requests/sec    resident ratio  items           data size       bucket      status  
#primary        0               0%              799,213,932     102,262.6 MB    Library     building 59%

However, when looking at the disk, I have over two TB of disk space in use, see:

image
image887×125 5.03 KB

Any idea what is going on here?

2

@orendb, please check if your setup has “circular write mode” enabled.

https://docs.couchbase.com/server/6.5/manage/manage-settings/configure-compact-settings.html#index-fragmentation

3

Hi,
Yes, this is enabled. I assume that I need to set it to “Append-only write mode with index fragmentation level trigger:” ?

4

My index is at 80% done (which took a several days). Can I modify this setting without causing a restart of indexing?

5

Circular write mode is the preferred option. You can change the setting without restart.