If a Couchbase Server node is starting up for the first time, it will create whatever DB files necessary and begin serving data immediately. However, if there is already data on disk (likely because the node rebooted or the service restarted) the node needs to read all of this data off of disk before it can begin serving data. This is called "warmup". Depending on the size of data, this can take some time.
When starting up a node, there are a few statistics to monitor. Use the cbstats command to watch the warmup and item stats:
shell> cbstats localhost:11210 all | » egrep "warm|curr_items"
| curr_items: | 0 |
| curr_items_tot: | 15687 |
| ep_warmed_up: | 15687 |
| ep_warmup: | false |
| ep_warmup_dups: | 0 |
| ep_warmup_oom: | 0 |
| ep_warmup_thread: | running |
| ep_warmup_time: | 787 |
And when it is complete:
shell> cbstats localhost:11210 all | » egrep "warm|curr_items"
| curr_items: | 10000 |
| curr_items_tot: | 20000 |
| ep_warmed_up: | 20000 |
| ep_warmup: | true |
| ep_warmup_dups: | 0 |
| ep_warmup_oom: | 0 |
| ep_warmup_thread: | complete |
| ep_warmup_time | 1400 |
Table 11.1. Monitoring — Stats
| Stat | Description |
|---|---|
| curr_items | The number of items currently active on this node. During warmup, this will be 0 until complete |
| curr_items_tot | The total number of items this node knows about (active and replica). During warmup, this will be increasing and should match ep_warmed_up |
| ep_warmed_up | The number of items retrieved from disk. During warmup, this should be increasing. |
| ep_warmup_dups | The number of duplicate items found on disk. Ideally should be 0, but a few is not a problem |
| ep_warmup_oom | How many times the warmup process received an Out of Memory response from the server while loading data into RAM |
| ep_warmup_thread | The status of the warmup thread. Can be either running or complete |
| ep_warmup_time | How long the warmup thread was running for. During warmup this number should be increasing, when complete it will tell you how long the process took |