Details
-
Type:
Improvement
-
Status:
Closed
-
Priority:
Blocker
-
Resolution: Fixed
-
Affects Version/s: 2.0
-
Fix Version/s: None
-
Component/s: None
-
Security Level: Public
-
Labels:None
Description
Need more content here:
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-introduction-architecture-eviction.html
-Describe new logic for eviction as of 2.0
-Tie into Access Logs topic
-Any command line which impacts this behavior
Input from Liang:
-ejection vs eviction: eviction a concept from memcached, it means the entire items (key, metadata and value) removed from RAM. Ejection is what we do now for 1.8 and 2.0: the value is removed but the keys and metadata still in memory
-warmup process - two iterations occur: firs tkeys and metadata loaded, then values for keys in access log. See info added on disk warmup on access scanner.
-expiry pager: remove expired items from memory. A disk cleanup process
-item pager: eject items if high water mark reached, eject hose that are not dirty or need to be replicated to node in cluster. High water mark configurable
-item pager: looks at NRU or if high water still breached, eject random, use % active vs replica. Also configurable via cbepctl
-goal improve memory efficiency, maintain balance of active/replica data.
defaults ejection % 40% for active with 50% cap, One parameter - adjust two items: active/replica-as-remainder
-goal- reach low water mark, but if under HWM, timer pager stops. if not repeats until under.
-NRU - replicated. if mutation on item, set to true. this true value replicated to other node in cluster. 1 bit.
-access scanner - adds all NRU true to access log, then sets all items to false after pass, thereby influencing item pager
-(percentage memory to be free) = (total memory - lwm)/ total memory
-config hwm at each node? Must be reset after node failure. Config are in memory on node.
-high active ratio reduce cache miss,
-too low replica% - if replica node promoted, will experience cache misses - post rebalance, post rebalance
Input from Jin on Configurations:
-Hi, you can change the default water marks by "flush_param [mem_high_wat/mem_low_wat] value" via cbepctl.
You can change the default percentage of active vbuckets being ejected items by item pager by
"flush_param pager_active_vb_pcnt value" where value is btwn 0 and 50%.
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-introduction-architecture-eviction.html
-Describe new logic for eviction as of 2.0
-Tie into Access Logs topic
-Any command line which impacts this behavior
Input from Liang:
-ejection vs eviction: eviction a concept from memcached, it means the entire items (key, metadata and value) removed from RAM. Ejection is what we do now for 1.8 and 2.0: the value is removed but the keys and metadata still in memory
-warmup process - two iterations occur: firs tkeys and metadata loaded, then values for keys in access log. See info added on disk warmup on access scanner.
-expiry pager: remove expired items from memory. A disk cleanup process
-item pager: eject items if high water mark reached, eject hose that are not dirty or need to be replicated to node in cluster. High water mark configurable
-item pager: looks at NRU or if high water still breached, eject random, use % active vs replica. Also configurable via cbepctl
-goal improve memory efficiency, maintain balance of active/replica data.
defaults ejection % 40% for active with 50% cap, One parameter - adjust two items: active/replica-as-remainder
-goal- reach low water mark, but if under HWM, timer pager stops. if not repeats until under.
-NRU - replicated. if mutation on item, set to true. this true value replicated to other node in cluster. 1 bit.
-access scanner - adds all NRU true to access log, then sets all items to false after pass, thereby influencing item pager
-(percentage memory to be free) = (total memory - lwm)/ total memory
-config hwm at each node? Must be reset after node failure. Config are in memory on node.
-high active ratio reduce cache miss,
-too low replica% - if replica node promoted, will experience cache misses - post rebalance, post rebalance
Input from Jin on Configurations:
-Hi, you can change the default water marks by "flush_param [mem_high_wat/mem_low_wat] value" via cbepctl.
You can change the default percentage of active vbuckets being ejected items by item pager by
"flush_param pager_active_vb_pcnt value" where value is btwn 0 and 50%.
Attached is the latest Server Guide with new and revised sections on eviction, ejection, and working set management. Please send your review comments to me by end of day (5PM PST) this Friday, January 18.
If there is anyone else in QA or the larger organization who would benefit from this content, let me know too.
Here are the sections
Summary/Introduction: Page 9 "Eviction, Ejection and Working Set Management"
2. Administrative Tasks/Technical Details: Page 76,
-Changing thresholds for ejection, page 77
-Specifying ejection preferences: page 77
Let me know if you have any additional input or questions.