[MB-3799] moxi default hashing choice is not compatible with spy, enyim for memcached buckets (which use ketama_weighted) Created: 10/May/11  Updated: 25/Jul/11  Resolved: 25/May/11

Status: Resolved
Project: Couchbase Server
Component/s: None
Affects Version/s: 1.6.0 GA, 1.6.1, 1.6.4, 1.6.5, 1.6.5.2, 1.6.5.3, 1.6.5.4
Fix Version/s: 1.7 GA
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Steve Yen
Resolution: Fixed Votes: 0
Labels: 1.7.0-release-notes, 1.7.1-release-notes
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
Investigation of observed different caching between spymemcached and moxi in one environment shows that moxi is inadvertantly using the default from libmemcached. I say inadvertantly because that default may actually be the less common choice. It is different than what Enyim, Spymemcached and libketama do by default when selecting Ketama.

libmemcached seems to use non-weighted Ketama, which seems to be specific to libmemcached and does not define the hash continuum the same way. Specifically, it seems to define the continuum with 100 points per server, rather than the 160 commonly used.

 Comments   
Comment by Matt Ingenthron [ 10/May/11 ]
I know it seems odd to file this as a low priority issue, and while it affects consistency when caching, it's believed that the number of times moxi is combined with Spy|Enyim is probably rare.

Also, a fix is available, but disabled by default.
Comment by Matt Ingenthron [ 10/May/11 ]
Fix is in MB-3798 and it's associated commit, but this bug should remain open since it really describes a different issue. Namely, this bug describes whether or not it should be turned on.
Comment by Steve Yen [ 10/May/11 ]
The right choice seems to be to turn on ketama-weighted by default...

ruby - fauna memcached (libmemcached wrapper) uses ketama-weighted

   :ketama_weighted => true,

For PHP, the default distribution is MODULA distribution. However, from documentation in http://www.php.net/manual/en/memcached.constants.php, the safety tips around ketama hashing...

Memcached::OPT_LIBKETAMA_COMPATIBLE

Enables or disables compatibility with libketama-like behavior. When enabled, the item key hashing algorithm is set to MD5 and distribution is set to be weighted consistent hashing distribution. This is useful because other libketama-based clients (Python, Ruby, etc.) with the same server configuration will be able to access the keys transparently.

Note:

It is highly recommended to enable this option if you want to use consistent hashing, and it may be enabled by default in future releases.
Comment by Matt Ingenthron [ 24/May/11 ]
I would argue against deferring this to 1.7.1. Why?

This will be a change of compatibility when integrated. The way moxi hashes to memcached buckets will change. As pointed out previously, this is probably somewhat okay (given that it's a cache) during an upgrade, but the time to make the change is during a dot-minor version change, not a dot-micro patch update.
Comment by Steve Yen [ 25/May/11 ]
Pulling this back to 1.7.
Comment by Steve Yen [ 25/May/11 ]
http://review.membase.org/6537
Generated at Tue Sep 16 08:41:56 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.