Auto fail over seems to be node dependant?
Im seeing some inconsistent behavior with auto failover, and wondered if anyone can explain whats going on. Here's my setup:
Ive got (in a testing environment) 3 nodes running ubuntu precise & couchbase 1.8.1 configured 1 bucket and 1 replica. I Also have a webserver running moxi, which is running a couple of small test php scripts (using the memcache library). One script sets 2000 keys to known values. Key1 =1, Key2 = 2 and so on, the other script gets those values, and checks that the result it gets back is correct. Im testing autofail over, which is enabled and set at 30 seconds.
Moxi has the address of all three nodes in its moxi-cluster.cfg.
The method im using for "downing" any node is to stop the couchbase service using the init.d script, or service couchbase-server stop if you're on redhat.
So here's what I'm seeing:
If I stop the service on node1, then no data is returned by the get script, until after (around) 30 seconds, when the auto fail over is performed. After this point, all data is accessible. Which is what I would expect.
If I stop the service on node3, then no data is return by the get script, EVEN AFTER the auto fail over has occurred.
Anyone any ideas as to why this might be happening?
Thanks in advance!