What happens when a node crashes with replication enabled? Not clear from docs.
I've been reading the forums and documentation trying to find an answer to this:
1/ Three nodes in a cluster;
2/ Replication set to "1".
3/ One of the nodes crashes.
On the one hand it seems that some forum posts and documentation pages specify that the data contained in the failed node becomes unavailable until either a manual or automated failover is initiated (minimum 30 seconds it would seem).
On the other hand I'm reading some documentation which states that clients will almost *immediately* become aware of the crashed node and begin to use the replica data as though nothing happened.
Please clear this up for me. I can't make a decision on whether to go live with Couchbase until then.
Regards
Bueller? Bueller?
Hello,
I asked myself the same question and it seems that you have to fail over the crashed server. This is done after a minimum of 30 seconds with automatic fail over activated.
Thus no 100% availability is possible unless you code some kind of fail over application with the APIs to fail over and rebalance your cluster.
Regards
That's what I thought. The website for couchbase is rather deceptive. :)
From the "Architecture / Cluster Server" section of the server web pages:
"In the event of a server failure, the cluster manager detects that a node is down, promotes replica documents to active status, and recomputes the cluster map so that the client application can continue operating without any downtime."
Not true. There will always be *at least* 30 seconds of downtime. While it is only the failed node's data that is unreachable, due to the nature of consistent hashing and the copious use of getMulti on web-scale sites a failed node will usually take down an entire web site.
This is why I was confused at first. On the one hand the couch folks explain that they aim for the CP of "CAP" (i.e. sacrifice availability), but on the other hand they put documentation online which speaks of "no downtime."
That being said, I'm happy with the caching side of the server -- the couchdb (i.e. views) side is close to useless for real-time production information. It has inconsistencies, crashes the servers regularly and is an *absolute nightmare* to deal with during rebalancing.
Anyone? :)
Further: I have a cluster of three nodes with replica = 1 (i.e. two copies of data). When I take one of the nodes down:
1/ smart clients cannot retrieve data which was on the downed node (that is to say, the "replica" vBuckets aren't marked as active by the cluster controller);
2/ attempting to set the downed node to "fail over" always generates an "incomplete replica" warning message -- even if the system has entirely replicated.