Auto Failover Question
Hello all - first, thanks for membase, great product so far. We're testing it for use in our production environment.
I've read the wiki article on auto failover, and we are able to use the "failover" rest call to properly allow a downed server not to affect keys that were stored there. I also understand why you might not do auto failover due to risks involved.
What is unclear is why, since the membase cluster is aware of a downed server, it doesn't automatically try to fetch a key from a replica who's primary server is down.
I.e. when a server goes down, some number of keys is lost. Until that down server is marked for 'failover', it's keys remain inaccessible. Seems it would make sense to automatically fetch from the replica copy in that case, even if the 'failover' function remains untriggered.
Thanks for any explanation.
Alok
Perhaps this can be made a configuration option - especially for certain types of uses. In our case we are so far using membase only for transient data, so we'd definitely tick the option on the bucket which allows fetching from a replica in a host down situation.
Further, a network split in a typical deployment is far less likely than a simple host down situation, so despite the risk of data becoming inconsistent from the differing views of the topology, you'd probably cause more good than harm from this option.
Thanks for thinking about this.
Yes, I tend to agree with you. We're currently planning a feature to allow only one (or a user-configurable amount) of servers to be automatically failed over. Any servers that go down after that would require administrator intervention.
As for the read-from-replicas, it could be a similar feature. However, the code currently disallows all requests from replicas so it would be a larger code change to implement. I'll forward it on to our product management.
Perry
Hi there Alok.
We've actually been considering this feature for some time now. The main issue is really around ensuring consistency and having ALL reads and writes be serviced from the same place. If the cluster is in a bad state and different nodes have different view of the topology (i.e. network partition) then some clients may be reading the same data from different places.
Overall though it's not a horrible idea and is something that we have on our list of features to add at some point.
Perry
Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!