Couchbase Situational Behaviors

Version 7 by ingenthr
on Feb 15, 2013 03:40.

compared with
Current by ingenthr
on Mar 01, 2013 13:52.

This line was removed.
This word was removed. This word was added.
This line was added.

Changes (2)

View Page History
{color:#000000}{*}When the Bootstrap Node for a Client Fails{*}{color}

In this case, there are two differ
If the bootstrap node, which is always the first URI in the list of URIs supplied to the client, happens to fail, the client will walk the list of URIs until it finds a system which can service it's requests.

This is currently implemented two different ways.

In some clients (Java), the persistent failure of operations over a period of time is treated as a reason to distrust the current operating configuration.  This will make the client attempt to re-bootstrap.  

In other clients (.NET), the client will make a small request of all nodes of the cluster on a regular basis to verify connectivity.  If any node is seen as not available, the client will attempt to re-bootstrap.

{color:#000000}{*}Starting a Client When the Cluster is Unreachable{*}{color}

The client will fail to initialize.  The application should handle this condition appropriately.

h2. What is Happening When My Application Sees...

h3. A TMPFAIL Response

A TMPFAIL, (which may also be 0x86 or 134 in particular circumstances) is returned by a Couchbase Server to indicate to the client application that the request couldn't be serviced at this time, but retrying the request at a later date will be successful. Application code do whatever is appropriate in this context. For example, doing a regular backoff/retry until some deadline.

This can arise in a few different situations.

First, this response may be received in response to running out of memory quota for a given bucket. This is frequently in response to heavier than expected or planned for load.

Second, it may be in response to a flush command issued either through the console or the application. During the flush, all operations will be responded to with a TMPFAIL. After the flush completes there should be no more TMPFAILs returned, but as of this writing version 2.0.0 server does return TMPFAIL longer than expected owing to MB-6232/MB-7160.

Third, it may be in response to a bucket being in warmup/pending. This can happen if, for instance, the Couchbase service is restarted on the OS.

In general, whenever appropriate an application should check for and provide error handling for when a TMPFAIL may occur. It is also highly recommended that application developers load/stress test their software to ensure this situation is handled appropriately if workload cannot be predicted. This is one of the primary ways unexpectedly high "pressure" from requests will be handled and it's a difference in Couchbase versus other systems. Couchbase is always fast by design, and gives the application, which has more context about what is appropriate in a particular situation, the ability to react well on behalf of the user rather than just slow down.