Skip to end of metadata
Go to start of metadata
You are viewing an old version of this page. View the current version. Compare with Current  |   View Page History

Official information may be found at the Java Client Library page.

Other Languages on the JVM

Michael Nitschinger has written a blog on Accessing Couchbase from Scala

Handling Timeouts

The Java client library has a set of synchronous and asynchronous methods. While it doesn't happen in most situations, occasionally network IO can become congested, nodes can fail, or memory pressure can lead to situations where an operation can timeout.

When this timeout occurs, most of the synchronous methods on the client will return a RuntimeException showing a timeout as the root cause. Since the asynchronous operations give finer grained control over how long is given for an operation to be successful or unsuccessful, this path throws a checked TimeoutException.

As an application developer, it's best to think about what you would do after this timeout. This may be something like showing the end user a message, it may be nothing, or it may be going to some other system for additional data.

In some cases, it may make sense to retry the operation, but this should be thought about carefully, as the cause of the timeout may be exacerbated. In the case of deciding to retry, doing this with a backoff or exponential backoff (see bulk loading below for an example) is advisable. This can be conceptually thought of as a pressure relief valve for intermittent resource contention.

Bulk Loading

When doing bulk loading of Couchbase Server, one can frequently overwhelm available memory in the Couchbase cluster before it can store data on disk. When this happens, Couchbase Server will immediately send a response back indicating the operation cannot be handled at the moment, but likely can be handled later. This is sometimes referred to as "handling Temp OOM" (where OOM means out of memory), though the actual temporary failure could be sent back for reasons other than OOM. Temporary OOM is the most common, however.

There are plans for future client support to handle this automatically, but handling it in your application is really a matter of a simple do-while.

One example of performing bulk loading when receiving temporary failures is:

StoreHander.java
package com.couchbase.sample.dataloader;

import com.couchbase.client.CouchbaseClient;
import java.io.IOException;
import java.net.URI;
import java.util.List;
import net.spy.memcached.internal.OperationFuture;
import net.spy.memcached.ops.OperationStatus;

/**
 *
   * The StoreHandler exists mainly to abstract the need to store things
   * to the Couchbase Cluster even in environments where we may receive
   * temporary failures.
 *
 * @author ingenthr
 */
public class StoreHandler {

  CouchbaseClient cbc;
  private final List<URI> baselist;
  private final String bucketname;
  private final String password;

  /**
   *
   * Create a new StoreHandler.  This will not be ready until it's initialized
   * with the init() call.
   *
   * @param baselist
   * @param bucketname
   * @param password
   */
  public StoreHandler(List<URI> baselist, String bucketname, String password) {
    this.baselist = baselist; // TODO: maybe copy this?
    this.bucketname = bucketname;
    this.password = password;


  }

  /**
   * Initialize this StoreHandler.
   *
   * This will build the connections for the StoreHandler and prepare it
   * for use.  Initialization is separated from creation to ensure we would
   * not throw exceptions from the constructor.
   *
   *
   * @return StoreHandler
   * @throws IOException
   */
  public StoreHandler init() throws IOException {
    // I prefer to avoid exceptions from constructors, a legacy we're kind
    // of stuck with, so wrapped here
    cbc = new CouchbaseClient(baselist, bucketname, password);
    return this;
  }

  /**
   *
   * Perform a regular, asynchronous set.
   *
   * @param key
   * @param exp
   * @param value
   * @return the OperationFuture<Boolean> that wraps this set operation
   */
  public OperationFuture<Boolean> set(String key, int exp, Object value) {
    return cbc.set(key, exp, cbc);
  }

  /**
   * Continuously try a set with exponential backoff until number of tries or
   * successful.  The exponential backoff will wait a maximum of 1 second, or
   * whatever
   *
   * @param key
   * @param exp
   * @param value
   * @param tries number of tries before giving up
   * @return the OperationFuture<Boolean> that wraps this set operation
   */
  public OperationFuture<Boolean> contSet(String key, int exp, Object value,
          int tries) {
    OperationFuture<Boolean> result = null;
    OperationStatus status;
    int backoffexp = 0;

    try {
      do {
        if (backoffexp > tries) {
          throw new RuntimeException("Could not perform a set after "
                  + tries + " tries.");
        }
        result = cbc.set(key, exp, value);
        status = result.getStatus(); // blocking call, improve if needed
        if (status.isSuccess()) {
          break;
        }
        if (backoffexp > 0) {
          double backoffMillis = Math.pow(2, backoffexp);
          backoffMillis = Math.min(1000, backoffMillis); // 1 sec max
          Thread.sleep((int) backoffMillis);
          System.err.println("Backing off, tries so far: " + backoffexp);
        }
        backoffexp++;

        if (!status.isSuccess()) {
          System.err.println("Failed with status: " + status.getMessage());
        }

      } while (status.getMessage().equals("Temporary failure"));
    } catch (InterruptedException ex) {
      System.err.println("Interrupted while trying to set.  Exception:"
              + ex.getMessage());
    }

    if (result == null) {
      throw new RuntimeException("Could not carry out operation."); // rare
    }

    // note that other failure cases fall through.  status.isSuccess() can be
    // checked for success or failure or the message can be retrieved.
    return result;
  }
}

This code sample originally appeared in this gist by Matt Ingenthron.

Configuring Logging

Occasionally when troubleshooting an issue with a clustered deployment, it may be required to gather additional information from the Couchbase Java Client's logging. It uses JDK logging and this can be configured by configuring a runtime define and adding some additional logging properties.

Configuring Logging via Properties

For example, if running a command line java program, run with

-Djava.util.logging.config.file=logging.properties

...or put a logging.properties in your classpath.

logging.properties
handlers = java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level = ALL
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
com.couchbase.client.vbucket.level = FINEST
com.couchbase.client.vbucket.config.level = FINEST
com.couchbase.client.level = FINEST

Other methods of enabling the additional logging may be appropriate depending on the deployment.

Since the Couchbase Java Client uses spymemcached and is compatible with spymemcached (we wrote it too, we like it!), if you want to set its built in logging to either JDK standard logging or log4j, simply use

-Dnet.spy.log.LoggerImpl=net.spy.memcached.compat.log.SunLogger

or

-Dnet.spy.log.LoggerImpl=net.spy.memcached.compat.log.Log4JLogger

Configuring Logging Programmatically

Sometimes, if you're writing in an IDE which is handling all the command line stuff for you, you just want to express your logging in code. That's possible too. Just add a section like this to your code:

        // Tell things using Spy's logging to use the SunLogger compat
        Properties systemProperties = System.getProperties();
        systemProperties.put("net.spy.log.LoggerImpl", "net.spy.memcached.compat.log.SunLogger");
        System.setProperties(systemProperties);

        Logger.getLogger("net.spy.memcached").setLevel(Level.FINEST);
        Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST);
        Logger.getLogger("com.couchbase.client.vbucket").setLevel(Level.FINEST);

        //get the top Logger
        Logger topLogger = java.util.logging.Logger.getLogger("");

        // Handler for console (reuse it if it already exists)
        Handler consoleHandler = null;
        //see if there is already a console handler
        for (Handler handler : topLogger.getHandlers()) {
            if (handler instanceof ConsoleHandler) {
                //found the console handler
                consoleHandler = handler;
                break;
            }
        }

        if (consoleHandler == null) {
            //there was no console handler found, create a new one
            consoleHandler = new ConsoleHandler();
            topLogger.addHandler(consoleHandler);
        }

        //set the console handler to fine:
        consoleHandler.setLevel(java.util.logging.Level.FINEST);

Loading Views Programatically

Currently, with Couchbase Server 2.0 loading views is done through the web console, which gives the developer a good way to iterate quickly with view development.

However, once the view is developed, it's convenient to load any views programmatically and store them in a source control repository alongside other application source code. Since loading views is simply a matter of an HTTP request, they can be loaded with the command line using curl or even a simple Java program.

For example:

ViewLoader.java
package com.couchbase.sample.designdlocloader;

import java.io.IOException;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;

public class App {

  public static void main(String[] args) throws IOException {
    String view = "{\"language\":\"javascript\",\"views\":{\""
            + "dev_testview\":{\"map\":\"function (doc) {  "
            + "emit(doc._id, 1)}\",\"reduce\":\"_sum\" }}}";

    System.out.println("Result of load is " + loadDesignDoc(view, "default",
      "dev_testview")); // do not use leading dev_ if you want a full cluster dataset view
  }

  public static boolean loadDesignDoc(String doc, String bucketname,
        String viewname) throws IOException {

    HttpClient httpclient = new DefaultHttpClient();
    HttpPut httpput = new HttpPut("http://localhost:8092/" + bucketname
      + "/_design/" + viewname); // use any one node in your cluster

    StringEntity reqEntity = new StringEntity(doc);


    httpput.setEntity(reqEntity);

    HttpResponse response = httpclient.execute(httpput);

    System.out.println("View loading result is "  + response.getStatusLine());

    if (response.getStatusLine().getStatusCode() < 300) {
      return true;
    } else {
      return false;
    }
  }
}

Retrying When Receiving a Tempfail

With Couchbase Server, if you overwhelm the amount of memory from workload, rather than become slower, Couchbase will send back temporary failures indicating you can retry the operation.  This gives the software a bit more control than with other database systems, where they become slower for all operations under load.  It is a "relief valve" of sorts, and gives you as an app developer the opportunity to backoff and retry operations if appropriate in your application logic.

Couchbase Exponential Backoff
package com.couchbase.sample.dataloader;

import com.couchbase.client.CouchbaseClient;
import java.io.IOException;
import java.net.URI;
import java.util.List;
import net.spy.memcached.internal.OperationFuture;
import net.spy.memcached.ops.OperationStatus;

/**
 *
   * The StoreHandler exists mainly to abstract the need to store things
   * to the Couchbase Cluster even in environments where we may receive
   * temporary failures.
 *
 * @author ingenthr
 */
public class StoreHandler {

  CouchbaseClient cbc;
  private final List<URI> baselist;
  private final String bucketname;
  private final String password;

  /**
   *
   * Create a new StoreHandler.  This will not be ready until it's initialized
   * with the init() call.
   *
   * @param baselist
   * @param bucketname
   * @param password
   */
  public StoreHandler(List<URI> baselist, String bucketname, String password) {
    this.baselist = baselist; // TODO: maybe copy this?
    this.bucketname = bucketname;
    this.password = password;


  }

  /**
   * Initialize this StoreHandler.
   *
   * This will build the connections for the StoreHandler and prepare it
   * for use.  Initialization is separated from creation to ensure we would
   * not throw exceptions from the constructor.
   *
   *
   * @return StoreHandler
   * @throws IOException
   */
  public StoreHandler init() throws IOException {
    // I prefer to avoid exceptions from constructors, a legacy we're kind
    // of stuck with, so wrapped here
    cbc = new CouchbaseClient(baselist, bucketname, password);
    return this;
  }

  /**
   *
   * Perform a regular, asynchronous set.
   *
   * @param key
   * @param exp
   * @param value
   * @return the OperationFuture<Boolean> that wraps this set operation
   */
  public OperationFuture<Boolean> set(String key, int exp, Object value) {
    return cbc.set(key, exp, cbc);
  }

  /**
   * Continuously try a set with exponential backoff until number of tries or
   * successful.  The exponential backoff will wait a maximum of 1 second, or
   * whatever
   *
   * @param key
   * @param exp
   * @param value
   * @param tries number of tries before giving up
   * @return the OperationFuture<Boolean> that wraps this set operation
   */
  public OperationFuture<Boolean> contSet(String key, int exp, Object value,
          int tries) {
    OperationFuture<Boolean> result = null;
    OperationStatus status;
    int backoffexp = 0;

    try {
      do {
        if (backoffexp > tries) {
          throw new RuntimeException("Could not perform a set after "
                  + tries + " tries.");
        }
        result = cbc.set(key, exp, value);
        status = result.getStatus(); // blocking call, improve if needed
        if (status.isSuccess()) {
          break;
        }
        if (backoffexp > 0) {
          double backoffMillis = Math.pow(2, backoffexp);
          backoffMillis = Math.min(1000, backoffMillis); // 1 sec max
          Thread.sleep((int) backoffMillis);
          System.err.println("Backing off, tries so far: " + backoffexp);
        }
        backoffexp++;

        if (!status.isSuccess()) {
          System.err.println("Failed with status: " + status.getMessage());
        }

      } while (status.getMessage().equals("Temporary failure"));
    } catch (InterruptedException ex) {
      System.err.println("Interrupted while trying to set.  Exception:"
              + ex.getMessage());
    }

    if (result == null) {
      throw new RuntimeException("Could not carry out operation."); // rare
    }

    // note that other failure cases fall through.  status.isSuccess() can be
    // checked for success or failure or the message can be retrieved.
    return result;
  }
}
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.