Inconsistent View Results

I am running into an issue where query results from a view are inconsistent when using the Java Couchbase Client 1.1. I run the same query directly in a browser and I get consistent results. I have run the query on each of the three nodes in the cluster; debugging shows that each node has returned inconsistent results through the java client, but in a browser the results are consistent for each node.

Here is the map:

function (doc, meta) {
  var data = doc.data;
  if(data) {
    emit([data.company_number, data.status], {
      applicationNumber: data.application_number,
      formId: data.id,
      status: data.status});
  }
}

And the reduce:

function(keys, values, rereduce) {
  var index;
  var maxApp = {applicationNumber:""};
  for (index in values) {
    if (values[index].applicationNumber > maxApp.applicationNumber) {
      maxApp.applicationNumber = values[index].applicationNumber;
      maxApp.formId = values[index].formId;
      maxApp.status = values[index].status;
    }
  }
  return maxApp;
}

The unreduced results are:

{"total_rows":89,"rows":[
{"id":"2b4325a8-0d12-11e3-9728-180373e4f256","key":[30951,"APPROVED"],"value":{"applicationNumber":"30951-00001","formId":"2b4325a8-0d12-11e3-9728-180373e4f256","status":"APPROVED"}},
{"id":"460_848934560","key":[30951,"APPROVED"],"value":{"applicationNumber":"30951-00002","formId":"460_848934560","status":"APPROVED"}}
]
}

Sometimes the java client returns the "30951-00001" application number incorrectly, and other times the "30951-00002" correctly - as if sometimes it doesn't know about the "30951-00002" row.

Any ideas?

The execution path is slightly different for the two, but mostly common. This is a bit surprising. I would recommend adding the debug parameter to each path and compare what the cluster logs on each node during execution of the view if through the browser and through the client library to get more info.

1 Answer

« Back to question.

Hello,

Were you able to identify the source of the problem?
Have you tested with debug enabled?

Regards
Tug
@tgrall

No, I never figured it out or ran it in debug. Because of this problem and because client.getView is slow to open connections, I instead created my own client for getting data from views.

Hmm, surprised by this do you mind sharing the code you have written with us?

Sure. It is basically a client to access the REST API that is available from couchbase.

public class ViewClient {
  private static final String PROTOCOL = "HTTP";
  private static final String METHOD = "GET";
  private static final int PORT = 8092;
  private static final int RESPONSE_RANGE = 100;
  private static final int OK_RANGE = 2;
  private static final String ROWS = "rows";
  private final SerializerFactory serializerFactory;
  private final SystemProperties systemProperties;
  @Logger
  private Log log;
 
  public ViewClient(SerializerFactory serializerFactory, SystemProperties systemProperties) {
    this.serializerFactory = serializerFactory;
    this.systemProperties = systemProperties;
  }
 
  public <T extends Serializable> List<T> getResultList(View view, Query query, Class<T> returnClass) {
    List<URL> urls = loadUrls(view, query);
    try {
      HttpURLConnection connection = loadConnection(urls);
      Iterator<JsonElement> results = getJsonResults(connection);
      return deserializeResults(results, returnClass);
    } catch (IOException e) {
      throw new HcfRuntimeException("Error connecting to couchbase", e);
    }
  }
 
  private <T extends Serializable> List<T> deserializeResults(Iterator<JsonElement> results, Class<T> returnClass) {
    List<T> deserializedResults = new LinkedList<T>();
    Deserializer<T> deserializer = serializerFactory.getDeserializer(returnClass);
    while (results.hasNext()) {
      String rowValue = results.next().getAsJsonObject().get("value").toString();
      T result = deserializer.deserialize(rowValue);
      deserializedResults.add(result);
    }
    return deserializedResults;
  }
 
  private Iterator<JsonElement> getJsonResults(HttpURLConnection connection) throws IOException {
    JsonObject jsonResult = new JsonParser().parse(getResult(connection)).getAsJsonObject();
    JsonArray rows = jsonResult.get(ROWS).getAsJsonArray();
    return rows.iterator();
  }
 
  private List<URL> loadUrls(View view, Query query) {
    List<URL> urls = new LinkedList<URL>();
    for (String host : loadHosts()) {
      String uri = view.generateUri();
      try {
        urls.add(new URL(PROTOCOL, host, PORT, uri + query.toString()));
      } catch (MalformedURLException e) {
        log.warn("Problem loading URL", e);
      }
    }
    return urls;
  }
 
  private List<String> loadHosts() {
    List<String> hosts = new LinkedList<String>();
    for (URI uri : systemProperties.getCouchbaseUris()) {
      hosts.add(uri.getHost());
    }
    return hosts;
  }
 
  private HttpURLConnection loadConnection(List<URL> urls) throws IOException {
    HttpURLConnection connection = null;
    for (URL url : urls) {
      try {
        connection = makeConnection(url);
        break;
      } catch (SocketTimeoutException e) {
        log.warn("Connection to couchbase could not be made. Retrying...", e);
        continue;
      }
    }
    return connection;
  }
 
  private HttpURLConnection makeConnection(URL url) throws IOException {
    HttpURLConnection connection = null;
    connection = (HttpURLConnection) url.openConnection();
    connection.setRequestMethod(METHOD);
    String unencryptedCredentials =
        systemProperties.getCouchbaseUsername() + ":" + systemProperties.getCouchbasePassword();
    String credentials = new String(new Base64().encode(unencryptedCredentials.getBytes()));
    connection.setRequestProperty("Authorization", "Basic " + credentials);
    connection.connect();
    return connection;
  }
 
  private String getResult(HttpURLConnection connection) throws IOException {
    BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
    String line = "";
    StringBuilder sb = new StringBuilder();
    while ((line = reader.readLine()) != null) {
      sb.append(line).append("\r");
    }
    if (responseIsValid(connection.getResponseCode())) {
      return sb.toString();
    } else {
      throw new HcfRuntimeException("Invalid response: [" + connection.getResponseCode() + "] - " + sb.toString());
    }
  }
 
  private boolean responseIsValid(int responseCode) {
    return (responseCode / RESPONSE_RANGE) == OK_RANGE;
  }
}

It should be noted that View is not the CouchbaseClient View class. It just holds the name of the document and view in order to build the path to the view REST resource. I don't think SerializerFactory and SystemProperties need an explanation but if anything does need an explanation let me know.