NoSQL Data Modeling Confusion

I'm not an expert with setting up Couchbase views (used for querying) yet so I'm going to let someone else answer those questions, but I do want to address you questions about Spymemcached. First off, you are correct that Membase is a key/value store, but Couchbase is a json document store. Since Couchbase is based on CouchDB you can read here for a short intro to what documents are and how they are queried (http://couchdb.apache.org/docs/intro.html). This link will also answer your question about how queries are accomplished.

In Spymemcached there is a way to do Queries. My guess is that you probably don't have the latest version of Spymecached. You will need at least version 2.8 (Note that at the moment we only offer developer preview of spy 2.8). Check out the tutorial for Spymemcached to see how to run queries on view http://docs.couchbase.org/couchbase-sdk-java-tutorial/index.html. You will have to scroll down to the end of the document for information on using Spymemcached with Couchbase Server.

CAS is a check and set operation which means "store this data but only if no one else has updated since I last fetched it." When doing certain calls to memcached you can get a CAS value with is a 64 bit number. Whenever a value is changed its CAS is incremented. So when you do a CAS you put the last CAS value you got for that key into the message and memcached will only change the value of the corresponding key if no one else has changed the key already.

Couchbase is very well suited for the scenario you described. Depending on what other Group related meta-data you want to store, you could might be able to get away with just a database of user documents that reference a restricted set of group names:

{"_id": "username", "password": "passw0rd", "groups":["accounting", "marketing", "engineering"]}

{"_id": "otheruser", "password": "passw0rd", "groups":["engineering"]}

In the documents above I've dropped a numeric or UUID-based ID in favor of using the username as the ID. That's not a requirement, but certainly makes things simpler.

The above docs alone satisfy your first two queries with simple GET (thinking CouchDB's HTTP API here) requests:
- search for user by ID or by username (as they're the same now).

It also satisfies the "get all groups for a user" query.

The remaining two queries ("search for group by ID" and "get all users for a group") can be satisfied by a single map/reduce query:

Map:

function(doc) {
  if (doc.groups && doc.groups.length > 0) {
    doc.groups.forEach(function(group) {
      emit(group, doc._id);
    });
  }
}

Reduce:
_count
This "_count" reducer is built-in and calls an Erlang-based counting function, so it's much faster than writing the same thing in JS.

The output of the above map would be a single line per group with the username associated to the group name:

[{"key":"accounting", "value":"username"},
 {"key":"engineering", "value":"otheruser"},
 {"key":"engineering", "value":"username"},
 {"key":"marketing", "value":"username"}]

The full reduction of this query would return:

[{"key": null, "value": 4}]

We can also do grouped reduction which is far more useful in this case. You'd get this result by adding "?group=true" to the _view request:

[{"key": "accounting", "value": 1},
 {"key": "engineering", "value": 2},
 {"key": "marketing", "value": 1}]

[{"key": "engineering", "value": 2}]

I hope that answered some of your questions. Even if your documents are a bit more complex than these, the approach should be repeatable on a wide range of scenarios.

Hope that helps. If not, keep asking questions. :)

Thanks,
Benjamin