what about lambda architecture, small&fast + fat&batch
in the light of nosql, we know that views are batched. If the buckets are large, the views could lag behind the actual data. While this is acceptable for offline systems, is not acceptable for more reactive system. Imaging setting up an account and have to wait minutes just to login.
At civolution.com we need fat&batch views, but we also would like responsive, uptodate results on the latest data created in the database. Playing around with the stale parameter is not an option, since it falls short of expectations in one way or another (either uptodate but long latency or fast and not uptodate)
Ideally what we would like is a system composed of two components where the fast components would "fill in" the state of the last hours providing the user with fully up-to-date views combining the best of both worlds. In this way we don't have to compromise coherency and stale result with speed.
The other solution would be so move back to a responsive mysql system and leave couchbase as background fat and batch system. But it's definitely not the way I envision using couchbase on the first place.
What are your takes on this?
One thing I'd say is that in the recent builds, the view materialization is much faster. You may find it meets your needs. Check out the recent builds at couchbase.com/downloads-all . Also, keep in mind you'll want to reserve a bit of space outside the quota for filesystem caching of views.
The other thing some have done is used a separate memcached bucket that has objects with indexes that represent recent changes in it, with a short expiration, and they'll grab the view as stale and the recent changes, merging in client logic. This is a slightly bigger burden on the client logic, but provides very high performance.
These are all slightly different than the MVCC consistency MySQL provides though. In the later case, it's why you can get such great performance.
I hope that helps!
For the case you mention, having to wait minutes to login, you should be using key design for this, instead of views, as views aren't necessary for modeling user creation/lookup. Views are incrementally indexed, in the cases where it lags behind, is usually in bulk load data scenarios where influx/flood of data exceeds rate of indexing.
Also, I am unfamiliar with the term Fat & Batch, nor did I find anything on Google, can you explain? I take it you mean quantity of data and how views are indexed?
@scalabl3
Technical Evangelist
Couchbase Inc.