Couchbase 2.0 view performance
Hi,
I'm running some performance becnhmarks on Couchbase 2.0 with a view to using it in a high throughput credit card related application.
On a simple 1 node cluster I can get 20,000+ insertions per second for simple JSON records. In total, the test database contains 10 million records. The problem is with retrieval performance.
Insertions are done using a unique transaction reference for the key field. I have set up a view indexed on credit card number to retrieve all transactions for a particular card - typically 10 or fewer. I find that the retrieval rate is around 200 queries per second, or 1% of the insertion rate.
Is this kind of result expected or is there likely to be something wrong with the way I have set up my benchmark test?
I'd posted this reply also over in the other forum for a different product where the same question was accidentally posted:
Which version of the server and which client library are you using and which options are you using when querying? For example, are you using the default for stale, or have you changed it?
Queries are definitely more expensive than sets, since all of the storage work happens after the operation is acknowledged. Index updates also happen a bit later, with the defaults. With what you're doing, if you're looking for non-stale queries, it could very much be the case. One technique here is to have something constantly issuing similar queries on the views so there's not as much update work when you go to query later.
We do have an updated build coming very soon now that I think you'll be pretty happy with. Keep an eye out!
I'm using developer preview 4. I'll wait for the beta release and run the same tests against that.
I've also arranged to talk to someone from the Couchbase solutions architect team to take this further.
It's not the Beta yet, but there is a new build up now and I believe it'll have some enhancements for you.
Please do let us know how things go, of course!
I don't think we have publicly released any benchmarks yet on our view performance so I don't think I can comment on your results at the moment. I would like to mention though that we have made great improvements on view performance since developer preview 4 and we plan on releasing a beta in August (I think) with the new changes.
Also, view results should be expected to be slower than CRUD operations. I have done hundreds of thousands of operations per second on Couchbase and those numbers are very difficult if not impossible to achieve in terms of view requests. We also don't currently cache view queries on the server. In the future we plan on adding that feature and it should provide even more of a performance improvement.