performance issues, can't find bottleneck
I need some guidance in isolating the bottlenecks on my Couchbase installation (2.0.0?).
Setup:
Single node cluster (desktop): Win2008R2 SP1, 16GB, 8 core (Intel), 3.4GHz, single SSD. Couchbase Ram/Quote Usage: 51.8MB/7.52GB (Persistence Enabled).
Test client (desktop): Win2008 SP2, 4GB, 2 core (Intel), 3.0GHz, .NET SDK v1.2, .NET runtime v4.0.
The desktops are connected to the same (sad) 10/100 switch, while I wait for a Gb switch (more on that later).
Test scenario:
Individually Set (ExecuteStore()) 48000 documents, averaging 350 characters of string data, with a 16 character key. Then read the same set back out (individual Gets). Test is single threaded, instantiates CouchBaseClient once, loops through set/get scenario, and then disposes everything.
Results:
At first I was getting only 30 ops/sec, and then I discovered that nLog was killing me with verbose logging. After I disabled logging I got ~1100 ops/sec for both read and write. I expected to be limited by my network, but it seems to be loping along at 5% utilization. The test set completes with zero failures (it's a tiny set after all).
If I just knew what perfmon counters to examine, and what to expect, I could probably find that one component that is maxed out. Can someone point me in the right direction?
[edit]This is just a POC performance test to help narrow down the NOSQL options my company is considering adopting. If we go live we'll be hosting CB on Linux, not Windows, but Windows is what I know and what I have to work with at the moment. The client will remain .NET.[/edit]
I found part of my problem: simple latency combined with synchronous calls. Spinning up additional threads scaled almost linearly up to ~9, yielding 10,000 ops/sec peak. Even with this sad hardware I've still got a ways to go, but at least the numbers are starting to look respectable.