[MB-4846] high disk write queue becomes very slow to drain if user is running views ( indexing is in progress ) Created: 28/Feb/12  Updated: 10/Jan/13  Resolved: 30/May/12

Status: Closed
Project: Couchbase Server
Component/s: couchbase-bucket
Affects Version/s: 2.0-developer-preview-4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Tommie McAfee Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: 2.0-dev-preview-4-release-notes, eperf
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: dp4 717
3 node cluster, 8 gb each

Attachments: PNG File 30min_slow_drain.png    

 Description   
Loaded 5 million docs using testrunner eperf: performance.eperf.EPerfClient.test_ept_mixed_original -p prefix=1,num_clients=1,batch=100,load_phase=1,access_phase=0,items=5000000,size=2,cur_items=3249129

Then the following to create/read/update/delete docs:
performance.eperf.EPerfClient.test_ept_mixed_original -p batch=1000,load_phase=0,access_phase=1,ratio_hot_gets=1.0,ratio_hot_sets=1.0,ratio_hot=0.5,ratio_misses=0.0,kind=nonjson,items=5000000,max_creates=5000000,num_clients=1,prefix=1,start_delay=0,size=2

(about 8k/sec), I noticed the write queues on servers would start to climb and ops/sec drop, with drain rate as low as 200 docs/sec. Doesn't seem to be timeout/retry behavior as know issue- MB-4693 as docs are draining, but at a very slow rate. (see screen attached) - will add diags if repro'd

Documenting behavior for DP4 as this code is being replaced.

 Comments   
Comment by Chiyoung Seo [ 30/May/12 ]
This is somehow an expected behavior because there is a high contention on disk between the ep-engine flusher and view indexer.
Generated at Thu Oct 02 04:12:42 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.