[MB-8319] [Doc'd] View compaction is triggered during rebalance when auto-compaction is disabled Created: 20/May/13  Updated: 20/Feb/14  Resolved: 20/Feb/14

Status: Closed
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 2.0.1, 2.1.0
Fix Version/s: 3.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Pavel Paulau Assignee: Anil Kumar
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Centos 64-bit

 Description   
Steps:
1. Set fragmentation threshold to 100% (both data and index)
2. Start workload, view fragmentation is high, >90%.
3. Rebalance cluster

As result view compaction kicks in. Bucket compaction doesn't kick in.

Not sure if it's a bug but behavior is very confusing. At least I didn't find documentation that explains it.

Diags for 2.0.2, just in case:
http://172.23.96.10:8080/view/views/job/apollo-views/116/artifact/

Basically I just want to run a test with no view compaction.

 Comments   
Comment by Aleksey Kondratenko [ 20/May/13 ]
The reason is that it's known that cleanup vbuckets (those that we're removing from index) are not properly accounted for in index fragmentation. After background cleanup is complete that stuff will be deleted (causing more fragmentation) after which compaction will need another pass. So our logic is to do view compaction after certain number of incoming or outgoing moves from certain nodes regardless of anything. This can be disabled via internal settings.
Comment by Aleksey Kondratenko [ 20/May/13 ]
I have already replied above. Let me know if that's not enough.
Comment by Pavel Paulau [ 21/May/13 ]
could we create a separate task for documentation and keep this open? imho we should change this behavior eventually.
Comment by Aleksey Kondratenko [ 29/May/13 ]
Eventually things (especially indexes during rebalance) will be a totally different approach. So imho not worth keeping this open.
Comment by kzeller [ 25/Jun/13 ]
Added to RN and cross ref to REST chapter:

Indexing and Querying

If you rebalance a cluster, index compaction starts but bucket compaction will not start. This is because vBucket cleanup from an index is not accounted for in index fragmentation. After this cleanup process completes it will cause more index fragmentation and therefore we run compaction again. Index compaction will therefore always run after a certain number of changes to vBuckets on nodes. You can change this setting using the REST-API, see Section 8.7.7, “Adjusting Rebalance during Compaction”.

Issues: MB-8319



REST Chapter content:

http://www.couchbase.com/docs/couchbase-manual-2.1.0/couchbase-admin-restapi-rebalance-before-compaction.html

Removing "doc" component until future release.
Comment by Aleksey Kondratenko [ 12/Aug/13 ]
I'm not sure at all it's a bug and thus worth fixing
Generated at Wed Oct 01 05:45:58 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.