[MB-7030] Improve performance of view engine for rebalance with consistent views enabled, and make it use less resources (CPU, IO) Created: 26/Oct/12  Updated: 02/Nov/12  Resolved: 01/Nov/12

Status: Resolved
Project: Couchbase Server
Component/s: view-engine
Affects Version/s: None
Fix Version/s: 2.0
Security Level: Public

Type: Improvement Priority: Major
Reporter: Filipe Manana Assignee: Filipe Manana
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The goal here is to make rebalance, with consistent views enabled, faster and consume less CPU and ideally do less IO as well.

This relates to MB-6799.

Each commit will be very specific describing each improvement. Some of them, as a side effect, will also improve rebalance with consistent views disabled.

 Comments   
Comment by Thuan Nguyen [ 26/Oct/12 ]
Integrated in github-couchdb-preview #527 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/527/])
    MB-7030 Always send new group snapshot to parent (Revision 2f17fe571193d00c6c4ee8522710903e3fe7052f)
MB-7030 Use some cheaper BIFs for list and orddicts (Revision a2bef2f295fbc198d9206a2a4d0cc28c025af56e)
MB-7030 Check mail box for new group snapshot (Revision c2dd289640f1097d8a9b285e25004cdaadc07196)
MB-7030 Avoid not useful anymore header commit (Revision a0b3bb69a9a1f0c62a36f622db16cb8c928a1b81)
MB-7030 Don't fsync when doing some header commits (Revision aa361e5de8d129a8244503ee3d0d11bfdef17794)
MB-7030 Don't restart cleanup process (Revision d2eb7d5e4f5cac7f0533bb7c793180fbc67f0447)
MB-7030 Avoid unnecessary group pid lookup (Revision 1a5f6290bc21a37a4eb3bf4be75f4e5c995e05cf)

     Result = SUCCESS
steve :
Files :
* src/couch_set_view/src/couch_set_view_updater.erl

steve :
Files :
* src/couch_set_view/src/couch_set_view_group.erl
* src/couch_set_view/src/couch_set_view_util.erl
* src/couch_set_view/src/couch_set_view_updater.erl

steve :
Files :
* src/couch_set_view/src/couch_set_view_group.erl
* src/couch_set_view/src/couch_set_view_updater.erl

steve :
Files :
* src/couch_set_view/src/couch_set_view_group.erl

steve :
Files :
* src/couch_set_view/src/couch_set_view_group.erl

steve :
Files :
* src/couch_set_view/src/couch_set_view_group.erl

steve :
Files :
* src/couch_set_view/src/couch_set_view.erl
* src/couch_set_view/src/couch_set_view_group.erl
Comment by Steve Yen [ 01/Nov/12 ]
resolving this per bug-scrub mtg
Comment by Filipe Manana [ 02/Nov/12 ]
I thought this was still not close to the goals, that's why it remained open.
Generated at Tue May 21 21:15:03 CDT 2013 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.