[MB-6096] Index building is seemingly taking too much ram Created: 02/Aug/12  Updated: 31/Jan/14  Resolved: 16/Aug/12

Status: Resolved
Project: Couchbase Server
Component/s: performance, storage-engine, view-engine
Affects Version/s: 2.0
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Aleksey Kondratenko Assignee: Filipe Manana
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I have a crash dump that shows that simple index build over 1.2 million of small docs eats tons of ram. I'm going to attach erlang crash dump where you can see second biggest process (0.7277.2 or sort by stack+heap) is index updater.

NOTE: crash itself was caused by attempt to grab diag so biggest process died because of that, but real cause seems to be index building.

 Comments   
Comment by Aleksey Kondratenko [ 02/Aug/12 ]
Crash dump can be downloaded here: https://docs.google.com/open?id=0B6NtGsLhIcf7QVpwVldNcVlZdkk
Comment by Thuan Nguyen [ 10/Aug/12 ]
Integrated in github-couchdb-preview #474 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/474/])
    MB-6096 Fix too high memory consumption during indexing (Revision c8003e55b9dc8edbee51e5d047c0317aaf192560)

     Result = SUCCESS
Filipe David Borba Manana :
Files :
* src/mapreduce/mapreduce.h
* src/couchdb/couch_util.erl
* src/couchdb/couch_db_updater.erl
* src/mapreduce/mapreduce.cc
* src/couch_set_view/src/couch_set_view_util.erl
* src/couchdb/couch_doc.erl
* src/couchdb/couch_btree.erl
* src/couch_set_view/src/couch_set_view_updater.erl
* src/mapreduce/mapreduce_nif.cc
* src/mapreduce/mapreduce.erl
Comment by Thuan Nguyen [ 13/Aug/12 ]
Integrated in github-couchdb-preview #475 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/475/])
    MB-6096 Use enif_alloc()/enif_free() in ejson decoder (Revision 2c4b88c46e1ce4500d987552e600eaa200546fe6)

     Result = SUCCESS
Filipe David Borba Manana :
Files :
* src/ejson/decode.c
Comment by Filipe Manana [ 16/Aug/12 ]
Re-opened to assign to myself, since I did all the investigation and code work
Comment by Filipe Manana [ 16/Aug/12 ]
Closing again.
Generated at Tue Apr 15 23:54:54 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.