[MB-4937] ep-engine provides disk fragmentation statistics Created: 22/Mar/12  Updated: 14/May/12  Resolved: 03/Apr/12

Status: Resolved
Project: Couchbase Server
Component/s: None
Affects Version/s: 1.7.2, 1.8.0
Fix Version/s: 1.8.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Dipti Borkar Assignee: Chiyoung Seo
Resolution: Fixed Votes: 0
Labels: 1.8.1-release-notes
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, there is no easy way to understand the disk fragmentation of SQLite. Expose this stat that should be calculated in ep_engine and exposed via REST API and the UI to better understand SQLite fragmentation over time so that users can understand when maintenance of the system using swap rebalance, vacuuming or other methods is required.

 Comments   
Comment by Steve Yen [ 03/Apr/12 ]
ns_server can grab histogram from ep-engine and calculate average.
Comment by Steve Yen [ 03/Apr/12 ]
to add usage notes
Comment by Chiyoung Seo [ 03/Apr/12 ]
The stat subkey is "kvtimings". For example, the following shows how we can grab read/write seek histograms from the underlying kvstore:

Chiyoung-MacBook:ep-engine chiyoung$ ./management/cbstats 127.0.0.1:11210 kvtimings
 readSeek (23 total)
    0 - 2 : ( 21.74%) 5 ###########################
    8 - 16 : ( 26.09%) 1 #####
    64 - 128 : ( 47.83%) 5 ###########################
    256 - 512 : ( 73.91%) 6 #################################
    512 - 1KB : ( 78.26%) 1 #####
    1KB - 2KB : ( 95.65%) 4 ######################
    2KB - 4KB : (100.00%) 1 #####
 writeSeek (77 total)
    4 - 8 : ( 28.57%) 22 ####################################
    8 - 16 : ( 36.36%) 6 #########
    16 - 32 : ( 57.14%) 16 ##########################
    256 - 512 : ( 61.04%) 3 ####
    512 - 1KB : ( 84.42%) 18 #############################
    1KB - 2KB : ( 97.40%) 10 ################
    2KB - 4KB : (100.00%) 2 ###
Comment by Dipti Borkar [ 13/May/12 ]
Chiyoung, can you provide some guidelines about what these mean, and how to interpret them so that we can add to the documentation.
Comment by Chiyoung Seo [ 14/May/12 ]
These timings show the distribution of disk head seek distance that have been performed on every read / write operation on Sqlite. We expect that these seek distance would be increased over the time as the sqlite is getting fragmented more and more.
Generated at Wed Aug 27 06:48:39 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.