Need a toy build to measure breakdown of persistence w/ xdcr

Jin Lim added a comment - 19/Mar/13 4:23 AM - edited Getting histogram for disk drain and write queue is not possible. I will explain to Abihinav why not. Even if we hacked the code, accrued values won't be accurately represent histograms of what we are looking for. Better just get average from UI.

Jin Lim added a comment - 19/Mar/13 4:32 AM Materia for a toy build is ready here: http://review.couchbase.org/#/c/25067/ get_with_meta now includes time spent for bgfetch set_with_meta now includes time spent for disk write del_with_meta fixed

Jin Lim added a comment - 19/Mar/13 4:33 AM Junyi - please pick up the above commit and kick off another toy build for Abihinav. Please let me know if you have any toy build issue. I will take care of it myself if any. Thanks.

Junyi Xie added a comment - 19/Mar/13 11:23 AM Jin, Thanks for commit. I looked at your code but I am not sure this is what we want. From Abhinav's previous test, >95% of latency is spent on source persistent time, that is, how long it takes an item to reach disk from memory, at source side. We want to repeat the test and verify it. The histogram stats you added in your commit are get_with_meta/set_with_meta/del_with_meta, which are OK but neither of them can measure the persistence time at source. In your code you measure the persistence time of set_with_meta, which actually we do not care much about it at destination side. This is because as long as the item reaches memory at destination, it is enough for us. If there is any existent ep_engine stat which can measure the persistent time from within ep_engine, please let me know. I would like to verify it locally to make sure the stat is valid and trustable, before making a toybuild for Abhinav. Thanks!

Junyi Xie added a comment - 19/Mar/13 11:23 AM Please see my comment

Jin Lim added a comment - 19/Mar/13 11:41 AM Thanks Junyi, if you looking at the action items from our previous discussion, we were talking about collecting setWithMeta latency (with disk write) and set_with_meta is now exactly doing that. You are suggesting that you need set latency (with disk write) as well?

Junyi Xie added a comment - 19/Mar/13 11:50 AM Jin， It was my fault that I did not realize this when reading your action items in previous email. I think for this particular test, we do need set latency (with disk write) as well, for the set/get/del_with_met stats you made in your commit, I think they are good and we should merge them into master, if Chiyoung is Ok with them.

Jin Lim added a comment - 19/Mar/13 12:05 PM No problem Junyi, it was just a question. So do you & Abhinav need set latency with disk write for another round of testing? If so I can quickly add the stat. Otherwise, please go ahead and build a toy build that includes your optimizations and my latest commit. EP Engine won't add these stats for the production code. I will explain later why. Thanks.

Junyi Xie added a comment - 19/Mar/13 12:22 PM Yes we need set latency with disk write for another round of testing.

Jin Lim added a comment - 22/Mar/13 12:40 PM Toy build has been provided and estimated number are being investigated.

Maria McDuff added a comment - 22/Apr/13 6:37 PM abhinav, ketaki, what is the update to this bug running the toy build?