XDCR: show replication stats on UI

Peter Wansch added a comment - 15/Aug/12 1:13 PM I believe some of the stats will be there for beta.

Thuan Nguyen added a comment - 16/Aug/12 6:40 PM Integrated in github-ns-server-2-0 #437 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/437/ ])      MB-5943 : Restructure XDCR for stats, performance and reliability (Revision a8fad0e67d70d73d570cfd25e46d0167958e3c2c)      Result = SUCCESS Damien Katz : Files : * src/xdc_vbucket_rep.erl * src/concurrency_throttle.erl * src/ns_server_sup.erl * src/xdc_rep_manager.erl * src/xdc_replication_sup.erl * src/xdc_vbucket_rep_sup.erl * src/xdc_rep_utils.erl * src/xdc_rep_notifier.erl * src/xdc_vbucket_rep_worker.erl * src/xdc_vbucket_rep_ckpt.erl * src/xdc_rep_manager_helper.erl * src/xdc_replication.erl * include/xdc_replicator.hrl

Thuan Nguyen added a comment - 18/Aug/12 5:56 PM Integrated in github-ns-server-2-0 #441 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/441/ ])      MB-5943 : Maintain and collect XDCR vb stats (Revision c0e0791c1433eb5d97c4f36704a8159b47b061e1)      Result = SUCCESS Damien Katz : Files : * src/xdc_replication_sup.erl * src/xdc_vbucket_rep_worker.erl * src/xdc_vbucket_rep_sup.erl * src/xdc_vbucket_rep.erl * src/xdc_replication.erl * include/xdc_replicator.hrl * src/xdc_rep_manager.erl * src/xdc_vbucket_rep_ckpt.erl * src/xdc_rep_utils.erl

Steve Yen added a comment - 21/Aug/12 1:58 PM Reopening to get a publically available REST API on this.

Aleksey Kondratenko added a comment - 22/Aug/12 11:25 AM Steve, what is proposed here http://review.couchbase.org/#/c/19993/1/src/menelaus_web.erl,unified is not acceptable for me. My comments there may look strange, but fact is I don't need random API that doesn't take into account what UI needs. Or at least current UI needs. I understand you need this stats badly. So here's proposal. POST this: BucketName = "default", {json, {struct, [{RId, {struct, KVs}} || {RId, KVs} <- xdc_rep_manager:stats(BucketName)]}} to /diag/eval and parse resultant JSON. E.g. wget -O- --user=Administrator --password=asdasd --post-data="BucketName = \"default\", {json, {struct, [{RId, {struct, KVs}} || {RId, KVs} <- xdc_rep_manager:stats(BucketName)]}}" http://lh:9000/diag/eval

Junyi Xie added a comment - 22/Aug/12 7:38 PM Thanks Aaron for taking care of this.

Thuan Nguyen added a comment - 23/Aug/12 5:25 PM Integrated in github-ns-server-2-0 #450 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/450/ ])      MB-5943 Replication stats on UI (Revision 4d4ed0a4bd497aeb66fc36f6b20b9cacb208c10b)      Result = SUCCESS Aliaksey Kandratsenka : Files : * src/menelaus_stats.erl * src/xdc_replication.erl * src/stats_collector.erl

Ketaki Gangal added a comment - 08/Sep/12 1:06 PM Not seeing "Deletes" on the destination cluster UI stats. on build 1697

Farshid Ghods added a comment - 08/Sep/12 1:08 PM is this a UI only issue ?

Junyi Xie added a comment - 08/Sep/12 4:47 PM Ketaki, In "XDCR Destination" section, there is a delete stat figure, can you verify that we do not see XDCR delete stat in that section?

Junyi Xie added a comment - 08/Sep/12 7:22 PM The deletion ops from XDCR is shown at XDCR Destination section but not summary section.

Ketaki Gangal added a comment - 09/Sep/12 3:38 PM On xdc stats on destination today,show - 1. XDC ops/sec- seen as expected 2. XDC sets/sec- seen as expected 3. XDC deletes/sec - seen as expected The summary UI 1. Creates - show xdc creates 2. Updates - show xdc updates 3. Deletes - does not show any xdc deletes. IMO this is very confusing for the users to understand that data is being deleted, but not displayed. For the destination if the incoming data via xdc is same as any other data from any other clients, then all creates/updates/deletes should be consistently displayed. Here, the xdc deletes/sec does show deletes, but this needs to also be reflected on the aggregate stats.

Aaron Miller added a comment - 09/Sep/12 4:12 PM ep_num_ops_del_meta, which is the stat used to show "XDC deletes/sec" is included in the sum stat that is used in the summary UI, so the code as I read it -should- be doing this. Not sure what's going on here... https://github.com/couchbase/ns_server/blob/master/src/stats_collector.erl#L286 https://github.com/couchbase/ns_server/blob/master/src/menelaus_stats.erl#L1103

Junyi Xie added a comment - 09/Sep/12 4:19 PM Stats in XDC sections are all correct. In Summary UI, however, the Delete stat in Summary UI comes from stat delete_hits, which seems to me maintained in bucket_engine, instead of ep_engine (not sure why it is this way, since update and create on Summary UI are from ep_engine) . Ketaki, You may want to assign to bucket engine owner to take a look.

Peter Wansch added a comment - 12/Sep/12 1:39 PM We want to pick up the delete stats from ep_engine in the summary like we do for the other XDC stats.

Dipti Borkar added a comment - 12/Sep/12 4:37 PM UI part for XDCR stats is done. cleanup of text in the works as well. ep_engine issue should be tracked separately

Thuan Nguyen added a comment - 17/Sep/12 9:44 PM Integrated in github-ns-server-2-0 #468 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/468/ ])      MB-5943 : Add xdcr stats for work and commit times (Revision 7ae1c343c58e31ddf39d61e6c844b50ccb4bfc36)      Result = SUCCESS pwansch : Files : * src/xdc_replication.erl * src/ns_doctor.erl * src/xdc_vbucket_rep.erl * src/stats_collector.erl * include/xdc_replicator.hrl * src/menelaus_stats.erl