How to Collect Cluster Manager Metrices

Couchbase Server
1

Hi Team,

While going through below 7.6 CB release notes there are new metrices avliable for to check failover and rebalance. Could you please advise on the utility we can use to capture these metrics?

https://docs.couchbase.com/server/current/introduction/whats-new.html#cluster-manager

Thanks,
Debasis

2

In the release note -

And the first line of that is

The following Cluster Manager metrics can be queried by means of the REST APIs described in Statistics.

Which shows the urls.

For example -

http://localhost:8091/pools/default/stats/range/cm_auto_failover_enabled

Gives:


{
  "data": [
    {
      "metric": {
        "nodes": [
          "[::1]:8091"
        ],
        "instance": "ns_server",
        "name": "cm_auto_failover_enabled"
      },
      "values": [
        [1722622069, "1"
        ],
        [1722622079, "1"
        ]
      ]
    }
  ],
  "errors": [],
  "startTimestamp": 1722622019,
  "endTimestamp": 1722622079
}

A utility? Use curl -


% curl -u Administrator:password http://localhost:8091/pools/default/stats/range/cm_auto_failover_enabled

{"data":[{"metric":{"nodes":["[::1]:8091"],"instance":"ns_server","name":"cm_auto_failover_enabled"},"values":[[1722622233,"1"],[1722622243,"1"],[1722622253,"1"],[1722622263,"1"],[1722622273,"1"],[1722622283,"1"],[1722622293,"1"]]}],"errors":[],"startTimestamp":1722622233,"endTimestamp":1722622293}
3

@Debasis_Mallick The new metrics will also appear in the Prometheus-formatted metrics, see Configure Prometheus to Collect Couchbase Metrics.

4

Thank you for the update.

We do not see the stats with the below commands. Is this mean no info presents.

[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_auto_failover_enabled
{“data”:,“errors”:,“startTimestamp”:1722858718,“endTimestamp”:1722858778}[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_auto_failover_count
{“data”:,“errors”:,“startTimestamp”:1722859033,“endTimestamp”:1722859093}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_auto_failover_max_count
{“data”:,“errors”:,“startTimestamp”:1722859122,“endTimestamp”:1722859182}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_failover_total
{“data”:,“errors”:,“startTimestamp”:1722859195,“endTimestamp”:1722859255}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_graceful_failover_total
{“data”:,“errors”:,“startTimestamp”:1722859201,“endTimestamp”:1722859261}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_is_balanced
{“data”:,“errors”:,“startTimestamp”:1722859206,“endTimestamp”:1722859266}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_rebalance_in_progress
{“data”:,“errors”:,“startTimestamp”:1722859212,“endTimestamp”:1722859272}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_rebalance_progress
{“data”:,“errors”:,“startTimestamp”:1722859217,“endTimestamp”:1722859277}[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# curl -u cbadmin:cbadmin http://10.10.19.167:8091/pools/default/stats/range/cm_rebalance_total
{“data”:,“errors”:,“startTimestamp”:1722859221,“endTimestamp”:1722859281}[root@localhost ~]#
[root@localhost ~]#

5

“data:” will be empty for servers without the properties (ie. 7.6.1 etc)

http://127.0.0.1:8091/pools/default/stats/range/property_name_that_does_not_exist

{
data: [ ],
errors: [ ],
startTimestamp: 1722878735,
endTimestamp: 1722878795
}

For 7.6.2, it will not empty…

data: [
{
metric: {
nodes: [
"127.0.0.1:8091"
],
instance: "ns_server",
name: "cm_auto_failover_enabled"
},
values: [
[
1722878620,
"1"
],
[
1722878630,
"1"
],
[
1722878640,
"1"
],
[
1722878650,
"1"
],
[
1722878660,
"1"
],
[
1722878670,
"1"
],
[
1722878680,
"1"
]
]
}
],
errors: [ ],
startTimestamp: 1722878620,
endTimestamp: 1722878680
}

The new properties are in 7.6.2. The version number appears under nodes

http://cbserver:8091/pools/default

6

@mreiche , thank you for providing the queries in the previous post.

What is the REST API command to query below statistics.

  • cm_is_balanced: Whether the Cluster Manager is balanced. Only reported by orchestrator nodes and only reported every 30 seconds.
  • cm_rebalance_in_progress: Whether there is a rebalance occurring. Only reported by the orchestrator node.
  • cm_rebalance_progress: An estimate of the progress of the current rebalance. Only reported by the orchestrator.
7

I would expect to find the REST API on the same page of the documentation where you found their descriptions. Where did you find the descriptions?

What’s New in Version 7.6 | Couchbase Docs →

The following Cluster Manager metrics can be queried by means of the REST APIs described in Statistics.

Which was in my August 2 post, your August 5 post. And again in my August 5 post along with the caveat

The new properties are in 7.6.2. The version number appears under nodes

http://cbserver:8091/pools/default
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.