{"id":10201,"date":"2023-06-23T01:43:43","date_gmt":"2023-06-23T08:43:43","guid":{"rendered":"https:\/\/www.couchbase.com\/developers\/tutorials\/general-support-best-practices\/"},"modified":"2023-06-23T01:43:43","modified_gmt":"2023-06-23T08:43:43","slug":"general-support-best-practices","status":"publish","type":"tutorials","link":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/","title":{"rendered":"\u4e00\u822c\u7684\u306a\u30b5\u30dd\u30fc\u30c8\u306e\u30d9\u30b9\u30c8\u30d7\u30e9\u30af\u30c6\u30a3\u30b9"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong> Have an Incident Leader<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A leader should be chosen to manage the discussion on each incident. That leader should manage the conversation.\nOften a thread of investigation can be cut off before it bears fruit because someone on the call has a different opinion of the potential problem or simply doesn\u2019t understand the current investigation and pushes their own agenda.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Keep an Incident Log<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>This leader should be responsible (perhaps through delegation) for the keeping of an incident log.  While ultimately this log should be transcribed to an official report, the best place to keep a running log is in the chat window of a Webex (or other) call. This allows someone who joins a call to catch up on what has happened, what has been discussed, and\/or what has been tried so far.  Webex chat is supported across many platforms and is available to anyone invited to the call, including vendors or people who may not be at their work desks.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Have Architectural Documents Ready<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>You can reduce MTTR significantly by reducing the number of questions that need to be asked during the incident.  Having information prepared can help expedite suggestions for remedial action and improve the accuracy and likelihood of success of any suggestions.<\/li>\n\n\n<li>For Couchbase, some information is standard, such as:\n<ul>\n<li>Product version, including server and SDKs used<\/li>\n<li>Cluster size<\/li>\n<li>Layout of Couchbase services<\/li>\n<li>Peak document count, average document size, normal document growth rates<\/li>\n<li>Overall data size and average data residency<\/li>\n<li>Types of buckets used (Couchbase, Ephemeral, Memcached) and the ejection types (Value, Full)<\/li>\n<\/ul>\n<\/li>\n\n\n<li>For any architecture, it is important to have\n<ul>\n<li>Information about the underlying hardware and\/or virtualization platform<\/li>\n<li>Upstream and downstream applications<\/li>\n<\/ul>\n<\/li>\n\n\n<li>It is also critical to have an accurate history maintained of changes to the system, such as version upgrades, nodes added\/removed\/restarted etc. These events should be logged when they happen so that an accurate history doesn\u2019t need to be recreated during an incident.<\/li>\n\n\n<li>Ideally, a (potentially sanitized) version of this information should be made available to any Couchbase Support Engineers working the incident.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Progress through well defined Incident Stages<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>There should be an ordered progression to the incident investigation.<\/li>\n\n\n<li>Suggested steps would be:\n<ul>\n<li>Preservation of logs and timeline of events preceding the incident<\/li>\n<li>Resumption of service<\/li>\n<li>Root Cause Analysis<\/li>\n<li>Remediation<\/li>\n<\/ul>\n<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Upload all available logs to a Support Ticket as soon as possible<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Updating logs takes time.  Having Couchbase support request logs takes additional time. Please upload all available logs immediately after opening a ticket.<\/li>\n\n\n<li>In most cases, these logs are used for Root Cause Analysis, so, log preservation and resumption of service should already have occurred.<\/li>\n\n\n<li>Ideally, the customer should be capable of resuming service without the help of Couchbase Support. If help is needed to achieve resumption of service, the priority of uploading all logs should be increased.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Have Additional Servers ready to deploy in each environment<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Cloud native<\/em> technologies are designed to have failing nodes replaced with additional nodes.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Properly Size your Machines &amp; Cluster<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Cloud database, including Couchbase, are designed to partition their load across multiple machines.  They are also designed to heal if a machine is lost and\/or removed from the cluster.  For Couchbase, this is either a <em>Failover<\/em>, <em>Recovery<\/em> or <em>Rebalance<\/em>. There must be enough additional resources in the cluster to perform the recovery options.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Triage from the Top Down<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Triage gives clarity to the person trying to restore service because they know clearly what they \u2018don\u2019t have to worry about\u2019.<\/li>\n\n\n<li>Triage has to start with the system as a whole, not a particular component. Develop a series of scripts particular to your environment and application that could validate which systems were responding and which were not.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Avoid Bias<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Experienced Support Engineers (like doctors) will often ignore your specific complaint until they have run through a series of tests or looked at key indicators in the logs.  While this is part of a triage process, it also avoids observation bias.  Bias can be introduced by any of the following factors:\n<ul>\n<li><em>This is the area of the system where the problem is manifesting or is observed<\/em> &#8211; this isn\u2019t necessarily where the problem is caused<\/li>\n<li><em>This part of the architecture is the newest<\/em> or <em>has caused us problems before<\/em> &#8211; while a useful observation, this can cause your analysis to focus on the wrong component<\/li>\n<li><em>The problem can\u2019t be in the area of the stack that I\u2019m responsible for<\/em> &#8211; personal pride could impede investigation into the area potentially responsible for the problem<\/li>\n<li><em>I don\u2019t understand that part of the architecture<\/em> &#8211; can lead to either avoidance or undue focus on one part of the architecture<\/li>\n<\/ul>\n<\/li>\n\n\n<li>One of the prime roles of the Incident Leader it to avoid bias and ensure all options are considered until effectively ruled out.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Recommendation:<\/strong>  Test Tickets<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>New applications that are preparing to go into production should go through the process of creating a Test ticket to make sure there are no impediments to updating logs.<\/li>\n\n<\/ul>\n\n\n\n<h3 id=\"h-providing-internal-tier-one-support\" class=\"wp-block-heading\">Providing internal tier one Support<\/h3>\n\n\n\n<h4 id=\"h-escalation-path\" class=\"wp-block-heading\">Escalation path<\/h4>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Internal Support (Center of Excellence)<\/li>\n\n\n<li>If the issue is related to debugging a product fault internal support should contact Couchbase technical support with the appropriate priority level<\/li>\n\n\n<li>If the issue is related to reviewing Architecture, index definitions, query syntax, or client SDK code, internal support should contact Solutions Engineer or professional services representative.<\/li>\n\n\n<li>Depending on the way to resolve your issue your ticket may be referred to Engineering.<\/li>\n\n<\/ol>\n\n\n\n<h4 id=\"h-building-an-internal-support-team\" class=\"wp-block-heading\">Building an internal support team<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">To start with, when building an internal tier one support team it is important that those providing the support review and understand all of the following guides and best practices.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Backup and Restore Guide (Coming Soon)<\/li>\n\n\n<li>Cluster Setup (Coming Soon)<\/li>\n\n\n<li><a href=\"\/learn\/json-data-modeling-guide\">Data Modeling<\/a><\/li>\n\n\n<li>Development Best Practices (Coming Soon)<\/li>\n\n\n<li><a href=\"\/learn\/couchbase-monitoring-guide\">Monitoring<\/a><\/li>\n\n\n<li>Security (Coming Soon)<\/li>\n\n\n<li>Testing (Coming Soon)<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">If the use case involves query:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"\/learn\/n1ql-query-performance-guide\">N1QL Tuning<\/a><\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">With a strong understanding of those guides, one should be able to provide support for most tier one support issues.<\/p>\n\n\n\n<h5 id=\"h-timeouts-and-rto\" class=\"wp-block-heading\">Timeouts and RTO<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\">Sometimes, client to Couchbase interactions can timeout. If you are having issues with timeouts, using the open-tracing API may help determine the cause or where the timeout is actually happening. How to use open-tracing is language dependent. The exposure of these APIs will allow you to use other tools like jaegertracing or other commercial products to dig into where the timeouts are happening.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/docs.couchbase.com\/java-sdk\/current\/threshold-logging.html#open-tracing\">Java<\/a><\/li>\n\n\n<li><a href=\"https:\/\/docs.couchbase.com\/python-sdk\/current\/threshold-logging.html#open-tracing\">Python<\/a><\/li>\n\n\n<li><a href=\"https:\/\/docs.couchbase.com\/dotnet-sdk\/current\/threshold-logging.html#open-tracing\">.Net<\/a><\/li>\n\n\n<li><a href=\"https:\/\/docs.couchbase.com\/php-sdk\/current\/threshold-logging.html#open-tracing\">PHP<\/a><\/li>\n\n\n<li><a href=\"https:\/\/docs.couchbase.com\/c-sdk\/current\/threshold-logging.html#open-tracing\">C<\/a><\/li>\n\n\n<li><a href=\"https:\/\/docs.couchbase.com\/go-sdk\/current\/threshold-logging.html#open-tracing\">Go<\/a><\/li>\n\n\n<li><a href=\"https:\/\/docs.couchbase.com\/nodejs-sdk\/current\/threshold-logging.html#open-tracing\">NodeJS<\/a><\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Response Time Observability (RTO) is part of open-tracing and as four parts:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Data Service (KV Engine): Logs KV operations that exceed a threshold<\/li>\n\n\n<li>Threshold Logging (SDK): Logs requests that exceed a per-service threshold<\/li>\n\n\n<li>Orphan Logging (SDK): Log responses when the request has timed out<\/li>\n\n\n<li>Improved Timeout Messages (SDK)<\/li>\n\n<\/ul>\n\n\n\n<h5 id=\"h-data-service\" class=\"wp-block-heading\">Data Service<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\">Data Service Tracing must first be enabled to use RTO on the data service.<\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>curl -u Administrator:password -X POST localhost:8091\/pools\/default\/settings\/memcached\/global \n    --data tracing_enabled=true\/false\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Which then gives the following messages in the memcached.log<\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>2018-07-17T15:30:05.308518Z INFO 37: HELO [{&quot;a&quot;:&quot;libcouchbase\/2.9.2 (Darwin-17.6.0; x86_64; \n    Clang 9.1.0.9020039)&quot;,&quot;i&quot;:&quot;00000000b35f58aa\/39cc1fd4eeaf1d67&quot;}] \n    TCP nodelay, XATTR, XERROR, Select bucket, Snappy, JSON, Tracing \n    [ 127.0.0.1:55200 - 127.0.0.1:11210 (not authenticated) ]\n...\n2018-07-17T15:30:07.084239Z WARNING 37: Slow operation. \n    {&quot;cid&quot;:&quot;00000000b35f58aa\/39cc1fd4eeaf1d67\/0&quot;,&quot;duration&quot;:&quot;1771 ms&quot;, \n    &quot;trace&quot;:&quot;request=329316910555375:1771975 \n             get=329316911686879:35 \n             bg.wait=329316911701108:4715 \n             bg.load=329318677724103:1766022 \n             get=329318682489143:21&quot;,\n    &quot;command&quot;:&quot;GET&quot;,&quot;peer&quot;:&quot;127.0.0.1:55200&quot;}\n<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th align=\"left\">Name<\/th>\n<th align=\"left\">Definition<\/th>\n<th align=\"left\">Comments<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td align=\"left\">request<\/td>\n<td align=\"left\">From KV-Engine receiving request (from OS) to sending response (to OS).<\/td>\n<td align=\"left\">Overall KV-Engine view of the request.<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">bg.wait<\/td>\n<td align=\"left\">From KV-Engine detecting background fetch needed, to starting to read from disk.<\/td>\n<td align=\"left\">Long duration suggests contention on  Reader Threads.<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">bg.load<\/td>\n<td align=\"left\">From KV-Engine starting to read from disk to loading the document.<\/td>\n<td align=\"left\">Long duration suggests slow disk subsystem.<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">get<\/td>\n<td align=\"left\">From KV-Engine parsing a GET request to processing it.<\/td>\n<td align=\"left\">For docs which are not resident, you&#8217;ll see two instances of this span.<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">get.if<\/td>\n<td align=\"left\">From KV-Engine parsing a GET _IF request to processing it.<\/td>\n<td align=\"left\">Similar to get, but used for certain request (e.g. XATTR handling)<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">get.stats<\/td>\n<td align=\"left\">From KV-Engine parsing a STATS request to processing it.<\/td>\n<td align=\"left\">Different STATS keys have different costs (e.g. disk stats are slower).<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">store<\/td>\n<td align=\"left\">From KV-Engine parsing a STORE request to processing it.<\/td>\n<td align=\"left\">Typically fast (as only has to write into HashTable).<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">set.with.meta<\/td>\n<td align=\"left\">From KV-Engine parsing a SET_WITH_META request to processing it.<\/td>\n<td align=\"left\">Similar to store but for XDCR \/ restore updates.<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h5 id=\"h-sdk-logging\" class=\"wp-block-heading\">SDK Logging<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\">SDK Logging is on by default. Below is an example from an RTO Log.<\/p>\n\n\n\n<pre class=\"wp-block-code language-json\"><code>{\n  &quot;service&quot;: &quot;kv&quot;,\n  &quot;count&quot;: 15,\n  &quot;top&quot;: [\n    {\n      &quot;operation_name&quot;: &quot;get&quot;,\n      &quot;last_operation_id&quot;: &quot;0x21&quot;,\n      &quot;last_local_address&quot;: &quot;10.211.55.3:52450&quot;,\n      &quot;last_remote_address&quot;: &quot;10.112.180.101:11210&quot;,\n      &quot;last_local_id&quot;: &quot;66388CF5BFCF7522\/18CC8791579B567C&quot;,\n      &quot;total_duration_us&quot;: 18908,\n      &quot;encode_us&quot;: 256,\n      &quot;dispatch_us&quot;: 825,\n      &quot;decode_us&quot;: 16,\n      &quot;server_duration_us&quot;: 14\n    }\n  ]\n}\n<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th align=\"left\">Name<\/th>\n<th align=\"left\">Definition<\/th>\n<th align=\"left\">Comments<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td align=\"left\">operation_name<\/td>\n<td align=\"left\">Type of operation<\/td>\n<td align=\"left\"><\/td>\n<\/tr>\n<tr>\n<td align=\"left\">last_operation_id<\/td>\n<td align=\"left\">A combination of type of operation and ID<\/td>\n<td align=\"left\">Useful for troubleshooting in combination with the local_id<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">last_local_address<\/td>\n<td align=\"left\">The local socket used for this operation<\/td>\n<td align=\"left\"><\/td>\n<\/tr>\n<tr>\n<td align=\"left\">last_remote_address<\/td>\n<td align=\"left\">Socket used on server for this request<\/td>\n<td align=\"left\">Useful when determining which node processed this request<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">last_local_id<\/td>\n<td align=\"left\">This ID is negotiated with the server<\/td>\n<td align=\"left\">Can be used to correlate logging information on both sides<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">total_duration_us<\/td>\n<td align=\"left\">The total time it took to perform the full operation<\/td>\n<td align=\"left\"><\/td>\n<\/tr>\n<tr>\n<td align=\"left\">encode_us<\/td>\n<td align=\"left\">Time the client took to encode the request<\/td>\n<td align=\"left\">Is longer the larger and more complex the json<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">dispatch_us<\/td>\n<td align=\"left\">Time from when client to sent the request to when it got a response into the clients ring buffer<\/td>\n<td align=\"left\">Amount of time spent traversing the network can be found by subtracting dispatch_us &#8211; server_duration_us<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">decode_us<\/td>\n<td align=\"left\">Time the client took to decode the response<\/td>\n<td align=\"left\">Is longer the larger and more complex the json<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">server_duration_us<\/td>\n<td align=\"left\">Time the server took to do its work.<\/td>\n<td align=\"left\">&#8211;<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h5 id=\"h-orphan-logging\" class=\"wp-block-heading\">Orphan Logging<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\">This is enabled by default in the SDK. This aggregates responses where the request has timed out. ie KV get exceeds time out, error returned to application, response received some time afterwards. The log interval is 10 seconds at the WARN level, and has a  Per-service sample size of 10. Below is an example of an orphan log.<\/p>\n\n\n\n<pre class=\"wp-block-code language-json\"><code>{\n  &quot;service&quot;: &quot;kv&quot;,\n  &quot;count&quot;: 2,\n  &quot;top&quot;: [\n    {\n      &quot;s&quot;: &quot;kv:get&quot;,\n      &quot;i&quot;: &quot;0x21&quot;,\n      &quot;c&quot;: &quot;66388CF5BFCF7522\/18CC8791579B567C&quot;,\n      &quot;b&quot;: &quot;default&quot;,\n      &quot;l&quot;: &quot;192.168.1.101:11210&quot;,\n      &quot;r&quot;: &quot;10.112.181.101:12110&quot;,\n      &quot;d&quot;: 120\n    }\n  ]\n}\n<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th align=\"left\">Name<\/th>\n<th align=\"left\">Definition<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td align=\"left\">s<\/td>\n<td align=\"left\">Service type<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">i<\/td>\n<td align=\"left\">Operation ID<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">c<\/td>\n<td align=\"left\">Connection ID<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">b<\/td>\n<td align=\"left\">Bucket<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">l<\/td>\n<td align=\"left\">LocalEndpoint and Port<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">r<\/td>\n<td align=\"left\">Remote Endpoint and Port<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">d<\/td>\n<td align=\"left\">Duration (us)<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">t<\/td>\n<td align=\"left\">Timeout<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Combining these three logs, a troubleshooter should be able to trace timings all the way from the client to server and back. This should help identify any operations that timed out, or are performing slowly. Providing this information to support will greatly help in determining the cause of timeouts.<\/p>\n\n\n\n<h4 id=\"h-restarting-services\" class=\"wp-block-heading\">Restarting services<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">On each Couchbase node there are multiple services running to maintain Couchbase and provide the services needed for operation. At times, and normally with the guidance of support, it may be necessary to restart some of those services in a recovery effort from degraded operations. A complete list of processes can be found at <a href=\"https:\/\/docs.couchbase.com\/server\/current\/install\/server-processes.html\">https:\/\/docs.couchbase.com\/server\/current\/install\/server-processes.html<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"assets\/babysitter.png\" alt=\"Couchbase Services\"><\/p>\n\n\n\n<h5 id=\"h-cluster-manager\" class=\"wp-block-heading\">Cluster Manager<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What it does:<\/strong>\nProvides admin UI &amp; RESTful administrative APIs, coordinates cluster membership, manages rebalancing, etc.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why to restart:<\/strong>\nConnections not being released, rebalance that won\u2019t \u2018stop\u2019<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact of restarting:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Should not impact operations unless it doesn\u2019t restart faster than the auto-failover timeout. In that case restarting may cause a failover.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>curl --data \u201cerlang:halt().\u201d -u &lt;Administrator&gt;:&lt;password&gt; http:\/\/&lt;host&gt;:8091\/diag\/eval\n<\/code><\/pre>\n\n\n\n<h5 id=\"h-memcached\" class=\"wp-block-heading\">Memcached<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What it does:<\/strong>\nThis is the data service. It manages all the vbuckets and the data within the vbuckets.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why to restart:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Do not restart without guidance from support.<\/li>\n\n\n<li>This would be done if a hard failover of a node was needed and the UI was unavailable.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact of restarting:<\/strong>\nCauses a hard failover of the data node.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>pkill memcached\n<\/code><\/pre>\n\n\n\n<h5 id=\"h-projector\" class=\"wp-block-heading\">Projector<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What it does:<\/strong>\nThis process runs on each Data node. It consumes DCP messages, Filters and formats index entries, and sends those messages to the indexer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why to restart:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>New index build seems \u2018stuck\u2019<\/li>\n\n\n<li>items_queued + items_pending increasing and not going down<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact of restarting:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Should continue from last checkpoint, no direct effect on inflight ops<\/li>\n\n\n<li>Likely already affecting At_plus or Request_plus queries<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>pkill projector\n<\/code><\/pre>\n\n\n\n<h5 id=\"h-indexer\" class=\"wp-block-heading\">Indexer<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What it does:<\/strong>\nThe indexer runs on each Index node. It manages index records in memory &amp; disk. This process is invoked by query engine to select query candidates.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why to restart:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Memory use of the indexer process is growing without bound<\/li>\n\n\n<li>Existing memory leak of metadata (fixed in 6.0.2)<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact of restarting:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Inflight queries may fail if they are using indexes maintained by this indexer. <em>YOUR CLIENTS SHOULD CODE FOR THIS EVENT<\/em><\/li>\n\n\n<li>Assuming you have redundant index definitions no impact should occur to new queries.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>pkill indexer\n<\/code><\/pre>\n\n\n\n<h5 id=\"h-query-service\" class=\"wp-block-heading\">Query Service<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What it does:<\/strong>\nThe query service runs on each Query node. Each N1QL query is assigned to one node. Calls from the query service go out to Index service, Data Service and FTS Service as required<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why to restart:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Memory use growing without bound<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact of restarting:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Inflight queries will fail if they are assigned to this query node. <em>YOUR CLIENTS SHOULD CODE FOR THIS EVENT<\/em><\/li>\n\n\n<li>New queries will not go to this node and should be fine<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>pkill cbq-engine\n<\/code><\/pre>\n\n\n\n<h5 id=\"h-xdcr-service\" class=\"wp-block-heading\">XDCR Service<\/h5>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What it does:<\/strong>\nthe goxdcr process runs on each Data node. It has responsibilities for data replication work done on both source &amp; target side.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why to restart:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>High CPU usage<\/li>\n\n\n<li>Connection timeouts in log<\/li>\n\n\n<li>changes_left stat increasing without going back down<\/li>\n\n\n<li>bandwidth_usage stat at 0<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact of restarting:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Will restart from last checkpoint, may see a temporary increase in CPU &amp; bandwidth usage<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code language-bash\"><code>pkill goxdcr\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Recommendation: Have an Incident Leader Recommendation: Keep an Incident Log Recommendation: Have Architectural Documents Ready Recommendation: Progress through well defined Incident Stages Recommendation: Upload all available logs to a Support Ticket as soon as possible Recommendation: Have Additional Servers ready to deploy in each environment Recommendation: Properly Size your Machines &amp; Cluster Recommendation: Triage from the Top Down Recommendation: Avoid Bias Recommendation: Test Tickets Providing internal tier one Support Escalation path Building an internal support team To start with, when building an internal tier one support team it is important that those providing the support review and understand all of the following guides and best practices. If the use case [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":0,"menu_order":0,"template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"_acf":"","footnotes":""},"tutorial-languages":[551],"tutorial-tags":[569,568],"tutorial-technologies":[554],"tutorial-types":[555],"class_list":["post-10201","tutorials","type-tutorials","status-publish","hentry","tutorial-languages-any","tutorial-tags-customer","tutorial-tags-support","tutorial-technologies-server","tutorial-types-tutorial"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v28.8 (Yoast SEO v28.9) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>General Support Best Practices - Couchbase<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/\" \/>\n<meta property=\"og:locale\" content=\"ja_JP\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"General Support Best Practices\" \/>\n<meta property=\"og:description\" content=\"Recommendation: Have an Incident Leader Recommendation: Keep an Incident Log Recommendation: Have Architectural Documents Ready Recommendation: Progress through well defined Incident Stages Recommendation: Upload all available logs to a Support Ticket as soon as possible Recommendation: Have Additional Servers ready to deploy in each environment Recommendation: Properly Size your Machines &amp; Cluster Recommendation: Triage from the Top Down Recommendation: Avoid Bias Recommendation: Test Tickets Providing internal tier one Support Escalation path Building an internal support team To start with, when building an internal tier one support team it is important that those providing the support review and understand all of the following guides and best practices. If the use case [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/\" \/>\n<meta property=\"og:site_name\" content=\"Couchbase\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Couchbase\/\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.couchbase.com\/wp-content\/uploads\/2026\/10\/Cloud-2.png\" \/>\n\t<meta property=\"og:image:width\" content=\"2548\" \/>\n\t<meta property=\"og:image:height\" content=\"1171\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@couchbase\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"12\u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/developers\\\/tutorials\\\/general-support-best-practices\\\/\",\"url\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/developers\\\/tutorials\\\/general-support-best-practices\\\/\",\"name\":\"General Support Best Practices - Couchbase\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/#website\"},\"datePublished\":\"2023-06-23T08:43:43+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/developers\\\/tutorials\\\/general-support-best-practices\\\/#breadcrumb\"},\"inLanguage\":\"ja\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/developers\\\/tutorials\\\/general-support-best-practices\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/developers\\\/tutorials\\\/general-support-best-practices\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.couchbase.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"General Support Best Practices\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/#website\",\"url\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/\",\"name\":\"Couchbase\",\"description\":\"The Operational Data Platform for AI\u2122\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/#organization\"},\"alternateName\":\"Couchbase, Inc.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ja\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/#organization\",\"name\":\"Couchbase\",\"alternateName\":\"Couchbase, Inc.\",\"url\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.couchbase.com\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/Couchbase_Logo_Couchbase_Token_Red-1.png\",\"contentUrl\":\"https:\\\/\\\/www.couchbase.com\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/Couchbase_Logo_Couchbase_Token_Red-1.png\",\"width\":1129,\"height\":1129,\"caption\":\"Couchbase\"},\"image\":{\"@id\":\"https:\\\/\\\/www.couchbase.com\\\/ja\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/Couchbase\\\/\",\"https:\\\/\\\/x.com\\\/couchbase\",\"https:\\\/\\\/www.youtube.com\\\/c\\\/CouchbaseServer\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/couchbase\\\/\",\"https:\\\/\\\/github.com\\\/couchbase\\\/\",\"https:\\\/\\\/en.wikipedia.org\\\/wiki\\\/Couchbase,_Inc.\"],\"description\":\"Couchbase is a database software company headquartered in San Jose, California, providing an operational data platform for AI used by hundreds of global enterprises.\",\"legalName\":\"Couchbase, Inc.\",\"foundingDate\":\"2011-01-01\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"General Support Best Practices - Couchbase","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/","og_locale":"ja_JP","og_type":"article","og_title":"General Support Best Practices","og_description":"Recommendation: Have an Incident Leader Recommendation: Keep an Incident Log Recommendation: Have Architectural Documents Ready Recommendation: Progress through well defined Incident Stages Recommendation: Upload all available logs to a Support Ticket as soon as possible Recommendation: Have Additional Servers ready to deploy in each environment Recommendation: Properly Size your Machines &amp; Cluster Recommendation: Triage from the Top Down Recommendation: Avoid Bias Recommendation: Test Tickets Providing internal tier one Support Escalation path Building an internal support team To start with, when building an internal tier one support team it is important that those providing the support review and understand all of the following guides and best practices. If the use case [&hellip;]","og_url":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/","og_site_name":"Couchbase","article_publisher":"https:\/\/www.facebook.com\/Couchbase\/","og_image":[{"width":2548,"height":1171,"url":"https:\/\/www.couchbase.com\/wp-content\/uploads\/2026\/10\/Cloud-2.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@couchbase","twitter_misc":{"Est. reading time":"12\u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/","url":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/","name":"General Support Best Practices - Couchbase","isPartOf":{"@id":"https:\/\/www.couchbase.com\/ja\/#website"},"datePublished":"2023-06-23T08:43:43+00:00","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/#breadcrumb"},"inLanguage":"ja","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/ja\/developers\/tutorials\/general-support-best-practices\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/"},{"@type":"ListItem","position":2,"name":"General Support Best Practices"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/ja\/#website","url":"https:\/\/www.couchbase.com\/ja\/","name":"Couchbase","description":"The Operational Data Platform for AI\u2122","publisher":{"@id":"https:\/\/www.couchbase.com\/ja\/#organization"},"alternateName":"Couchbase, Inc.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/ja\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ja"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/ja\/#organization","name":"Couchbase","alternateName":"Couchbase, Inc.","url":"https:\/\/www.couchbase.com\/ja\/","logo":{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.couchbase.com\/ja\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/wp-content\/uploads\/2026\/11\/Couchbase_Logo_Couchbase_Token_Red-1.png","contentUrl":"https:\/\/www.couchbase.com\/wp-content\/uploads\/2026\/11\/Couchbase_Logo_Couchbase_Token_Red-1.png","width":1129,"height":1129,"caption":"Couchbase"},"image":{"@id":"https:\/\/www.couchbase.com\/ja\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Couchbase\/","https:\/\/x.com\/couchbase","https:\/\/www.youtube.com\/c\/CouchbaseServer","https:\/\/www.linkedin.com\/company\/couchbase\/","https:\/\/github.com\/couchbase\/","https:\/\/en.wikipedia.org\/wiki\/Couchbase,_Inc."],"description":"Couchbase\u306f\u30ab\u30ea\u30d5\u30a9\u30eb\u30cb\u30a2\u5dde\u30b5\u30f3\u30ce\u30bc\u306b\u672c\u793e\u3092\u7f6e\u304f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u30bd\u30d5\u30c8\u30a6\u30a7\u30a2\u4f01\u696d\u3067\u3001\u6570\u767e\u306e\u30b0\u30ed\u30fc\u30d0\u30eb\u4f01\u696d\u304c\u4f7f\u7528\u3059\u308bAI\u7528\u306e\u904b\u7528\u30c7\u30fc\u30bf\u30d7\u30e9\u30c3\u30c8\u30d5\u30a9\u30fc\u30e0\u3092\u63d0\u4f9b\u3057\u3066\u3044\u307e\u3059\u3002.","legalName":"Couchbase, Inc.","foundingDate":"2011-01-01"}]}},"acf":[],"permalink_manager":null,"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorials\/10201","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorials"}],"about":[{"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/types\/tutorials"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorials\/10201\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/media?parent=10201"}],"wp:term":[{"taxonomy":"tutorial-languages","embeddable":true,"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorial-languages?post=10201"},{"taxonomy":"tutorial-tags","embeddable":true,"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorial-tags?post=10201"},{"taxonomy":"tutorial-technologies","embeddable":true,"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorial-technologies?post=10201"},{"taxonomy":"tutorial-types","embeddable":true,"href":"https:\/\/www.couchbase.com\/ja\/wp-json\/wp\/v2\/tutorial-types?post=10201"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}