I have a Couchbase cluster consisting of 4 nodes, CB version 4.1.0-5005 Community Edition, I use PHP SDK ver. 2.2.3.
In my system I create periodically create some documents that reflect the current state of campaigns. Today I realised that starting from Friday some documents were not created. When I tried to create those documents using UI, I got the following error in CB error.log file:
[ns_server:error,2017-02-27T13:00:46.749+01:00,ns_1@cb4.savecart:<0.7427.5348>:menelaus_web:loop:184]Server error during processing: [“web request failed”,
{path,
"/pools/default/buckets/default/docs/active_flights::sweetlivingpl"},
{method,‘POST’},
{type,error},
{what,
{badmatch,
{badrpc,
{‘EXIT’,
{function_clause,
[{capi_crud,handle_mutation_rv,
[{mc_header,1,134,0,0,0,17,0},
{mc_entry,<<>>,<<>>,0,0,0,
<<“Temporary failure”>>,0}],
[{file,“src/capi_crud.erl”},
{line,43}]},
{rpc,local_call,3,
[{file,“rpc.erl”},{line,329}]},
{menelaus_web_crud,attempt,6,
[{file,“src/menelaus_web_crud.erl”},
{line,229}]},
{menelaus_web_crud,handle_post,3,
[{file,“src/menelaus_web_crud.erl”},
{line,200}]},
{request_throttler,do_request,3,
[{file,“src/request_throttler.erl”},
{line,59}]},
{menelaus_web,loop,2,
[{file,“src/menelaus_web.erl”},
{line,162}]},
{mochiweb_http,headers,5,
[{file,
"/home/couchbase/jenkins/workspace/sherlock-unix/couchdb/src/mochiweb/mochiweb_http.erl"},
{line,94}]},
{proc_lib,init_p_do_apply,3,
[{file,“proc_lib.erl”},
{line,239}]}]}}}}},
{trace,
[{menelaus_web_crud,handle_post,3,
[{file,“src/menelaus_web_crud.erl”},
{line,200}]},
{request_throttler,do_request,3,
[{file,“src/request_throttler.erl”},
{line,59}]},
{menelaus_web,loop,2,
[{file,“src/menelaus_web.erl”},
{line,162}]},
{mochiweb_http,headers,5,
[{file,
"/home/couchbase/jenkins/workspace/sherlock-unix/couchdb/src/mochiweb/mochiweb_http.erl"},
{line,94}]},
{proc_lib,init_p_do_apply,3,
[{file,“proc_lib.erl”},{line,239}]}]}]
I have 160M docs in my cluster. I assume that this problem may touch other documents as well. I can also see this error on one machine only.