Home | Forums | Couchbase | Couchbase Server 1.8.x

Couchbase causeing error events every 8 seconds

7 replies [Last post]
Mon, 07/16/2012 - 11:11
ngabello
Offline
Joined: 07/16/2012
Groups: None

Earlier today my pc started acting crazy my windows would gain and lose focus randomly, finally when I examined the event log I found this error every 8 seconds:

Faulting application name: memcached.exe, version: 0.0.0.0, time stamp: 0x4efc900f
Faulting module name: ep.so, version: 0.0.0.0, time stamp: 0x4efc90af
Exception code: 0x40000015
Fault offset: 0x000de764
Faulting process id: 0x1fe4
Faulting application start time: 0x01cd637c74131478
Faulting application path: c:\Program Files\Couchbase\Server\bin\memcached.exe
Faulting module path: c:\Program Files\Couchbase\Server\lib\memcached\ep.so
Report Id: b2aa11a2-cf6f-11e1-a90a-842b2baee8df

this cleared up when I stopped the CouchBaseServer in services, however this continued to show up in event log:

Fault bucket 3031124166, type 5
Event Name: WindowsUpdateFailure
Response: Not available
Cab Id: 0

Problem signature:
P1: 7.6.7600.256
P2: 80072ee2
P3: 00000000-0000-0000-0000-000000000000
P4: Scan
P5: 101
P6: Unmanaged
P7:
P8:
P9:
P10:

Attached files:

These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportArchive\NonCritical_7.6.7600.256_d657698cc2203130accf771c18dbe77b92e59fa7_1968e417

Analysis symbol:
Rechecking for solution: 0
Report Id: 93f8950f-cf5d-11e1-a069-842b2baee8df
Report Status: 0

after shutting down the memcached service all returned to normal. Funny thing is I was not even actively using Couchbase. Can anyone give any insight

Top
Mon, 07/16/2012 - 15:02
ingenthr
Offline
Joined: 03/16/2010
Groups:

Given the short timeframe, it looks like perhaps it's crashing during warmup.

Which version of Couchbase Server do you have installed? We may want to try to get some further diagnostic info.

Top
Mon, 07/16/2012 - 15:03
katana
Offline
Joined: 05/09/2012
Groups: None

This is likely due to Windows Update Agent (the Microsoft Windows task which looks for updates to the Operating system) possibly conflicting with Couchbase runtime / DLLs. memcached.exe is one of the key processes behind Couchbase server. From what you mentioned, it appears you had to manually kill a memcached.exe process apart from Stopping the Couchbase Service from the Services Console - is that correct?

To address this issue, I'd suggest looking into updating the Windows Update Agent(WUA) to the latest as mentioned in Microsoft KB article http://support.microsoft.com/kb/949104

Once you've updated WUA, you can turn Couchbase service back on and see if there is still an issue.

The exception code 0x40000015 is somewhat generic (it literally means STATUS_FATAL_APP_EXIT). In other words the application hit a condition which resulted in the O/S signalling the application to handle it. In the absence of a default handler for that condition, the application (i.e. Couchbase / memcached.exe) called Abort() resulting in the crash. If you'd turned on a Windows debugger, this would have resulted in a crash dump file.

As an aside, I'd be curious to see what Windows captured as debug info in the file it generated
C:\ProgramData\Microsoft\Windows\WER\ReportArchive\NonCritical_7.6.7600.256_d657698cc2203130accf771c18dbe77b92e59fa7_1968e417

Top
Wed, 07/18/2012 - 10:51
ingenthr
Offline
Joined: 03/16/2010
Groups:

In a separate email...

ngabello wrote:
Did a full Update, restarted computer and still getting same errors.

Can you go into the couchbase web console, and in the log click on "generate diagnostic report" and post it somewhere for us or post an issue to couchbase.com/issues with an attachment? It'd be a great help to try to get us to the bottom of this.

Top
Wed, 07/18/2012 - 11:20
hari
Offline
Joined: 05/09/2012
Groups: None

ngabello, also curious to see what is the 'exact' error that is reported in the Event log.

The Windows Update issue might be misleading. If memcached.exe is indeed crashing, I'd like to see what Exception was caught by the process.

A few things to check, are you near full on the disk during these error timestamps? Do you have a large # of buckets configured?

I'd also keep the Windows Performance Monitor / TaskMan (performance tab) open to see if any CPU spikes correspond to these exceptions in the Event log. And like ingenthr suggested the diag report would help answer some of what is going on. Curious, you are on 1.8.0 CE right? Any chance you can try 1.8.1 EE?

Hope this helps

Top
Mon, 09/03/2012 - 08:54
Lubos
Offline
Joined: 09/03/2012
Groups: None

Hi,

is there any solution to this issue ? i installed CouchBase server 1.8 on Windows Server 2008 R2 and i'm facing same issue...

update : i found following. when i install older membase installation (1.7.2) then it seems to be working correctly - when i create bucket, i see server is up and running. However when i update to couchbase server 1.8 or 2.0, then node is in pending state and entries in event log are created...

Thanks
Lubos

Top
Wed, 09/05/2012 - 08:28
ingenthr
Offline
Joined: 03/16/2010
Groups:

Hi Lubos,

Just to confirm, you see memcached.exe errors like above in the event log with Couchbase Server 1.8.1 installed, is that correct? We'd need to get some additional info to identify the issue more specifically.

Thanks,

Matt

Top
Thu, 09/06/2012 - 00:21
Lubos
Offline
Joined: 09/03/2012
Groups: None

hi,

yes. correct - at least very similar symptoms despite the root cause might be completely different...

Everything seems to be ok as long i have no bucket created. server node is shown to be up and running.
When i create bucket (no matter if i use coachbase or memcached), server node goes to pending state. Then i see eventlog and nlog console entry created each few seconds. In general problem seems to be with starting memcached.exe for probably different reason than in original post...

thanks for any help
Lubos

This is log in event log
Faulting application name: memcached.exe, version: 0.0.0.0, time stamp: 0x4fea5398
Faulting module name: KERNELBASE.dll, version: 6.1.7601.17514, time stamp: 0x4ce78f36
Exception code: 0x80000003
Fault offset: 0x0000000000017f32
Faulting process id: 0xe3c
Faulting application start time: 0x01cd8bf8baf39e78
Faulting application path: c:\Program Files\Couchbase\Server\bin\memcached.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: f8b3345a-f7eb-11e1-bc48-000c29073e5c

This log is generated by the console
Event Module Code Server Node Time
Service memcached exited on node 'ns_1@192.168.226.129' in 0.62s
supervisor_cushion001 ns_1@192.168.226.129 09:13:07 - Thu Sep 6, 2012
Service memcached exited on node 'ns_1@192.168.226.129' in 0.60s
supervisor_cushion001 ns_1@192.168.226.129 09:13:02 - Thu Sep 6, 2012
Service memcached exited on node 'ns_1@192.168.226.129' in 1.39s
supervisor_cushion001 ns_1@192.168.226.129 09:12:56 - Thu Sep 6, 2012
Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ns_port_server000 ns_1@192.168.226.129 09:12:24 - Thu Sep 6, 2012
Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ns_port_server000 ns_1@192.168.226.129 09:11:54 - Thu Sep 6, 2012
Couchbase Server has started on web port 8091 on node 'ns_1@192.168.226.129'. menelaus_sup001 ns_1@192.168.226.129 09:11:23 - Thu Sep 6, 2012

per_node_diag = [{'ns_1@192.168.226.129',
{badrpc,{'EXIT',{child_died,killed}}}}]

nodes_info = [{struct,
[{systemStats,
{struct,
[{cpu_utilization_rate,6.215538847117794},
{swap_total,4293902336},
{swap_used,1180749824}]}},
{interestingStats,{struct,[]}},
{uptime,<<"74">>},
{memoryTotal,2146951168},
{memoryFree,1163870208},
{mcdMemoryReserved,1637},
{mcdMemoryAllocated,1637},
{otpNode,<<"ns_1@192.168.226.129">>},
{otpCookie,<<"wvssrcakuifibgjg">>},
{clusterMembership,<<"active">>},
{status,<<"warmup">>},
{hostname,<<"192.168.226.129:8091">>},
{clusterCompatibility,1},
{version,<<"1.8.1-937-rel-enterprise">>},
{os,<<"windows">>},
{ports,{struct,[{proxy,11211},{direct,11210}]}}]}]

buckets = [{"default",
[{sasl_password,[]},
{num_replicas,0},
{ram_quota,524288000},
{auth_type,sasl},
{type,membase},
{num_vbuckets,1024},
{servers,['ns_1@192.168.226.129']}]}]

logs:
-------------------------------

2012-09-06 08:23:37.234 ns_cookie_manager:3:info:cookie update(ns_1@192.168.226.129) - Initial otp cookie generated: wvssrcakuifibgjg
2012-09-06 08:23:37.280 mb_master:0:info:message(ns_1@192.168.226.129) - I'm the only node, so I'm the master.
2012-09-06 08:23:37.405 menelaus_sup:1:info:web start ok(ns_1@192.168.226.129) - Couchbase Server has started on web port 8091 on node 'ns_1@192.168.226.129'.
2012-09-06 08:23:40.323 ns_port_server:0:info:message(ns_1@192.168.226.129) - Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
2012-09-06 08:23:40.323 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 2.86s

2012-09-06 08:23:46.013 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.69s

2012-09-06 08:23:51.504 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.47s

2012-09-06 08:23:56.901 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.39s

2012-09-06 08:24:00.552 menelaus_web:12:info:message(ns_1@192.168.226.129) - Created bucket "default" of type: membase
[{num_replicas,0},{ram_quota,524288000},{auth_type,sasl}]
2012-09-06 08:24:02.439 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.53s

2012-09-06 08:24:07.884 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.44s

2012-09-06 08:24:13.266 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.37s

2012-09-06 08:24:18.695 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.42s

2012-09-06 08:24:24.061 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.36s

2012-09-06 08:24:37.228 ns_port_server:0:info:message(ns_1@192.168.226.129) - Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: (repeated 10 times)
2012-09-06 08:24:37.228 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.37s
(repeated 1 times)
2012-09-06 08:24:37.228 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.44s
(repeated 1 times)
2012-09-06 08:24:40.301 ns_port_server:0:info:message(ns_1@192.168.226.129) - Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
2012-09-06 08:24:40.301 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.41s

2012-09-06 08:24:45.714 supervisor_cushion:1:warning:port exited too soon after restart(ns_1@192.168.226.129) - Service memcached exited on node 'ns_1@192.168.226.129' in 0.40s

logs_node (debug):
-------------------------------
[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:<0.67.0>:ns_server:init_logging:138] Started & configured logging
[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:<0.67.0>:ns_server:log_pending:30] Static config terms:
[{error_logger_mf_dir,"C:\\Program Files\\Couchbase\\Server\\var\\lib\\couchbase\\logs"},
{error_logger_mf_maxbytes,10485760},
{error_logger_mf_maxfiles,20},
{path_config_bindir,"C:\\Program Files\\Couchbase\\Server\\bin"},
{path_config_etcdir,"C:\\Program Files\\Couchbase\\Server\\etc\\couchbase"},
{path_config_libdir,"C:\\Program Files\\Couchbase\\Server\\lib"},
{path_config_datadir,"C:\\Program Files\\Couchbase\\Server\\var\\lib\\couchbase"},
{path_config_tmpdir,"C:\\Program Files\\Couchbase\\Server\\var\\lib\\couchbase\\tmp"},
{loglevel_default,info},
{loglevel_ns_server,info},
{loglevel_error_logger,info},
{loglevel_user,info},
{loglevel_menelaus,info},
{loglevel_ns_doctor,info},
{loglevel_stats,info},
{loglevel_rebalance,info},
{loglevel_cluster,info}]
[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:ns_server_cluster_sup:log_os_info:start_link:25] OS type: {win32,nt} Version: {6,1,7601}
Runtime info: [{otp_release,"R14B02"},
{erl_version,"5.8.3"},
{erl_version_long,
"Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:16]\n"},
{system_arch_raw,"win32"},
{system_arch,"windows"},
{localtime,{{2012,9,6},{8,23,34}}},
{memory,
[{total,6191456},
{processes,944632},
{processes_used,936336},
{system,5246824},
{atom,446233},
{atom_used,438310},
{binary,17160},
{code,3414838},
{ets,193140}]},
{loaded,
[ns_info,log_os_info,cb_init_loggers,ns_server_cluster_sup,
ale_default_formatter,otp_internal,misc,
'ale_logger-cluster','ale_logger-rebalance',
'ale_logger-stats','ale_logger-ns_doctor',
'ale_logger-menelaus','ale_logger-user',
'ale_logger-ns_server',ns_log_sink,disk_log_sup,
disk_log_server,disk_log_1,disk_log,ale_disk_sink,
ns_server,timer,io_lib_fread,memsup,disksup,
os_mon_sysinfo,os_mon,sasl_report,release_handler,
calendar,overload,alarm_handler,sasl_report_tty_h,sasl,
ale_error_logger_handler,'ale_logger-ale_logger',
'ale_logger-error_logger',beam_opcodes,beam_dict,beam_asm,
beam_validator,beam_flatten,beam_trim,beam_receive,
beam_bsm,beam_peep,beam_dead,beam_type,beam_bool,
beam_clean,beam_utils,beam_jump,beam_block,v3_codegen,
v3_life,v3_kernel,sys_core_dsetel,erl_bifs,sys_core_fold,
cerl_trees,sys_core_inline,core_lib,cerl,v3_core,erl_bits,
erl_expand_records,sys_pre_expand,sofs,erl_internal,
compile,dynamic_compile,ale_utils,io_lib_pretty,
ale_codegen,ale,ale_dynamic_sup,ale_sup,io_lib_format,
ale_app,io_lib,io,sets,dict,ordsets,erl_lint,ram_file,
beam_lib,ns_bootstrap,file_io_server,orddict,erl_eval,
file,c,error_logger_tty_h,kernel_config,queue,shell,user,
user_drv,user_sup,supervisor_bridge,standard_error,
unicode,binary,ets,gb_sets,hipe_unified_loader,packages,
code_server,code,file_server,global_group,gen_tcp,
inet_tcp,auth,erl_epmd,inet_tcp_dist,net_kernel,
erl_distribution,filename,win32reg,inet_parse,inet,
inet_udp,os,inet_config,inet_db,global,gb_trees,rpc,
supervisor,kernel,application_master,sys,application,
gen_server,erl_parse,proplists,erl_scan,lists,
application_controller,proc_lib,gen,gen_event,
error_logger,heart,error_handler,erlang,erl_prim_loader,
prim_zip,zlib,prim_file,prim_inet,init,otp_ring0]},
{applications,
[{ale,"Another Logger for Erlang","8cffe61"},
{os_mon,"CPO CXC 138 46","2.2.5"},
{kernel,"ERTS CXC 138 10","2.14.3"},
{sasl,"SASL CXC 138 11","2.1.9.3"},
{ns_server,"Couchbase server","1.8.1-937-rel-enterprise"},
{stdlib,"ERTS CXC 138 10","1.17.3"}]},
{pre_loaded,
[erlang,erl_prim_loader,prim_zip,zlib,prim_file,prim_inet,
init,otp_ring0]},
{process_count,63},
{node,'ns_1@192.168.226.129'},
{nodes,[]},
{registered,
[init,inet_db,rex,net_sup,kernel_sup,global_name_server,
kernel_safe_sup,release_handler,file_server_2,overload,
'sink-disk_default',global_group,alarm_handler,
erl_prim_loader,erl_epmd,sasl_safe_sup,auth,error_logger,
net_kernel,disk_log_sup,disk_log_server,
ns_server_cluster_sup,os_mon_sup,code_server,
os_mon_sysinfo,application_controller,memsup,ale_sup,
disksup,standard_error,ale,ale_dynamic_sup,timer_server,
standard_error_sup,sasl_sup,'sink-ns_log',user,
'sink-disk_debug','sink-disk_error']},
{cookie,nocookie},
{wordsize,4},
{wall_clock,1}]
[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:ns_server_cluster_sup:log_os_info:start_link:27] Manifest:
["<?xml version=\"1.0\" encoding=\"UTF-8\"?>","",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" "," ",
" ",
" ",
" ",
" ",""]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.129.0>},
{name,timeout_diag_logger},
{mfargs,{timeout_diag_logger,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:dist_manager:dist_manager:read_address_config:52] reading ip config from "c:/Program Files/Couchbase/Server/var/lib/couchbase/ip"
[ns_server:error] [2012-09-06 8:23:34] [ns_1@192.168.226.129:dist_manager:dist_manager:read_address_config:58] Got error:einval. Ignoring bad address:[]
[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:dist_manager:dist_manager:bringup:145] Attempting to bring up net_kernel with name 'ns_1@127.0.0.1'
[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:dist_manager:dist_manager:bringup:153] Attempted to save node name to disk: undefined
[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.130.0>},
{name,dist_manager},
{mfargs,{dist_manager,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.131.0>},
{name,ns_cookie_manager},
{mfargs,{ns_cookie_manager,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.132.0>},
{name,ns_cluster},
{mfargs,{ns_cluster,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mb_mnesia_sup}
started: [{pid,<0.134.0>},
{name,mb_mnesia_events},
{mfargs,
{gen_event,start_link,[{local,mb_mnesia_events}]}},
{restart_type,permanent},
{shutdown,10},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_sup}
started: [{pid,<0.141.0>},
{name,mnesia_event},
{mfargs,{mnesia_sup,start_event,[]}},
{restart_type,permanent},
{shutdown,30000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.143.0>},
{name,mnesia_monitor},
{mfargs,{mnesia_monitor,start,[]}},
{restart_type,permanent},
{shutdown,3000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.144.0>},
{name,mnesia_subscr},
{mfargs,{mnesia_subscr,start,[]}},
{restart_type,permanent},
{shutdown,3000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.145.0>},
{name,mnesia_locker},
{mfargs,{mnesia_locker,start,[]}},
{restart_type,permanent},
{shutdown,3000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.146.0>},
{name,mnesia_recover},
{mfargs,{mnesia_recover,start,[]}},
{restart_type,permanent},
{shutdown,180000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.147.0>},
{name,mnesia_tm},
{mfargs,{mnesia_tm,start,[]}},
{restart_type,permanent},
{shutdown,30000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.148.0>},
{name,mnesia_checkpoint_sup},
{mfargs,{mnesia_checkpoint_sup,start,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.149.0>},
{name,mnesia_snmp_sup},
{mfargs,{mnesia_snmp_sup,start,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.150.0>},
{name,mnesia_controller},
{mfargs,{mnesia_controller,start,[]}},
{restart_type,permanent},
{shutdown,3000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_kernel_sup}
started: [{pid,<0.151.0>},
{name,mnesia_late_loader},
{mfargs,{mnesia_late_loader,start,[]}},
{restart_type,permanent},
{shutdown,3000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mnesia_sup}
started: [{pid,<0.142.0>},
{name,mnesia_kernel_sup},
{mfargs,{mnesia_kernel_sup,start,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
application: mnesia
started_at: 'ns_1@192.168.226.129'

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,kernel_safe_sup}
started: [{pid,<0.157.0>},
{name,dets_sup},
{mfargs,{dets_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,kernel_safe_sup}
started: [{pid,<0.158.0>},
{name,dets},
{mfargs,{dets_server,start_link,[]}},
{restart_type,permanent},
{shutdown,2000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:ensure_schema:410] Committed schema to disk.
[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:init:246] Current config:
[{access_module,mnesia},
{auto_repair,true},
{backup_module,mnesia_backup},
{checkpoints,[]},
{db_nodes,['ns_1@192.168.226.129']},
{debug,verbose},
{directory,"c:/Program Files/Couchbase/Server/var/lib/couchbase/mnesia"},
{dump_log_load_regulation,false},
{dump_log_time_threshold,180000},
{dump_log_update_in_place,true},
{dump_log_write_threshold,1000},
{embedded_mnemosyne,false},
{event_module,mnesia_event},
{extra_db_nodes,[]},
{fallback_activated,false},
{held_locks,[]},
{ignore_fallback_at_startup,false},
{fallback_error_function,{mnesia,lkill}},
{is_running,yes},
{local_tables,[local_config,cluster,schema]},
{lock_queue,[]},
{log_version,"4.3"},
{master_node_tables,[]},
{max_wait_for_decision,10000},
{protocol_version,{7,6}},
{running_db_nodes,['ns_1@192.168.226.129']},
{schema_location,opt_disc},
{schema_version,{3,0}},
{subscribers,[<0.141.0>,<0.135.0>]},
{tables,[local_config,cluster,schema]},
{transaction_commits,5},
{transaction_failures,0},
{transaction_log_writes,3},
{transaction_restarts,0},
{transactions,[]},
{use_dir,true},
{core_dir,false},
{no_table_loaders,2},
{dc_dump_limit,4},
{send_compressed,0},
{version,"4.4.17"}]
Peers: ['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:167] Info from Mnesia:
Create Directory "c:/Program Files/Couchbase/Server/var/lib/couchbase/mnesia"

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mb_mnesia_sup}
started: [{pid,<0.135.0>},
{name,mb_mnesia},
{mfargs,{mb_mnesia,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,schema,
[{name,schema},
{type,set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,false},
{record_name,schema},
{attributes,[table,cstruct]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912614,831000},'ns_1@192.168.226.129'}},
{version,{{3,0},
{'ns_1@192.168.226.129',{1346,912614,862002}}}}]},
{tid,3,<0.154.0>}}
[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.133.0>},
{name,mb_mnesia_sup},
{mfargs,{mb_mnesia_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:195] Peers: ['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:195] Peers: ['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,local_config,
[{name,local_config},
{type,set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,local_config},
{attributes,[key,val]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912614,956002},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,5,<0.177.0>}}
[ns_server:debug] [2012-09-06 8:23:34] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,local_config,
[{name,local_config},
{type,set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,local_config},
{attributes,[key,val]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912614,956002},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,5,<0.177.0>}}
[ns_server:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:ns_config_sup:ns_config_sup:init:32] loading static ns_config from "c:/Program Files/Couchbase/Server/etc/couchbase/config"
[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_config_sup}
started: [{pid,<0.184.0>},
{name,ns_config_events},
{mfargs,
{gen_event,start_link,[{local,ns_config_events}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_config_sup}
started: [{pid,<0.185.0>},
{name,ns_config_events_local},
{mfargs,
{gen_event,start_link,
[{local,ns_config_events_local}]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:load_config:621] Loading static config from "c:/Program Files/Couchbase/Server/etc/couchbase/config"
[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:load_config:635] Loading dynamic config from "c:/Program Files/Couchbase/Server/var/lib/couchbase/config/config.dat"
[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:load_config:639] No dynamic config file found. Assuming we're brand new node
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:load_config:642] Here's full dynamic config we loaded:
[]
[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:load_config:653] Here's full dynamic config we loaded + static & default config:
[{auto_failover_cfg,[{enabled,false},{timeout,30},{max_nodes,1},{count,0}]},
{replication,[{enabled,true}]},
{email_alerts,
[{recipients,["root@localhost"]},
{sender,"couchbase@localhost"},
{enabled,false},
{email_server,
[{user,[]},{pass,[]},{host,"localhost"},{port,25},{encrypt,false}]},
{alerts,
[auto_failover_node,auto_failover_maximum_reached,
auto_failover_other_nodes_down,auto_failover_cluster_too_small]}]},
{{node,'ns_1@192.168.226.129',ns_log},
[{filename,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log"}]},
{{node,'ns_1@192.168.226.129',port_servers},
[{moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
{"port_listen=~B,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
[port]},
"-z",
{"url=http://127.0.0.1:~B/pools/default/saslBucketsStreaming",
[{misc,this_node_rest_port,[]}]},
"-p","0","-Y","y","-O","stderr",
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",{"~s",[{ns_moxi_sup,rest_user,[]}]}},
{"MOXI_SASL_PLAIN_PWD",{"~s",[{ns_moxi_sup,rest_pass,[]}]}}]},
use_stdio,exit_status,port_server_send_eol,stderr_to_stdout,stream]},
{memcached,"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l",
{"0.0.0.0:~B,0.0.0.0:~B:1000",[port,dedicated_port]},
"-p",
{"~B",[port]},
"-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
{"admin=~s;default_bucket_name=default;auto_create=false",
[admin_user]},
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",{"~s",[{isasl,path}]}},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,port_server_send_eol,stream]}]},
{{node,'ns_1@192.168.226.129',moxi},[{port,11211},{verbosity,[]}]},
{buckets,[{configs,[]}]},
{memory_quota,1637},
{{node,'ns_1@192.168.226.129',memcached},
[{port,11210},
{dedicated_port,11209},
{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{verbosity,[]}]},
{{node,'ns_1@192.168.226.129',isasl},
[{path,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"}]},
{rest_creds,[{creds,[]}]},
{{node,'ns_1@192.168.226.129',rest},[{port,8091},{port_meta,global}]},
{rest,[{port,8091}]},
{{node,'ns_1@192.168.226.129',membership},active},
{nodes_wanted,['ns_1@192.168.226.129']}]
[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config_default:upgrade_config_from_1_6_to_1_7:279] Upgrading config from 1.6 to 1.7
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:do_upgrade_config:420] Upgrading config by changes:
[{set,{node,'ns_1@192.168.226.129',config_version},{1,7}},
{set,
{node,'ns_1@192.168.226.129',memcached},
[{port,11210},
{dedicated_port,11209},
{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{verbosity,[]}]},
{set,
{node,'ns_1@192.168.226.129',ns_log},
[{filename,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log"}]},
{set,
{node,'ns_1@192.168.226.129',isasl},
[{path,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"}]},
{set,directory,"c:/Program Files/Couchbase/Server/var/lib/couchbase/config"}]

[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config_default:upgrade_config_from_1_7_to_1_7_1:302] Upgrading config from 1.7 to 1.7.1
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:do_upgrade_config:420] Upgrading config by changes:
[{set,{node,'ns_1@192.168.226.129',config_version},{1,7,1}},
{set,email_alerts,
[{recipients,["root@localhost"]},
{sender,"couchbase@localhost"},
{enabled,false},
{email_server,[{user,[]},
{pass,[]},
{host,"localhost"},
{port,25},
{encrypt,false}]},
{alerts,[auto_failover_node,auto_failover_maximum_reached,
auto_failover_other_nodes_down,
auto_failover_cluster_too_small]}]},
{set,auto_failover_cfg,
[{enabled,false},{timeout,30},{max_nodes,1},{count,0}]}]

[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config_default:upgrade_config_from_1_7_1_to_1_7_2:313] Upgrading config from 1.7.1 to 1.7.2
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:do_upgrade_config:420] Upgrading config by changes:
[{set,{node,'ns_1@192.168.226.129',config_version},{1,7,2}}]

[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config_default:upgrade_config_from_1_7_2_to_1_8_0:367] Upgrading config from 1.7.2 to 1.8.0
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:do_upgrade_config:420] Upgrading config by changes:
[{set,{node,'ns_1@192.168.226.129',config_version},{1,8,0}},
{set,{node,'ns_1@192.168.226.129',port_servers},
[{moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
{"port_listen=~B,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
[port]},
"-z",
{"url=http://127.0.0.1:~B/pools/default/saslBucketsStreaming",
[{misc,this_node_rest_port,[]}]},
"-p","0","-Y","y","-O","stderr",
{"~s",[verbosity]}],
[{env,[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",
{"~s",[{ns_moxi_sup,rest_user,[]}]}},
{"MOXI_SASL_PLAIN_PWD",
{"~s",[{ns_moxi_sup,rest_pass,[]}]}}]},
use_stdio,exit_status,port_server_send_eol,stderr_to_stdout,
stream]},
{memcached,"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l",
{"0.0.0.0:~B,0.0.0.0:~B:1000",[port,dedicated_port]},
"-p",
{"~B",[port]},
"-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
{"admin=~s;default_bucket_name=default;auto_create=false",
[admin_user]},
{"~s",[verbosity]}],
[{env,[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",{"~s",[{isasl,path}]}},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}]}]

[ns_server:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config_default:upgrade_config_from_1_8_0_to_1_8_1:404] Upgrading config from 1.8.0 to 1.8.1
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:do_upgrade_config:420] Upgrading config by changes:
[{set,{node,'ns_1@192.168.226.129',config_version},{1,8,1}},
{set,vbucket_map_history,[]},
{set,
{node,'ns_1@192.168.226.129',memcached},
[{dedicated_port,11209},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{port,11210},
{dedicated_port,11209},
{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{verbosity,[]}]},
{set,
{node,'ns_1@192.168.226.129',isasl},
[{path,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"}]},
{set,
{node,'ns_1@192.168.226.129',port_servers},
[{moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
{"port_listen=~B,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
[port]},
"-z",
{"url=http://127.0.0.1:~B/pools/default/saslBucketsStreaming",
[{misc,this_node_rest_port,[]}]},
"-p","0","-Y","y","-O","stderr",
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",{"~s",[{ns_moxi_sup,rest_user,[]}]}},
{"MOXI_SASL_PLAIN_PWD",{"~s",[{ns_moxi_sup,rest_pass,[]}]}}]},
use_stdio,exit_status,port_server_send_eol,stderr_to_stdout,
stream]},
{memcached,"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l",
{"0.0.0.0:~B,0.0.0.0:~B:1000",[port,dedicated_port]},
"-p",
{"~B",[port]},
"-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
{"admin=~s;default_bucket_name=default;auto_create=false",
[admin_user]},
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",{"~s",[{isasl,path}]}},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,port_server_send_eol,
stream]}]},
{set,
{node,'ns_1@192.168.226.129',ns_log},
[{filename,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log"}]}]

[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config:ns_config:do_init:465] Upgraded initial config:
{config,
{full,"c:/Program Files/Couchbase/Server/etc/couchbase/config",undefined,
ns_config_default},
[[],
[{directory,"c:/Program Files/Couchbase/Server/var/lib/couchbase/config"},
{nodes_wanted,['ns_1@192.168.226.129']},
{{node,'ns_1@192.168.226.129',membership},active},
{rest,[{port,8091}]},
{{node,'ns_1@192.168.226.129',rest},[{port,8091},{port_meta,global}]},
{rest_creds,[{creds,[]}]},
{{node,'ns_1@192.168.226.129',isasl},
[{path,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"}]},
{{node,'ns_1@192.168.226.129',memcached},
[{port,11210},
{dedicated_port,11209},
{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{verbosity,[]}]},
{memory_quota,1637},
{buckets,[{configs,[]}]},
{{node,'ns_1@192.168.226.129',moxi},[{port,11211},{verbosity,[]}]},
{{node,'ns_1@192.168.226.129',port_servers},
[{moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
{"port_listen=~B,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
[port]},
"-z",
{"url=http://127.0.0.1:~B/pools/default/saslBucketsStreaming",
[{misc,this_node_rest_port,[]}]},
"-p","0","-Y","y","-O","stderr",
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",{"~s",[{ns_moxi_sup,rest_user,[]}]}},
{"MOXI_SASL_PLAIN_PWD",
{"~s",[{ns_moxi_sup,rest_pass,[]}]}}]},
use_stdio,exit_status,port_server_send_eol,stderr_to_stdout,
stream]},
{memcached,"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l",
{"0.0.0.0:~B,0.0.0.0:~B:1000",[port,dedicated_port]},
"-p",
{"~B",[port]},
"-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
{"admin=~s;default_bucket_name=default;auto_create=false",
[admin_user]},
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",{"~s",[{isasl,path}]}},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,port_server_send_eol,
stream]}]},
{{node,'ns_1@192.168.226.129',ns_log},
[{filename,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log"}]},
{email_alerts,
[{recipients,["root@localhost"]},
{sender,"couchbase@localhost"},
{enabled,false},
{email_server,
[{user,[]},
{pass,[]},
{host,"localhost"},
{port,25},
{encrypt,false}]},
{alerts,
[auto_failover_node,auto_failover_maximum_reached,
auto_failover_other_nodes_down,
auto_failover_cluster_too_small]}]},
{replication,[{enabled,true}]},
{auto_failover_cfg,
[{enabled,false},{timeout,30},{max_nodes,1},{count,0}]}]],
[[{vbucket_map_history,
[{'_vclock',[{'ns_1@192.168.226.129',{1,63514131815}}]}]},
{directory,
[{'_vclock',[{'ns_1@192.168.226.129',{1,63514131815}}]},
99,58,47,80,114,111,103,114,97,109,32,70,105,108,101,115,47,67,111,
117,99,104,98,97,115,101,47,83,101,114,118,101,114,47,118,97,114,
47,108,105,98,47,99,111,117,99,104,98,97,115,101,47,99,111,110,102,
105,103]},
{{node,'ns_1@192.168.226.129',config_version},
[{'_vclock',[{'ns_1@192.168.226.129',{5,63514131815}}]}|{1,8,1}]},
{auto_failover_cfg,
[{'_vclock',[{'ns_1@192.168.226.129',{1,63514131815}}]},
{enabled,false},
{timeout,30},
{max_nodes,1},
{count,0}]},
{buckets,[{configs,[]}]},
{email_alerts,
[{'_vclock',[{'ns_1@192.168.226.129',{1,63514131815}}]},
{recipients,["root@localhost"]},
{sender,"couchbase@localhost"},
{enabled,false},
{email_server,
[{user,[]},
{pass,[]},
{host,"localhost"},
{port,25},
{encrypt,false}]},
{alerts,
[auto_failover_node,auto_failover_maximum_reached,
auto_failover_other_nodes_down,
auto_failover_cluster_too_small]}]},
{memory_quota,1637},
{nodes_wanted,['ns_1@192.168.226.129']},
{replication,[{enabled,true}]},
{rest,[{port,8091}]},
{rest_creds,[{creds,[]}]},
{{node,'ns_1@192.168.226.129',isasl},
[{'_vclock',[{'ns_1@192.168.226.129',{2,63514131815}}]},
{path,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"}]},
{{node,'ns_1@192.168.226.129',membership},active},
{{node,'ns_1@192.168.226.129',memcached},
[{'_vclock',[{'ns_1@192.168.226.129',{2,63514131815}}]},
{dedicated_port,11209},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{port,11210},
{dedicated_port,11209},
{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{verbosity,[]}]},
{{node,'ns_1@192.168.226.129',moxi},[{port,11211},{verbosity,[]}]},
{{node,'ns_1@192.168.226.129',ns_log},
[{'_vclock',[{'ns_1@192.168.226.129',{2,63514131815}}]},
{filename,
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log"}]},
{{node,'ns_1@192.168.226.129',port_servers},
[{'_vclock',[{'ns_1@192.168.226.129',{2,63514131815}}]},
{moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
{"port_listen=~B,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
[port]},
"-z",
{"url=http://127.0.0.1:~B/pools/default/saslBucketsStreaming",
[{misc,this_node_rest_port,[]}]},
"-p","0","-Y","y","-O","stderr",
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",{"~s",[{ns_moxi_sup,rest_user,[]}]}},
{"MOXI_SASL_PLAIN_PWD",
{"~s",[{ns_moxi_sup,rest_pass,[]}]}}]},
use_stdio,exit_status,port_server_send_eol,stderr_to_stdout,
stream]},
{memcached,"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l",
{"0.0.0.0:~B,0.0.0.0:~B:1000",[port,dedicated_port]},
"-p",
{"~B",[port]},
"-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
{"admin=~s;default_bucket_name=default;auto_create=false",
[admin_user]},
{"~s",[verbosity]}],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",{"~s",[{isasl,path}]}},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,port_server_send_eol,
stream]}]},
{{node,'ns_1@192.168.226.129',rest},[{port,8091},{port_meta,global}]}]],
ns_config_default,
{ns_config,save_config_sync,[]},
undefined,false}

[error_logger:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_config_sup}
started: [{pid,<0.186.0>},
{name,ns_config},
{mfargs,
{ns_config,start_link,
["c:/Program Files/Couchbase/Server/etc/couchbase/config",
ns_config_default]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_config_sup}
started: [{pid,<0.189.0>},
{name,ns_config_remote},
{mfargs,
{ns_config_replica,start_link,
[{local,ns_config_remote}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config_isasl_sync:ns_config_isasl_sync:init:53] isasl_sync init: ["c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw",
"_admin","_admin"]
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config_isasl_sync:ns_config_isasl_sync:init:61] isasl_sync init buckets: []
[ns_server:debug] [2012-09-06 8:23:35] [ns_1@192.168.226.129:ns_config_isasl_sync:ns_config_isasl_sync:writeSASLConf:133] Writing isasl passwd file: "c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"
[error_logger:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,inet_gethost_native_sup}
started: [{pid,<0.194.0>},{mfa,{inet_gethost_native,init,[[]]}}]

[error_logger:info] [2012-09-06 8:23:35] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,kernel_safe_sup}
started: [{pid,<0.193.0>},
{name,inet_gethost_native_sup},
{mfargs,{inet_gethost_native,start_link,[]}},
{restart_type,temporary},
{shutdown,1000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:23:36] [ns_1@192.168.226.129:ns_config_isasl_sync:ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_config_sup}
started: [{pid,<0.190.0>},
{name,ns_config_isasl_sync},
{mfargs,{ns_config_isasl_sync,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_config_sup}
started: [{pid,<0.195.0>},
{name,ns_config_log},
{mfargs,{ns_config_log,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.183.0>},
{name,ns_config_sup},
{mfargs,{ns_config_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.197.0>},
{name,vbucket_filter_changes_registry},
{mfargs,
{ns_process_registry,start_link,
[vbucket_filter_changes_registry]}},
{restart_type,permanent},
{shutdown,100},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_log:ns_log:init:64] Couldn't load logs from "c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log" (perhaps it's first startup): {error,
enoent}
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.199.0>},
{name,ns_log},
{mfargs,{ns_log,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.200.0>},
{name,ns_config_ets_dup},
{mfargs,{ns_config_ets_dup,start_link,[]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.201.0>},
{name,ns_log_events},
{mfargs,{gen_event,start_link,[{local,ns_log_events}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_node_disco_sup}
started: [{pid,<0.204.0>},
{name,ns_node_disco_events},
{mfargs,
{gen_event,start_link,
[{local,ns_node_disco_events}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_node_disco:ns_node_disco:init:98] Initting ns_node_disco with []
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_cookie_manager:ns_cookie_manager:do_cookie_sync:115] ns_cookie_manager do_cookie_sync
[user:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_cookie_manager:ns_cookie_manager:do_cookie_init:91] Initial otp cookie generated: wvssrcakuifibgjg
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
otp ->
[{cookie,wvssrcakuifibgjg}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.206.0>:ns_node_disco:do_nodes_wanted_updated_fun:182] ns_node_disco: nodes_wanted updated: ['ns_1@192.168.226.129'], with cookie: wvssrcakuifibgjg
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.206.0>:ns_node_disco:do_nodes_wanted_updated_fun:188] ns_node_disco: nodes_wanted pong: ['ns_1@192.168.226.129'], with cookie: wvssrcakuifibgjg
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_node_disco_sup}
started: [{pid,<0.205.0>},
{name,ns_node_disco},
{mfargs,{ns_node_disco,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_node_disco_sup}
started: [{pid,<0.209.0>},
{name,ns_node_disco_log},
{mfargs,{ns_node_disco_log,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_node_disco_sup}
started: [{pid,<0.210.0>},
{name,ns_node_disco_conf_events},
{mfargs,{ns_node_disco_conf_events,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:init:63] init pulling

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:init:65] init pushing

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:init:69] init reannouncing

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_events:ns_node_disco_conf_events:handle_event:50] ns_node_disco_conf_events config on otp
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
otp ->
[{cookie,wvssrcakuifibgjg}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
vbucket_map_history ->
[]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
directory ->
"c:/Program Files/Couchbase/Server/var/lib/couchbase/config"
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',config_version} ->
{1,8,1}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
auto_failover_cfg ->
[{enabled,false},{timeout,30},{max_nodes,1},{count,0}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
buckets ->
[{configs,[]}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
email_alerts ->
[{recipients,["root@localhost"]},
{sender,"couchbase@localhost"},
{enabled,false},
{email_server,[{user,[]},
{pass,[]},
{host,"localhost"},
{port,25},
{encrypt,false}]},
{alerts,[auto_failover_node,auto_failover_maximum_reached,
auto_failover_other_nodes_down,auto_failover_cluster_too_small]}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
memory_quota ->
1637
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_cookie_manager:ns_cookie_manager:do_cookie_sync:115] ns_cookie_manager do_cookie_sync
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_events:ns_node_disco_conf_events:handle_event:44] ns_node_disco_conf_events config on nodes_wanted
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
nodes_wanted ->
['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
replication ->
[{enabled,true}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
rest ->
[{port,8091}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_cookie_manager:ns_cookie_manager:do_cookie_sync:115] ns_cookie_manager do_cookie_sync
[ns_server:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:handle_info:57] config change: rest_creds -> ********
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.214.0>:ns_node_disco:do_nodes_wanted_updated_fun:182] ns_node_disco: nodes_wanted updated: ['ns_1@192.168.226.129'], with cookie: wvssrcakuifibgjg
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',isasl} ->
[{path,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.215.0>:ns_node_disco:do_nodes_wanted_updated_fun:182] ns_node_disco: nodes_wanted updated: ['ns_1@192.168.226.129'], with cookie: wvssrcakuifibgjg
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',membership} ->
active
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.214.0>:ns_node_disco:do_nodes_wanted_updated_fun:188] ns_node_disco: nodes_wanted pong: ['ns_1@192.168.226.129'], with cookie: wvssrcakuifibgjg
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.215.0>:ns_node_disco:do_nodes_wanted_updated_fun:188] ns_node_disco: nodes_wanted pong: ['ns_1@192.168.226.129'], with cookie: wvssrcakuifibgjg
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',memcached} ->
[{dedicated_port,11209},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{port,11210},
{dedicated_port,11209},
{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{verbosity,[]}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',moxi} ->
[{port,11211},{verbosity,[]}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',ns_log} ->
[{filename,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/ns_log"}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',port_servers} ->
[{moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
{"port_listen=~B,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
[port]},
"-z",
{"url=http://127.0.0.1:~B/pools/default/saslBucketsStreaming",
[{misc,this_node_rest_port,[]}]},
"-p","0","-Y","y","-O","stderr",
{"~s",[verbosity]}],
[{env,[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",{"~s",[{ns_moxi_sup,rest_user,[]}]}},
{"MOXI_SASL_PLAIN_PWD",{"~s",[{ns_moxi_sup,rest_pass,[]}]}}]},
use_stdio,exit_status,port_server_send_eol,stderr_to_stdout,stream]},
{memcached,"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l",
{"0.0.0.0:~B,0.0.0.0:~B:1000",[port,dedicated_port]},
"-p",
{"~B",[port]},
"-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
{"admin=~s;default_bucket_name=default;auto_create=false",
[admin_user]},
{"~s",[verbosity]}],
[{env,[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",{"~s",[{isasl,path}]}},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,port_server_send_eol,
stream]}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',rest} ->
[{port,8091},{port_meta,global}]
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([auto_failover_cfg,buckets,directory,
email_alerts,memory_quota,nodes_wanted]..)
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_node_disco_sup}
started: [{pid,<0.211.0>},
{name,ns_config_rep},
{mfargs,{ns_config_rep,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.203.0>},
{name,ns_node_disco_sup},
{mfargs,{ns_node_disco_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.217.0>},
{name,ns_tick_event},
{mfargs,{gen_event,start_link,[{local,ns_tick_event}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.218.0>},
{name,mb_master_events},
{mfargs,
{gen_event,start_link,[{local,mb_master_events}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_server_sup:mb_master:check_master_takeover_needed:150] Was unable to discover master, not going to force mastership takeover
[user:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_master:mb_master:init:89] I'm the only node, so I'm the master.
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_master_sup:misc:start_singleton:824] start_singleton(gen_fsm, ns_orchestrator, [], []): started as <0.225.0> on 'ns_1@192.168.226.129'

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mb_master_sup}
started: [{pid,<0.225.0>},
{name,ns_orchestrator},
{mfargs,{ns_orchestrator,start_link,[]}},
{restart_type,permanent},
{shutdown,20},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_master_sup:misc:start_singleton:824] start_singleton(gen_server, ns_tick, [], []): started as <0.226.0> on 'ns_1@192.168.226.129'

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mb_master_sup}
started: [{pid,<0.226.0>},
{name,ns_tick},
{mfargs,{ns_tick,start_link,[]}},
{restart_type,permanent},
{shutdown,10},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.227.0>:auto_failover:init:120] init auto_failover.
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_master_sup:misc:start_singleton:824] start_singleton(gen_server, auto_failover, [], []): started as <0.227.0> on 'ns_1@192.168.226.129'

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,mb_master_sup}
started: [{pid,<0.227.0>},
{name,auto_failover},
{mfargs,{auto_failover,start_link,[]}},
{restart_type,permanent},
{shutdown,10},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.222.0>},
{name,mb_master},
{mfargs,{mb_master,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.228.0>},
{name,master_activity_events},
{mfargs,
{gen_event,start_link,
[{local,master_activity_events}]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.229.0>},
{name,master_activity_events_ingress},
{mfargs,
{gen_event,start_link,
[{local,master_activity_events_ingress}]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.230.0>},
{name,master_activity_events_timestamper},
{mfargs,
{master_activity_events,start_link_timestamper,[]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.231.0>},
{name,master_activity_events_pids_watcher},
{mfargs,
{master_activity_events_pids_watcher,start_link,
[]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.232.0>},
{name,master_activity_events_keeper},
{mfargs,{master_activity_events_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.234.0>},
{name,buckets_events},
{mfargs,
{gen_event,start_link,[{local,buckets_events}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_mail_sup}
started: [{pid,<0.236.0>},
{name,ns_mail},
{mfargs,{ns_mail,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_log_events:ns_mail_log:init:45] ns_mail_log started up
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_mail_sup}
started: [{pid,<0.237.0>},
{name,ns_mail_log},
{mfargs,{ns_mail_log,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.235.0>},
{name,ns_mail_sup},
{mfargs,{ns_mail_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.238.0>},
{name,ns_heart},
{mfargs,{ns_heart,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.242.0>},
{name,ns_doctor},
{mfargs,{ns_doctor,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:error] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:144] Failed to grab system stats:
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-@system','ns_1@192.168.226.129'},
{latest,"minute"}]}}}

[ns_doctor:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_doctor:ns_doctor:handle_info:89] Got initial status [{'ns_1@192.168.226.129',
[{last_heard,{1346,912617,374000}},
{active_buckets,[]},
{ready_buckets,[]},
{replication,[]},
{memory,
[{total,10351120},
{processes,2901836},
{processes_used,2898740},
{system,7449284},
{atom,648601},
{atom_used,645848},
{binary,109296},
{code,5188337},
{ets,317900}]},
{system_stats,
[{cpu_utilization_rate,0},
{swap_total,0},
{swap_used,0}]},
{interesting_stats,[]},
{cluster_compatibility_version,1},
{version,
[{ale,"8cffe61"},
{os_mon,"2.2.5"},
{mnesia,"4.4.17"},
{kernel,"2.14.3"},
{sasl,"2.1.9.3"},
{ns_server,"1.8.1-937-rel-enterprise"},
{stdlib,"1.17.3"}]},
{system_arch,"windows"},
{wall_clock,4},
{memory_data,{2146951168,909393920,{<0.7.0>,159012}}},
{disk_data,[{"C:\\",41940988,33}]},
{system_memory_data,
[{total_memory,2146951168},
{free_memory,1233145856},
{system_total_memory,2146951168}]},
{statistics,
[{wall_clock,{3994,2699}},
{context_switches,{20550,0}},
{garbage_collection,{4320,18869454,0}},
{io,{{input,5537395},{output,788426}}},
{reductions,{10776092,10776092}},
{run_queue,0},
{runtime,{1045,1045}}]}]}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.245.0>},
{name,menelaus_web},
{mfargs,{menelaus_web,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.247.0>},
{name,menelaus_event},
{mfargs,{menelaus_event,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.248.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.249.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[user:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:ns_server_sup:menelaus_sup:start_link:44] Couchbase Server has started on web port 8091 on node 'ns_1@192.168.226.129'.
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.244.0>},
{name,menelaus},
{mfargs,{menelaus_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.251.0>},
{name,ns_port_init},
{mfargs,{ns_port_init,start_link,[]}},
{restart_type,permanent},
{shutdown,10},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.252.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.254.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.252.0>},
{name,
{moxi,
"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
"port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
"-z",
"url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming",
"-p","0","-Y","y","-O","stderr",[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",[]},
{"MOXI_SASL_PLAIN_PWD",[]}]},
use_stdio,exit_status,port_server_send_eol,
stderr_to_stdout,stream]}},
{mfargs,
{supervisor_cushion,start_link,
[moxi,5000,ns_port_server,start_link,
[moxi,
"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
"port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
"-z",
"url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming",
"-p","0","-Y","y","-O","stderr",[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",[]},
{"MOXI_SASL_PLAIN_PWD",[]}]},
use_stdio,exit_status,port_server_send_eol,
stderr_to_stdout,stream]]]}},
{restart_type,permanent},
{shutdown,10000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.254.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.250.0>},
{name,ns_port_sup},
{mfargs,{ns_port_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,60000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.256.0>},
{name,ns_port_memcached_killer},
{mfargs,{ns_port_sup,start_memcached_force_killer,[]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.257.0>},
{name,ns_stats_event},
{mfargs,
{gen_event,start_link,[{local,ns_stats_event}]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.259.0>},
{name,ns_bucket_worker},
{mfargs,{work_queue,start_link,[ns_bucket_worker]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_bucket_sup}
started: [{pid,<0.261.0>},
{name,buckets_observing_subscription},
{mfargs,{ns_bucket_sup,subscribe_on_config_events,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.260.0>},
{name,ns_bucket_sup},
{mfargs,{ns_bucket_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.262.0>},
{name,system_stats_collector},
{mfargs,{system_stats_collector,start_link,[]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_call:109] Creating table 'stats_archiver-@system-minute'
[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.264.0>},
{name,{stats_archiver,"@system"}},
{mfargs,{stats_archiver,start_link,["@system"]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.266.0>},
{name,{stats_reader,"@system"}},
{mfargs,{stats_reader,start_link,["@system"]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.267.0>},
{name,ns_moxi_sup_work_queue},
{mfargs,
{work_queue,start_link,[ns_moxi_sup_work_queue]}},
{restart_type,permanent},
{shutdown,1000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_sup}
started: [{pid,<0.268.0>},
{name,ns_moxi_sup},
{mfargs,{ns_moxi_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_server_cluster_sup}
started: [{pid,<0.198.0>},
{name,ns_server_sup},
{mfargs,{ns_server_sup,start_link,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:23:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
application: ns_server
started_at: 'ns_1@192.168.226.129'

[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-minute',
[{name,'stats_archiver-@system-minute'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,514025},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,6,<0.271.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-minute',
[{name,'stats_archiver-@system-minute'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,514025},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,6,<0.271.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_call:109] Creating table 'stats_archiver-@system-hour'
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-hour',
[{name,'stats_archiver-@system-hour'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,530003},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,7,<0.278.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-hour',
[{name,'stats_archiver-@system-hour'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,530003},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,7,<0.278.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_call:109] Creating table 'stats_archiver-@system-day'
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-day',
[{name,'stats_archiver-@system-day'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,530012},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,8,<0.285.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-day',
[{name,'stats_archiver-@system-day'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,530012},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,8,<0.285.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_call:109] Creating table 'stats_archiver-@system-week'
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-week',
[{name,'stats_archiver-@system-week'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,546005},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,9,<0.292.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-week',
[{name,'stats_archiver-@system-week'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,546005},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,9,<0.292.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_call:109] Creating table 'stats_archiver-@system-month'
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-month',
[{name,'stats_archiver-@system-month'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,546014},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,10,<0.299.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-month',
[{name,'stats_archiver-@system-month'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,546014},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,10,<0.299.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_call:109] Creating table 'stats_archiver-@system-year'
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-year',
[{name,'stats_archiver-@system-year'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,561005},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,11,<0.306.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:mb_mnesia:mb_mnesia:handle_info:200] Mnesia table event:
{write,{schema,'stats_archiver-@system-year',
[{name,'stats_archiver-@system-year'},
{type,ordered_set},
{ram_copies,[]},
{disc_copies,['ns_1@192.168.226.129']},
{disc_only_copies,[]},
{load_order,0},
{access_mode,read_write},
{index,[]},
{snmp,[]},
{local_content,true},
{record_name,stat_entry},
{attributes,[timestamp,values]},
{user_properties,[]},
{frag_properties,[]},
{cookie,{{1346,912617,561005},'ns_1@192.168.226.129'}},
{version,{{2,0},[]}}]},
{tid,11,<0.306.0>}}
[ns_server:debug] [2012-09-06 8:23:37] [ns_1@192.168.226.129:<0.246.0>:menelaus_web:handle_streaming:673] Starting streaming for 127.0.0.1:49200 path /pools/default/saslBucketsStreaming
[user:info] [2012-09-06 8:23:40] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:23:40] [ns_1@192.168.226.129:<0.254.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:23:40] [ns_1@192.168.226.129:<0.254.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 2.86s

[error_logger:error] [2012-09-06 8:23:40] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.255.0> terminating
** Last message in was {#Port<0.4854>,{exit_status,3}}
** When Server state == {state,#Port<0.4854>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:23:40] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.255.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.254.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.4854>,normal}]
links: [<0.254.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[error_logger:error] [2012-09-06 8:23:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.254.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912617,468000},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:23:45] [ns_1@192.168.226.129:<0.333.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:23:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.254.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2394
neighbours:

[error_logger:error] [2012-09-06 8:23:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.254.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.333.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:23:46] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:23:46] [ns_1@192.168.226.129:<0.333.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[error_logger:error] [2012-09-06 8:23:46] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.334.0> terminating
** Last message in was {#Port<0.5285>,{exit_status,3}}
** When Server state == {state,#Port<0.5285>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[user:warn] [2012-09-06 8:23:46] [ns_1@192.168.226.129:<0.333.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.69s

[error_logger:error] [2012-09-06 8:23:46] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.334.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.333.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5285>,normal}]
links: [<0.333.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:info] [2012-09-06 8:23:46] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 1 times in the past 5.69025 secs (last seen 5.69025 secs ago
[error_logger:error] [2012-09-06 8:23:51] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.333.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912625,326256},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:23:51] [ns_1@192.168.226.129:<0.345.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:23:51] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.333.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:23:51] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.333.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:51] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.345.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:23:51] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:23:51] [ns_1@192.168.226.129:<0.345.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[ns_server:info] [2012-09-06 8:23:51] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 2 times in the past 11.18125 secs (last seen 5.491 secs ago
[user:warn] [2012-09-06 8:23:51] [ns_1@192.168.226.129:<0.345.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.47s

[error_logger:error] [2012-09-06 8:23:51] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.346.0> terminating
** Last message in was {#Port<0.5465>,{exit_status,3}}
** When Server state == {state,#Port<0.5465>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:23:51] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.346.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.345.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5465>,normal}]
links: [<0.345.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:debug] [2012-09-06 8:23:56] [ns_1@192.168.226.129:<0.361.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:23:56] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.345.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912631,36251},undefined}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:23:56] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.345.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:23:56] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.345.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:23:56] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.361.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:23:56] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:23:56] [ns_1@192.168.226.129:<0.361.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:23:56] [ns_1@192.168.226.129:<0.361.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.39s

[error_logger:error] [2012-09-06 8:23:56] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.362.0> terminating
** Last message in was {#Port<0.5472>,{exit_status,3}}
** When Server state == {state,#Port<0.5472>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[ns_server:info] [2012-09-06 8:23:56] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 3 times in the past 16.56325 secs (last seen 5.382 secs ago
[error_logger:error] [2012-09-06 8:23:56] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.362.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.361.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5472>,normal}]
links: [<0.361.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:debug] [2012-09-06 8:23:57] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([{node,'ns_1@192.168.226.129',memcached}]..)
[ns_server:debug] [2012-09-06 8:23:57] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([memory_quota]..)
[ns_server:debug] [2012-09-06 8:23:57] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
{node,'ns_1@192.168.226.129',memcached} ->
[{dbdir,"c:/Program Files/Couchbase/Server/var/lib/couchbase/data"},
{dedicated_port,11209},
{bucket_engine,
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so"},
{engines,
[{membase,
[{engine,"c:/Program Files/Couchbase/Server/lib/memcached/ep.so"},
{initfile,
"c:/Program Files/Couchbase/Server/etc/couchbase/init.sql"},
{static_config_string,
"vb0=false;waitforwarmup=false;failpartialwarmup=false;shardpattern=%d/%b-%i.mb;db_strategy=multiMTVBDB"}]},
{memcached,
[{engine,
"c:/Program Files/Couchbase/Server/lib/memcached/default_engine.so"},
{static_config_string,"vb0=true"}]}]},
{port,11210},
{dedicated_port,11209},
{admin_user,"_admin"},
{admin_pass,"_admin"},
{verbosity,[]}]
[ns_server:debug] [2012-09-06 8:23:57] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
memory_quota ->
500
[ns_server:error] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_config:ns_config:handle_call:573] Failed to update config: {exit,{not_found,"default"}}
Stacktrace:
[{ns_bucket,'-delete_bucket/1-fun-0-',2},
{misc,key_update_rec,4},
{ns_config,'-update_sub_key/3-fun-0-',3},
{ns_config,'-update_key/2-fun-0-',3},
{ns_config,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]
[ns_server:debug] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
buckets ->
[{configs,[[{map,[]},
{fastForwardMap,[]},
{sasl_password,[]},
{num_replicas,0},
{ram_quota,524288000},
{auth_type,sasl},
{type,membase},
{num_vbuckets,1024},
{servers,[]}]]}]
[menelaus:info] [2012-09-06 8:24:00] [ns_1@192.168.226.129:<0.325.0>:menelaus_web_buckets:do_bucket_create:266] Created bucket "default" of type: membase
[{num_replicas,0},{ram_quota,524288000},{auth_type,sasl}]
[ns_server:debug] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_config_isasl_sync:ns_config_isasl_sync:writeSASLConf:133] Writing isasl passwd file: "c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"
[ns_server:debug] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([buckets]..)
[ns_server:debug] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
buckets ->
[{configs,[{"default",
[{map,[]},
{fastForwardMap,[]},
{sasl_password,[]},
{num_replicas,0},
{ram_quota,524288000},
{auth_type,sasl},
{type,membase},
{num_vbuckets,1024},
{servers,['ns_1@192.168.226.129']}]}]}]
[ns_server:info] [2012-09-06 8:24:00] [ns_1@192.168.226.129:<0.379.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_bucket_worker:ns_bucket_sup:update_childs:84] Starting new child: {{per_bucket_sup,"default"},
{single_bucket_sup,start_link,["default"]},
permanent,infinity,supervisor,
[single_bucket_sup]}

[ns_server:debug] [2012-09-06 8:24:00] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([buckets]..)
[error_logger:info] [2012-09-06 8:24:00] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_bucket_sup}
started: [{pid,<0.385.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[ns_server:info] [2012-09-06 8:24:00] [ns_1@192.168.226.129:<0.253.0>:ns_port_server:log:166] moxi<0.253.0>: 2012-09-06 08:24:25: (agent_config.c.705) ERROR: bad JSON configuration from http://127.0.0.1:8091/pools/default/saslBucketsStreaming: Number of buckets must be a power of two > 0 and <= MAX_BUCKETS ({
moxi<0.253.0>: "name": "default",
moxi<0.253.0>: "nodeLocator": "vbucket",
moxi<0.253.0>: "saslPassword": "",
moxi<0.253.0>: "nodes": [{
moxi<0.253.0>: "hostname": "192.168.226.129:8091",
moxi<0.253.0>: "ports": {
moxi<0.253.0>: "direct": 11210,
moxi<0.253.0>: "proxy": 11211
moxi<0.253.0>: }
moxi<0.253.0>: }],
moxi<0.253.0>: "vBucketServerMap": {
moxi<0.253.0>: "hashAlgorithm": "CRC",
moxi<0.253.0>: "numReplicas": 0,
moxi<0.253.0>: "serverList": ["192.168.226.129:11210"],
moxi<0.253.0>: "vBucketMap": []
moxi<0.253.0>: }
moxi<0.253.0>: })

[error_logger:error] [2012-09-06 8:24:01] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.249.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:01] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.391.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:01] [ns_1@192.168.226.129:ns_config_isasl_sync:ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:warn] [2012-09-06 8:24:01] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:debug] [2012-09-06 8:24:01] [ns_1@192.168.226.129:<0.393.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:01] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.361.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912636,511256},undefined}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:01] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.361.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:24:01] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.361.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:01] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.393.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:error] [2012-09-06 8:24:02] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[user:info] [2012-09-06 8:24:02] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:02] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 4 times in the past 22.11625 secs (last seen 5.553 secs ago
[ns_server:info] [2012-09-06 8:24:02] [ns_1@192.168.226.129:<0.393.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:02] [ns_1@192.168.226.129:<0.393.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.53s

[error_logger:error] [2012-09-06 8:24:02] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.394.0> terminating
** Last message in was {#Port<0.5538>,{exit_status,3}}
** When Server state == {state,#Port<0.5538>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:02] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.394.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.393.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5538>,normal}]
links: [<0.393.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1596
neighbours:

[ns_server:warn] [2012-09-06 8:24:03] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:04] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.391.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:04] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.399.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:05] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:info] [2012-09-06 8:24:06] [ns_1@192.168.226.129:<0.379.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[ns_server:info] [2012-09-06 8:24:07] [ns_1@192.168.226.129:<0.225.0>:ns_orchestrator:handle_info:209] Skipping janitor in state janitor_running: {janitor_state,
["default"],
<0.379.0>}
[ns_server:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[error_logger:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.399.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.407.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.393.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912641,909256},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:24:07] [ns_1@192.168.226.129:<0.408.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.393.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2400
neighbours:

[error_logger:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.393.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.408.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:07] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[user:info] [2012-09-06 8:24:07] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:07] [ns_1@192.168.226.129:<0.408.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[error_logger:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.409.0> terminating
** Last message in was {#Port<0.5545>,{exit_status,3}}
** When Server state == {state,#Port<0.5545>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[ns_server:info] [2012-09-06 8:24:07] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 5 times in the past 27.56125 secs (last seen 5.445 secs ago
[user:warn] [2012-09-06 8:24:07] [ns_1@192.168.226.129:<0.408.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.44s

[error_logger:error] [2012-09-06 8:24:07] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.409.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.408.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5545>,normal}]
links: [<0.408.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1596
neighbours:

[ns_server:debug] [2012-09-06 8:24:07] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([settings]..)
[ns_server:debug] [2012-09-06 8:24:07] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
settings ->
[{stats,[{send_stats,false}]}]
[ns_server:warn] [2012-09-06 8:24:09] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.407.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.416.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,'single_bucket_sup-default'}
Context: start_error
Reason: couldnt_connect_to_memcached
Offender: [{pid,undefined},
{name,{ns_memcached,"default"}},
{mfargs,{ns_memcached,start_link,["default"]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:info] [2012-09-06 8:24:10] [ns_1@192.168.226.129:<0.379.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[error_logger:error] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_memcached:init/1
pid: <0.387.0>
registered_name: []
exception exit: couldnt_connect_to_memcached
in function gen_server:init_it/6
ancestors: ['single_bucket_sup-default',<0.385.0>]
messages: [check_started,check_started,
{'$gen_call',{<0.392.0>,
{#Ref<0.0.0.2857>,'ns_1@192.168.226.129'}},
connected},
check_started,
{'$gen_call',{<0.238.0>,#Ref<0.0.0.2910>},connected},
{'$gen_call',{<0.395.0>,#Ref<0.0.0.2912>},topkeys},
check_started,check_started,check_started,check_started,
check_started,check_started,check_started,check_started,
check_started,check_started,
{'$gen_call',{<0.238.0>,#Ref<0.0.0.3092>},connected},
check_started,
{'$gen_call',{<0.410.0>,
{#Ref<0.0.0.3138>,'ns_1@192.168.226.129'}},
connected},
check_started,check_started,check_started,check_started,
check_started,check_started]
links: [<0.64.0>,<0.386.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 28657
stack_size: 24
reductions: 7427
neighbours:

[error_logger:error] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {couldnt_connect_to_memcached,
{gen_server,call,['ns_memcached-default',topkeys,60000]}}
Offender: [{pid,<0.248.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_bucket_sup}
Context: child_terminated
Reason: {{badmatch,{error,shutdown}},
[{single_bucket_sup,'-start_link/1-fun-0-',2}]}
Offender: [{pid,<0.385.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_bucket_sup}
started: [{pid,<0.419.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.420.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:10] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] Error in process <0.385.0> on node 'ns_1@192.168.226.129' with exit value: {{badmatch,{error,shutdown}},[{single_bucket_sup,'-start_link/1-fun-0-',2}]}

[ns_server:warn] [2012-09-06 8:24:11] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:error] [2012-09-06 8:24:12] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[error_logger:error] [2012-09-06 8:24:12] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.408.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912647,447256},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:24:12] [ns_1@192.168.226.129:<0.428.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:12] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.408.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2400
neighbours:

[error_logger:error] [2012-09-06 8:24:12] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.408.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:12] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.428.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:24:13] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:13] [ns_1@192.168.226.129:<0.428.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:13] [ns_1@192.168.226.129:<0.428.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.37s

[ns_server:info] [2012-09-06 8:24:13] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 6 times in the past 32.94325 secs (last seen 5.382 secs ago
[error_logger:error] [2012-09-06 8:24:13] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.429.0> terminating
** Last message in was {#Port<0.5553>,{exit_status,3}}
** When Server state == {state,#Port<0.5553>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:13] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.429.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.428.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5553>,normal}]
links: [<0.428.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[error_logger:error] [2012-09-06 8:24:13] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.416.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:13] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.433.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:13] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:warn] [2012-09-06 8:24:15] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:16] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.433.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.438.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:debug] [2012-09-06 8:24:16] [ns_1@192.168.226.129:ns_config_rep:ns_config_rep:do_push_keys:312] Replicating some config keys ([rest,rest_creds]..)
[ns_server:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:ns_config_events:ns_port_sup:terminate_port:129] unsupervising port: {moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
"port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
"-z",
"url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming",
"-p","0","-Y","y","-O","stderr",[]],
[{env,[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR",[]},
{"MOXI_SASL_PLAIN_PWD",[]}]},
use_stdio,exit_status,port_server_send_eol,
stderr_to_stdout,stream]}
[ns_server:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.253.0>:ns_port_server:handle_info:104] Port server moxi exited with status 0
[ns_server:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.253.0>:ns_port_server:log:166] moxi<0.253.0>: EOL on stdin. Exiting

[ns_server:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:ns_config_events:ns_port_sup:launch_port:74] supervising port: {moxi,"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
"port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
"-z",
"url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming",
"-p","0","-Y","y","-O","stderr",[]],
[{env,[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR","Administrator"},
{"MOXI_SASL_PLAIN_PWD","aaAA1111"}]},
use_stdio,exit_status,port_server_send_eol,
stderr_to_stdout,stream]}
[ns_server:debug] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.442.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[menelaus:debug] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.246.0>:menelaus_web:handle_streaming:721] menelaus_web streaming socket closed by client
[ns_server:debug] [2012-09-06 8:24:16] [ns_1@192.168.226.129:ns_config_log:ns_config_log:log_common:111] config change:
rest ->
[{port,8091}]
[error_logger:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.442.0>},
{name,
{moxi,
"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
"port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
"-z",
"url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming",
"-p","0","-Y","y","-O","stderr",[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR","Administrator"},
{"MOXI_SASL_PLAIN_PWD","aaAA1111"}]},
use_stdio,exit_status,port_server_send_eol,
stderr_to_stdout,stream]}},
{mfargs,
{supervisor_cushion,start_link,
[moxi,5000,ns_port_server,start_link,
[moxi,
"c:/Program Files/Couchbase/Server/bin/moxi",
["-Z",
"port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200",
"-z",
"url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming",
"-p","0","-Y","y","-O","stderr",[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MOXI_SASL_PLAIN_USR","Administrator"},
{"MOXI_SASL_PLAIN_PWD","aaAA1111"}]},
use_stdio,exit_status,port_server_send_eol,
stderr_to_stdout,stream]]]}},
{restart_type,permanent},
{shutdown,10000},
{child_type,worker}]

[ns_server:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:ns_config_log:ns_config_log:handle_info:57] config change: rest_creds -> ********
[ns_server:info] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.379.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[stats:error] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.325.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.448.0>:ns_pubsub:do_subscribe_link:145] Deleting {ns_pubsub,#Ref<0.0.0.3771>} event handler: ok
[stats:error] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.312.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[ns_server:debug] [2012-09-06 8:24:16] [ns_1@192.168.226.129:<0.330.0>:menelaus_web:handle_streaming:673] Starting streaming for 127.0.0.1:49271 path /pools/default/saslBucketsStreaming
[ns_server:info] [2012-09-06 8:24:17] [ns_1@192.168.226.129:<0.443.0>:ns_port_server:log:166] moxi<0.443.0>: 2012-09-06 08:24:16: (cproxy_config.c.317) env: MOXI_SASL_PLAIN_USR (13)
moxi<0.443.0>: 2012-09-06 08:24:16: (cproxy_config.c.326) env: MOXI_SASL_PLAIN_PWD (8)
moxi<0.443.0>: 2012-09-06 08:24:18: (agent_config.c.705) ERROR: bad JSON configuration from http://127.0.0.1:8091/pools/default/saslBucketsStreaming: Number of buckets must be a power of two > 0 and <= MAX_BUCKETS ({
moxi<0.443.0>: "name": "default",
moxi<0.443.0>: "nodeLocator": "vbucket",
moxi<0.443.0>: "saslPassword": "",
moxi<0.443.0>: "nodes": [{
moxi<0.443.0>: "hostname": "192.168.226.129:8091",
moxi<0.443.0>: "ports": {
moxi<0.443.0>: "direct": 11210,
moxi<0.443.0>: "proxy": 11211
moxi<0.443.0>: }
moxi<0.443.0>: }],
moxi<0.443.0>: "vBucketServerMap": {
moxi<0.443.0>: "hashAlgorithm": "CRC",
moxi<0.443.0>: "numReplicas": 0,
moxi<0.443.0>: "serverList": ["192.168.226.129:11210"],
moxi<0.443.0>: "vBucketMap": []
moxi<0.443.0>: }
moxi<0.443.0>: })

[ns_server:info] [2012-09-06 8:24:17] [ns_1@192.168.226.129:<0.225.0>:ns_orchestrator:handle_info:209] Skipping janitor in state janitor_running: {janitor_state,
["default"],
<0.379.0>}
[ns_server:error] [2012-09-06 8:24:17] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[ns_server:warn] [2012-09-06 8:24:17] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:debug] [2012-09-06 8:24:18] [ns_1@192.168.226.129:<0.460.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:18] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.428.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912652,892256},undefined}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:18] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.428.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:24:18] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.428.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:18] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.460.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:24:18] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:18] [ns_1@192.168.226.129:<0.460.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:18] [ns_1@192.168.226.129:<0.460.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.42s

[error_logger:error] [2012-09-06 8:24:18] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.461.0> terminating
** Last message in was {#Port<0.5617>,{exit_status,3}}
** When Server state == {state,#Port<0.5617>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[ns_server:info] [2012-09-06 8:24:18] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 7 times in the past 38.37225 secs (last seen 5.429 secs ago
[error_logger:error] [2012-09-06 8:24:18] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.461.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.460.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5617>,normal}]
links: [<0.460.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:debug] [2012-09-06 8:24:18] [ns_1@192.168.226.129:<0.462.0>:ns_pubsub:do_subscribe_link:145] Deleting {ns_pubsub,#Ref<0.0.0.3997>} event handler: ok
[error_logger:error] [2012-09-06 8:24:19] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.438.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:19] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.467.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[stats:error] [2012-09-06 8:24:19] [ns_1@192.168.226.129:<0.329.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[ns_server:warn] [2012-09-06 8:24:19] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,'single_bucket_sup-default'}
Context: start_error
Reason: couldnt_connect_to_memcached
Offender: [{pid,undefined},
{name,{ns_memcached,"default"}},
{mfargs,{ns_memcached,start_link,["default"]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:20] [ns_1@192.168.226.129:<0.225.0>:ns_orchestrator:handle_info:218] Janitor run exited for bucket "default" with reason {noproc,
{gen_server,call,
[{'ns_memcached-default',
'ns_1@192.168.226.129'},
list_vbuckets_prevstate,
60000]}}

[error_logger:error] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_memcached:init/1
pid: <0.423.0>
registered_name: []
exception exit: couldnt_connect_to_memcached
in function gen_server:init_it/6
ancestors: ['single_bucket_sup-default',<0.419.0>]
messages: [check_started,
{'$gen_call',{<0.427.0>,
{#Ref<0.0.0.3355>,'ns_1@192.168.226.129'}},
connected},
check_started,check_started,
{'$gen_call',{<0.238.0>,#Ref<0.0.0.3388>},connected},
check_started,check_started,check_started,check_started,
check_started,check_started,check_started,
{'$gen_call',{<0.436.0>,#Ref<0.0.0.3539>},topkeys},
check_started,check_started,check_started,
{'$gen_call',{<0.238.0>,#Ref<0.0.0.3897>},connected},
check_started,
{'$gen_call',{<0.459.0>,
{#Ref<0.0.0.3902>,'ns_1@192.168.226.129'}},
connected},
check_started,check_started,check_started,check_started,
check_started,check_started]
links: [<0.64.0>,<0.422.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 28657
stack_size: 24
reductions: 7431
neighbours:

[error_logger:error] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {couldnt_connect_to_memcached,
{gen_server,call,['ns_memcached-default',topkeys,60000]}}
Offender: [{pid,<0.420.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_bucket_sup}
Context: child_terminated
Reason: {{badmatch,{error,shutdown}},
[{single_bucket_sup,'-start_link/1-fun-0-',2}]}
Offender: [{pid,<0.419.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_bucket_sup}
started: [{pid,<0.473.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:error] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_janitor:cleanup/2
pid: <0.379.0>
registered_name: []
exception exit: {noproc,
{gen_server,call,
[{'ns_memcached-default','ns_1@192.168.226.129'},
list_vbuckets_prevstate,60000]}}
in function gen_server:call/3
in call from ns_memcached:do_call/3
in call from ns_janitor:'-read_existing_map/4-lc$^0/1-0-'/2
in call from ns_janitor:read_existing_map/4
in call from ns_janitor:do_cleanup/3
ancestors: [<0.225.0>,mb_master_sup,mb_master,ns_server_sup,
ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.225.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 28657
stack_size: 24
reductions: 4412
neighbours:

[error_logger:info] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.472.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:20] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] Error in process <0.419.0> on node 'ns_1@192.168.226.129' with exit value: {{badmatch,{error,shutdown}},[{single_bucket_sup,'-start_link/1-fun-0-',2}]}

[ns_server:debug] [2012-09-06 8:24:20] [ns_1@192.168.226.129:<0.478.0>:ns_pubsub:do_subscribe_link:145] Deleting {ns_pubsub,#Ref<0.0.0.4207>} event handler: ok
[stats:error] [2012-09-06 8:24:21] [ns_1@192.168.226.129:<0.312.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[ns_server:warn] [2012-09-06 8:24:21] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:error] [2012-09-06 8:24:22] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[error_logger:error] [2012-09-06 8:24:22] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.467.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:22] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.488.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[stats:error] [2012-09-06 8:24:22] [ns_1@192.168.226.129:<0.325.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[error_logger:error] [2012-09-06 8:24:23] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.460.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912658,274256},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:24:23] [ns_1@192.168.226.129:<0.494.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:23] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.460.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:24:23] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.460.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:23] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.494.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:23] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[user:info] [2012-09-06 8:24:24] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:24] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 8 times in the past 43.73825 secs (last seen 5.366 secs ago
[ns_server:info] [2012-09-06 8:24:24] [ns_1@192.168.226.129:<0.494.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[error_logger:error] [2012-09-06 8:24:24] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.495.0> terminating
** Last message in was {#Port<0.5623>,{exit_status,3}}
** When Server state == {state,#Port<0.5623>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[user:warn] [2012-09-06 8:24:24] [ns_1@192.168.226.129:<0.494.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.36s

[error_logger:error] [2012-09-06 8:24:24] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.495.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.494.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5623>,normal}]
links: [<0.494.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[error_logger:error] [2012-09-06 8:24:25] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.488.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:25] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.499.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:26] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:error] [2012-09-06 8:24:27] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[ns_server:warn] [2012-09-06 8:24:28] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:28] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.499.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:28] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.511.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:29] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.494.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912663,702254},undefined}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:29] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.494.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[ns_server:debug] [2012-09-06 8:24:29] [ns_1@192.168.226.129:<0.512.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:29] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.494.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:29] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.512.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:24:29] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:29] [ns_1@192.168.226.129:<0.512.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:29] [ns_1@192.168.226.129:<0.512.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.37s

[ns_server:info] [2012-09-06 8:24:29] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 9 times in the past 49.12025 secs (last seen 5.382 secs ago
[ns_server:info] [2012-09-06 8:24:29] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log supervisor_cushion:1("Service memcached exited on node 'ns_1@192.168.226.129' in 0.37s\n") because it's been seen 1 times in the past 16.177 secs (last seen 16.177 secs ago
[error_logger:error] [2012-09-06 8:24:29] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.513.0> terminating
** Last message in was {#Port<0.5633>,{exit_status,3}}
** When Server state == {state,#Port<0.5633>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:29] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.513.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.512.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5633>,normal}]
links: [<0.512.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:warn] [2012-09-06 8:24:30] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:info] [2012-09-06 8:24:31] [ns_1@192.168.226.129:<0.503.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[error_logger:error] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,'single_bucket_sup-default'}
Context: start_error
Reason: couldnt_connect_to_memcached
Offender: [{pid,undefined},
{name,{ns_memcached,"default"}},
{mfargs,{ns_memcached,start_link,["default"]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_memcached:init/1
pid: <0.477.0>
registered_name: []
exception exit: couldnt_connect_to_memcached
in function gen_server:init_it/6
ancestors: ['single_bucket_sup-default',<0.473.0>]
messages: [check_started,check_started,
{'$gen_call',{<0.238.0>,#Ref<0.0.0.4372>},connected},
check_started,check_started,check_started,check_started,
check_started,check_started,check_started,check_started,
{'$gen_call',{<0.500.0>,#Ref<0.0.0.4589>},topkeys},
check_started,check_started,
{'$gen_call',{<0.505.0>,
{#Ref<0.0.0.4626>,'ns_1@192.168.226.129'}},
connected},
{'$gen_call',{<0.238.0>,#Ref<0.0.0.4663>},connected},
check_started,check_started,check_started,check_started,
check_started,check_started,check_started,check_started]
links: [<0.64.0>,<0.474.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 28657
stack_size: 24
reductions: 7441
neighbours:

[error_logger:error] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] Error in process <0.473.0> on node 'ns_1@192.168.226.129' with exit value: {{badmatch,{error,shutdown}},[{single_bucket_sup,'-start_link/1-fun-0-',2}]}

[error_logger:error] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_bucket_sup}
Context: child_terminated
Reason: {{badmatch,{error,shutdown}},
[{single_bucket_sup,'-start_link/1-fun-0-',2}]}
Offender: [{pid,<0.473.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_bucket_sup}
started: [{pid,<0.516.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:error] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {couldnt_connect_to_memcached,
{gen_server,call,['ns_memcached-default',topkeys,60000]}}
Offender: [{pid,<0.472.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.518.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.511.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:31] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.525.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:32] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:error] [2012-09-06 8:24:32] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[ns_server:warn] [2012-09-06 8:24:34] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.512.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912669,69256},undefined}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.512.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[ns_server:debug] [2012-09-06 8:24:34] [ns_1@192.168.226.129:<0.530.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.512.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.530.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.525.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.533.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[user:info] [2012-09-06 8:24:34] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:34] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 10 times in the past 54.56525 secs (last seen 5.445 secs ago
[ns_server:info] [2012-09-06 8:24:34] [ns_1@192.168.226.129:<0.530.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:34] [ns_1@192.168.226.129:<0.530.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.44s

[ns_server:info] [2012-09-06 8:24:34] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log supervisor_cushion:1("Service memcached exited on node 'ns_1@192.168.226.129' in 0.44s\n") because it's been seen 1 times in the past 27.004 secs (last seen 27.004 secs ago
[error_logger:error] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.531.0> terminating
** Last message in was {#Port<0.5636>,{exit_status,3}}
** When Server state == {state,#Port<0.5636>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:34] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.531.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.530.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5636>,normal}]
links: [<0.530.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:warn] [2012-09-06 8:24:36] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[stats:error] [2012-09-06 8:24:36] [ns_1@192.168.226.129:<0.329.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[stats:error] [2012-09-06 8:24:36] [ns_1@192.168.226.129:<0.538.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[ns_server:info] [2012-09-06 8:24:37] [ns_1@192.168.226.129:<0.503.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[ns_server:info] [2012-09-06 8:24:37] [ns_1@192.168.226.129:<0.225.0>:ns_orchestrator:handle_info:209] Skipping janitor in state janitor_running: {janitor_state,
["default"],
<0.503.0>}
[ns_doctor:debug] [2012-09-06 8:24:37] [ns_1@192.168.226.129:ns_doctor:ns_doctor:handle_info:93] Current node statuses:
[{'ns_1@192.168.226.129',
[{last_heard,{1346,912671,66258}},
{active_buckets,["default"]},
{ready_buckets,[]},
{replication,[{"default",1.0}]},
{memory,
[{total,14115152},
{processes,5826352},
{processes_used,5817824},
{system,8288800},
{atom,709869},
{atom_used,691679},
{binary,335632},
{code,5637249},
{ets,415012}]},
{system_stats,
[{cpu_utilization_rate,2.303455182774161},
{swap_total,4293902336},
{swap_used,1175003136}]},
{interesting_stats,[]},
{cluster_compatibility_version,1},
{version,
[{ale,"8cffe61"},
{os_mon,"2.2.5"},
{mnesia,"4.4.17"},
{kernel,"2.14.3"},
{sasl,"2.1.9.3"},
{ns_server,"1.8.1-937-rel-enterprise"},
{stdlib,"1.17.3"}]},
{system_arch,"windows"},
{wall_clock,57},
{memory_data,{2146951168,909393920,{<0.7.0>,159012}}},
{disk_data,[{"C:\\",41940988,33}]},
{system_memory_data,
[{total_memory,2146951168},
{free_memory,1158967296},
{system_total_memory,2146951168}]},
{statistics,
[{wall_clock,{54004,0}},
{context_switches,{32954,0}},
{garbage_collection,{6715,26701771,0}},
{io,{{input,7455623},{output,2869945}}},
{reductions,{13030949,602331}},
{run_queue,0},
{runtime,{1372,62}}]}]}]
[ns_server:error] [2012-09-06 8:24:37] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[error_logger:error] [2012-09-06 8:24:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.533.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:37] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.555.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:38] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:39] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.530.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912674,451256},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:24:39] [ns_1@192.168.226.129:<0.559.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:39] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.530.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:24:39] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.530.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:39] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.559.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:40] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[user:info] [2012-09-06 8:24:40] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:40] [ns_1@192.168.226.129:<0.559.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:40] [ns_1@192.168.226.129:<0.559.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.41s

[error_logger:error] [2012-09-06 8:24:40] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.560.0> terminating
** Last message in was {#Port<0.5646>,{exit_status,3}}
** When Server state == {state,#Port<0.5646>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:40] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.560.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.559.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5646>,normal}]
links: [<0.559.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[error_logger:error] [2012-09-06 8:24:40] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.555.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:40] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.562.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,'single_bucket_sup-default'}
Context: start_error
Reason: couldnt_connect_to_memcached
Offender: [{pid,undefined},
{name,{ns_memcached,"default"}},
{mfargs,{ns_memcached,start_link,["default"]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[ns_server:info] [2012-09-06 8:24:41] [ns_1@192.168.226.129:<0.503.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[error_logger:error] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_memcached:init/1
pid: <0.519.0>
registered_name: []
exception exit: couldnt_connect_to_memcached
in function gen_server:init_it/6
ancestors: ['single_bucket_sup-default',<0.516.0>]
messages: [check_started,check_started,
{'$gen_call',{<0.526.0>,
{#Ref<0.0.0.4899>,'ns_1@192.168.226.129'}},
connected},
{'$gen_call',{<0.238.0>,#Ref<0.0.0.4928>},connected},
check_started,check_started,check_started,check_started,
check_started,check_started,check_started,check_started,
{'$gen_call',{<0.536.0>,#Ref<0.0.0.5081>},topkeys},
check_started,check_started,
{'$gen_call',{<0.238.0>,#Ref<0.0.0.5548>},connected},
check_started,check_started,
{'$gen_call',{<0.556.0>,
{#Ref<0.0.0.5575>,'ns_1@192.168.226.129'}},
connected},
check_started,check_started,check_started,check_started,
check_started,check_started]
links: [<0.64.0>,<0.517.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 28657
stack_size: 24
reductions: 7444
neighbours:

[error_logger:error] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_bucket_sup}
Context: child_terminated
Reason: {{badmatch,{error,shutdown}},
[{single_bucket_sup,'-start_link/1-fun-0-',2}]}
Offender: [{pid,<0.516.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:info] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_bucket_sup}
started: [{pid,<0.563.0>},
{name,{per_bucket_sup,"default"}},
{mfargs,{single_bucket_sup,start_link,["default"]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]

[error_logger:error] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {couldnt_connect_to_memcached,
{gen_server,call,['ns_memcached-default',topkeys,60000]}}
Offender: [{pid,<0.518.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.564.0>},
{name,hot_keys_keeper},
{mfargs,{hot_keys_keeper,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:error] [2012-09-06 8:24:41] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] Error in process <0.516.0> on node 'ns_1@192.168.226.129' with exit value: {{badmatch,{error,shutdown}},[{single_bucket_sup,'-start_link/1-fun-0-',2}]}

[stats:error] [2012-09-06 8:24:41] [ns_1@192.168.226.129:<0.312.0>:stats_reader:log_bad_responses:191] Some nodes didn't respond: ['ns_1@192.168.226.129']
[ns_server:warn] [2012-09-06 8:24:42] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:error] [2012-09-06 8:24:42] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[error_logger:error] [2012-09-06 8:24:43] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.562.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:43] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.580.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:warn] [2012-09-06 8:24:44] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.559.0> terminating
** Last message in was {die,{abnormal,3}}
** When Server state == {state,memcached,5000,{1346,912679,895254},undefined}
** Reason for termination ==
** {abnormal,3}

[ns_server:debug] [2012-09-06 8:24:45] [ns_1@192.168.226.129:<0.583.0>:supervisor_cushion:init:43] starting ns_port_server with delay of 5000
[error_logger:error] [2012-09-06 8:24:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: supervisor_cushion:init/1
pid: <0.559.0>
registered_name: []
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [ns_port_sup,ns_server_sup,ns_server_cluster_sup,<0.67.0>]
messages: []
links: [<0.250.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 2397
neighbours:

[error_logger:error] [2012-09-06 8:24:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,ns_port_sup}
Context: child_terminated
Reason: {abnormal,3}
Offender: [{pid,<0.559.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,ns_port_sup}
started: [{pid,<0.583.0>},
{name,
{memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000","-p",
"11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]}},
{mfargs,
{erlang,apply,
[#Fun,
[memcached,
"c:/Program Files/Couchbase/Server/bin/memcached",
["-X",
"c:/Program Files/Couchbase/Server/lib/memcached/stdin_term_handler.so",
"-l","0.0.0.0:11210,0.0.0.0:11209:1000",
"-p","11210","-E",
"c:/Program Files/Couchbase/Server/lib/memcached/bucket_engine.so",
"-B","binary","-r","-c","10000","-e",
"admin=_admin;default_bucket_name=default;auto_create=false",
[]],
[{env,
[{"EVENT_NOSELECT","1"},
{"MEMCACHED_TOP_KEYS","100"},
{"ISASL_PWFILE",
"c:/Program Files/Couchbase/Server/var/lib/couchbase/data/isasl.pw"},
{"ISASL_DB_CHECK_TIME","1"}]},
use_stdio,stderr_to_stdout,exit_status,
port_server_send_eol,stream]]]}},
{restart_type,permanent},
{shutdown,86400000},
{child_type,worker}]

[user:info] [2012-09-06 8:24:45] [ns_1@192.168.226.129:ns_port_memcached:ns_port_server:handle_info:107] Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages:
[ns_server:info] [2012-09-06 8:24:45] [ns_1@192.168.226.129:ns_log:ns_log:handle_cast:115] suppressing duplicate log ns_port_server:0("Port server memcached on node 'ns_1@192.168.226.129' exited with status 3. Restarting. Messages: ") because it's been seen 1 times in the past 5.412999 secs (last seen 5.412999 secs ago
[ns_server:info] [2012-09-06 8:24:45] [ns_1@192.168.226.129:<0.583.0>:supervisor_cushion:handle_info:61] Cushion managed supervisor for memcached failed: {abnormal,3}
[user:warn] [2012-09-06 8:24:45] [ns_1@192.168.226.129:<0.583.0>:supervisor_cushion:die_slowly:80] Service memcached exited on node 'ns_1@192.168.226.129' in 0.40s

[error_logger:error] [2012-09-06 8:24:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_msg:76] ** Generic server <0.584.0> terminating
** Last message in was {#Port<0.5652>,{exit_status,3}}
** When Server state == {state,#Port<0.5652>,memcached,
{[empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty,
empty,empty,empty,empty,empty,empty,empty],
[empty,empty]},
undefined,[],0,true}
** Reason for termination ==
** {abnormal,3}

[error_logger:error] [2012-09-06 8:24:45] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: ns_port_server:init/1
pid: <0.584.0>
registered_name: ns_port_memcached
exception exit: {abnormal,3}
in function gen_server:terminate/6
ancestors: [<0.583.0>,ns_port_sup,ns_server_sup,ns_server_cluster_sup,
<0.67.0>]
messages: [{'EXIT',#Port<0.5652>,normal}]
links: [<0.583.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 1594
neighbours:

[ns_server:warn] [2012-09-06 8:24:46] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:46] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.580.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:46] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.592.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[ns_server:info] [2012-09-06 8:24:47] [ns_1@192.168.226.129:<0.503.0>:ns_janitor:wait_for_memcached:293] Waiting for "default" on ['ns_1@192.168.226.129']
[ns_server:info] [2012-09-06 8:24:47] [ns_1@192.168.226.129:<0.225.0>:ns_orchestrator:handle_info:209] Skipping janitor in state janitor_running: {janitor_state,
["default"],
<0.503.0>}
[ns_server:error] [2012-09-06 8:24:47] [ns_1@192.168.226.129:ns_heart:ns_heart:current_status:163] Failed to get stats for bucket: "default":
{'EXIT',{noproc,{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute}]}}}

[ns_server:warn] [2012-09-06 8:24:48] [ns_1@192.168.226.129:'ns_memcached-default':ns_memcached:connect:757] Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[error_logger:error] [2012-09-06 8:24:49] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================SUPERVISOR REPORT=========================
Supervisor: {local,menelaus_sup}
Context: child_terminated
Reason: {noproc,
{gen_server,call,
[{'stats_reader-default','ns_1@192.168.226.129'},
{latest,minute,1}]}}
Offender: [{pid,<0.592.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

[error_logger:info] [2012-09-06 8:24:49] [ns_1@192.168.226.129:error_logger:ale_error_logger_handler:log_report:72]
=========================PROGRESS REPORT=========================
supervisor: {local,menelaus_sup}
started: [{pid,<0.601.0>},
{name,menelaus_web_alerts_srv},
{mfargs,{menelaus_web_alerts_srv,start_link,[]}},
{restart_type,permanent},
{shutdown,5000},
{child_type,worker}]

Top