============================================================================== couchbase logs (error) cbbrowse_logs error ============================================================================== [ns_server:error,2012-11-12T12:26:35.531,ns_1@127.0.0.1:ns_heart<0.373.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [ns_server:error,2012-11-12T12:26:35.542,ns_1@127.0.0.1:ns_heart<0.373.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [error_logger:error,2012-11-12T12:26:41.734,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_bucket_sup} Context: shutdown_error Reason: normal Offender: [{pid,<0.452.0>}, {name,buckets_observing_subscription}, {mfargs,{ns_bucket_sup,subscribe_on_config_events,[]}}, {restart_type,permanent}, {shutdown,1000}, {child_type,worker}] [error_logger:error,2012-11-12T12:26:41.945,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_server_cluster_sup} Context: shutdown_error Reason: killed Offender: [{pid,<0.344.0>}, {name,vbucket_filter_changes_registry}, {mfargs, {ns_process_registry,start_link, [vbucket_filter_changes_registry]}}, {restart_type,permanent}, {shutdown,100}, {child_type,worker}] [error_logger:error,2012-11-12T12:26:46.949,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,mb_mnesia_sup} Context: shutdown_error Reason: killed Offender: [{pid,<0.282.0>}, {name,mb_mnesia}, {mfargs,{mb_mnesia,start_link,[]}}, {restart_type,permanent}, {shutdown,5000}, {child_type,worker}] [error_logger:error,2012-11-12T12:26:47.016,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_file:spawn_writer/2 pid: <0.230.0> registered_name: [] exception exit: {noproc, {gen_server,call, [couch_file_write_guard, {remove,<0.230.0>}, infinity]}} in function gen_server:call/3 in call from couch_file:writer_loop/4 ancestors: [<0.227.0>,couch_server,couch_primary_services, couch_server_sup,cb_couch_sup,ns_server_cluster_sup, <0.59.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 233 stack_size: 24 reductions: 2140 neighbours: [error_logger:error,2012-11-12T12:26:47.016,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Generic server <0.231.0> terminating ** Last message in was {'EXIT',<0.217.0>,killed} ** When Server state == {db,<0.231.0>,<0.232.0>,nil,<<"1352751994152542">>, <0.227.0>,<0.233.0>, {db_header,11,1, <<0,0,0,0,13,103,0,0,0,0,0,51,0,0,0,0,1,0,0,0, 0,0,0,0,0,0,13,69>>, <<0,0,0,0,13,154,0,0,0,0,0,49,0,0,0,0,1>>, nil,0,nil,nil}, 1, {btree,<0.227.0>, {3431, <<0,0,0,0,1,0,0,0,0,0,0,0,0,0,13,69>>, 51}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.227.0>, {3482,<<0,0,0,0,1>>,49}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.227.0>,nil, #Fun, #Fun, #Fun,nil,1279,2558, true}, 1,<<"_users">>, "/opt/couchbase/var/lib/couchbase/data/_users.couch.1", [],nil, {user_ctx,null,[],undefined}, nil, [before_header,after_header,on_file_open], [create, {user_ctx, {user_ctx,null,[<<"_admin">>],undefined}}, sys_db]} ** Reason for termination == ** killed [error_logger:error,2012-11-12T12:26:47.020,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_db:init/1 pid: <0.231.0> registered_name: [] exception exit: killed in function gen_server:terminate/6 ancestors: [couch_server,couch_primary_services,couch_server_sup, cb_couch_sup,ns_server_cluster_sup,<0.59.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 987 stack_size: 24 reductions: 284 neighbours: [error_logger:error,2012-11-12T12:26:47.021,ns_1@127.0.0.1:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_file:spawn_writer/2 pid: <0.445.0> registered_name: [] exception exit: {noproc, {gen_server,call, [couch_file_write_guard, {remove,<0.445.0>}, infinity]}} in function gen_server:call/3 in call from couch_file:writer_loop/4 ancestors: [<0.442.0>,couch_server,couch_primary_services, couch_server_sup,cb_couch_sup,ns_server_cluster_sup, <0.59.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 377 stack_size: 24 reductions: 2154 neighbours: [ns_server:error,2012-11-12T12:26:53.197,ns_1@192.168.0.21:ns_heart<0.373.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [ns_server:error,2012-11-12T12:26:53.209,ns_1@192.168.0.21:ns_heart<0.373.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [error_logger:error,2012-11-12T12:26:59.456,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_bucket_sup} Context: shutdown_error Reason: normal Offender: [{pid,<0.453.0>}, {name,buckets_observing_subscription}, {mfargs,{ns_bucket_sup,subscribe_on_config_events,[]}}, {restart_type,permanent}, {shutdown,1000}, {child_type,worker}] [error_logger:error,2012-11-12T12:27:00.484,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_server_cluster_sup} Context: shutdown_error Reason: killed Offender: [{pid,<0.344.0>}, {name,vbucket_filter_changes_registry}, {mfargs, {ns_process_registry,start_link, [vbucket_filter_changes_registry]}}, {restart_type,permanent}, {shutdown,100}, {child_type,worker}] [error_logger:error,2012-11-12T12:27:00.803,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Generic server <0.237.0> terminating ** Last message in was {'EXIT',<0.223.0>,killed} ** When Server state == {db,<0.237.0>,<0.238.0>,nil,<<"1352752011808812">>, <0.233.0>,<0.239.0>, {db_header,11,1, <<0,0,0,0,13,103,0,0,0,0,0,51,0,0,0,0,1,0,0,0, 0,0,0,0,0,0,13,69>>, <<0,0,0,0,13,154,0,0,0,0,0,49,0,0,0,0,1>>, nil,0,nil,nil}, 1, {btree,<0.233.0>, {3431, <<0,0,0,0,1,0,0,0,0,0,0,0,0,0,13,69>>, 51}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.233.0>, {3482,<<0,0,0,0,1>>,49}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.233.0>,nil, #Fun, #Fun, #Fun,nil,1279,2558, true}, 1,<<"_users">>, "/opt/couchbase/var/lib/couchbase/data/_users.couch.1", [],nil, {user_ctx,null,[],undefined}, nil, [before_header,after_header,on_file_open], [create, {user_ctx, {user_ctx,null,[<<"_admin">>],undefined}}, sys_db]} ** Reason for termination == ** killed [error_logger:error,2012-11-12T12:27:00.811,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_db:init/1 pid: <0.237.0> registered_name: [] exception exit: killed in function gen_server:terminate/6 ancestors: [couch_server,couch_primary_services,couch_server_sup, cb_couch_sup,ns_server_cluster_sup,<0.65.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 987 stack_size: 24 reductions: 288 neighbours: [error_logger:error,2012-11-12T12:27:00.814,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_file:spawn_writer/2 pid: <0.446.0> registered_name: [] exception exit: {noproc, {gen_server,call, [couch_file_write_guard, {remove,<0.446.0>}, infinity]}} in function gen_server:call/3 in call from couch_file:writer_loop/4 ancestors: [<0.443.0>,couch_server,couch_primary_services, couch_server_sup,cb_couch_sup,ns_server_cluster_sup, <0.65.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 377 stack_size: 24 reductions: 2154 neighbours: [error_logger:error,2012-11-12T12:27:00.814,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Generic server <0.447.0> terminating ** Last message in was {'EXIT',<0.223.0>,killed} ** When Server state == {db,<0.447.0>,<0.448.0>,nil,<<"1352752013272568">>, <0.443.0>,<0.449.0>, {db_header,11,1, <<0,0,0,0,11,84,0,0,0,0,0,62,0,0,0,0,1,0,0,0,0, 0,0,0,0,0,11,50>>, <<0,0,0,0,11,146,0,0,0,0,0,60,0,0,0,0,1>>, nil,0,nil,nil}, 1, {btree,<0.443.0>, {2900, <<0,0,0,0,1,0,0,0,0,0,0,0,0,0,11,50>>, 62}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.443.0>, {2962,<<0,0,0,0,1>>,60}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.443.0>,nil, #Fun, #Fun, #Fun,nil,1279,2558, true}, 1,<<"_replicator">>, "/opt/couchbase/var/lib/couchbase/data/_replicator.couch.1", [],nil, {user_ctx,null,[],undefined}, nil, [before_header,after_header,on_file_open], [create,sys_db, {user_ctx, {user_ctx,null, [<<"_admin">>,<<"_replicator">>], undefined}}]} ** Reason for termination == ** killed [error_logger:error,2012-11-12T12:27:00.815,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_db:init/1 pid: <0.447.0> registered_name: [] exception exit: killed in function gen_server:terminate/6 ancestors: [couch_server,couch_primary_services,couch_server_sup, cb_couch_sup,ns_server_cluster_sup,<0.65.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 987 stack_size: 24 reductions: 341 neighbours: [ns_server:error,2012-11-12T12:27:03.286,ns_1@192.168.0.21:ns_heart<0.731.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [ns_server:error,2012-11-12T12:27:03.290,ns_1@192.168.0.21:ns_heart<0.731.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [error_logger:error,2012-11-12T12:27:08.133,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_bucket_sup} Context: shutdown_error Reason: normal Offender: [{pid,<0.804.0>}, {name,buckets_observing_subscription}, {mfargs,{ns_bucket_sup,subscribe_on_config_events,[]}}, {restart_type,permanent}, {shutdown,1000}, {child_type,worker}] [ns_server:error,2012-11-12T12:27:08.285,ns_1@192.168.0.21:ns_heart<0.731.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [error_logger:error,2012-11-12T12:27:08.587,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_server_cluster_sup} Context: shutdown_error Reason: killed Offender: [{pid,<0.704.0>}, {name,vbucket_filter_changes_registry}, {mfargs, {ns_process_registry,start_link, [vbucket_filter_changes_registry]}}, {restart_type,permanent}, {shutdown,100}, {child_type,worker}] [error_logger:error,2012-11-12T12:27:13.594,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,mb_mnesia_sup} Context: shutdown_error Reason: killed Offender: [{pid,<0.608.0>}, {name,mb_mnesia}, {mfargs,{mb_mnesia,start_link,[]}}, {restart_type,permanent}, {shutdown,5000}, {child_type,worker}] [error_logger:error,2012-11-12T12:27:13.631,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_file:spawn_writer/2 pid: <0.795.0> registered_name: [] exception exit: {noproc, {gen_server,call, [couch_file_write_guard, {remove,<0.795.0>}, infinity]}} in function gen_server:call/3 in call from couch_file:writer_loop/4 ancestors: [<0.792.0>,couch_server,couch_primary_services, couch_server_sup,cb_couch_sup,ns_server_cluster_sup, <0.65.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 610 stack_size: 24 reductions: 1933 neighbours: [error_logger:error,2012-11-12T12:27:13.631,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Generic server <0.796.0> terminating ** Last message in was {'EXIT',<0.555.0>,killed} ** When Server state == {db,<0.796.0>,<0.797.0>,nil,<<"1352752023347016">>, <0.792.0>,<0.800.0>, {db_header,11,1, <<0,0,0,0,11,84,0,0,0,0,0,62,0,0,0,0,1,0,0,0,0, 0,0,0,0,0,11,50>>, <<0,0,0,0,11,146,0,0,0,0,0,60,0,0,0,0,1>>, nil,0,nil,nil}, 1, {btree,<0.792.0>, {2900, <<0,0,0,0,1,0,0,0,0,0,0,0,0,0,11,50>>, 62}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.792.0>, {2962,<<0,0,0,0,1>>,60}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.792.0>,nil, #Fun, #Fun, #Fun,nil,1279,2558, true}, 1,<<"_replicator">>,"/data/_replicator.couch.1", [],nil, {user_ctx,null,[],undefined}, nil, [before_header,after_header,on_file_open], [create,sys_db, {user_ctx, {user_ctx,null, [<<"_admin">>,<<"_replicator">>], undefined}}]} ** Reason for termination == ** killed [error_logger:error,2012-11-12T12:27:13.633,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_db:init/1 pid: <0.796.0> registered_name: [] exception exit: killed in function gen_server:terminate/6 ancestors: [couch_server,couch_primary_services,couch_server_sup, cb_couch_sup,ns_server_cluster_sup,<0.65.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 987 stack_size: 24 reductions: 279 neighbours: [error_logger:error,2012-11-12T12:27:13.633,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Generic server <0.569.0> terminating ** Last message in was {'EXIT',<0.555.0>,killed} ** When Server state == {db,<0.569.0>,<0.570.0>,nil,<<"1352752020954991">>, <0.565.0>,<0.571.0>, {db_header,11,1, <<0,0,0,0,13,103,0,0,0,0,0,51,0,0,0,0,1,0,0,0, 0,0,0,0,0,0,13,69>>, <<0,0,0,0,13,154,0,0,0,0,0,49,0,0,0,0,1>>, nil,0,nil,nil}, 1, {btree,<0.565.0>, {3431, <<0,0,0,0,1,0,0,0,0,0,0,0,0,0,13,69>>, 51}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.565.0>, {3482,<<0,0,0,0,1>>,49}, #Fun, #Fun, #Fun, #Fun,1279,2558, true}, {btree,<0.565.0>,nil, #Fun, #Fun, #Fun,nil,1279,2558, true}, 1,<<"_users">>,"/data/_users.couch.1",[],nil, {user_ctx,null,[],undefined}, nil, [before_header,after_header,on_file_open], [create, {user_ctx, {user_ctx,null,[<<"_admin">>],undefined}}, sys_db]} ** Reason for termination == ** killed [error_logger:error,2012-11-12T12:27:13.634,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================CRASH REPORT========================= crasher: initial call: couch_db:init/1 pid: <0.569.0> registered_name: [] exception exit: killed in function gen_server:terminate/6 ancestors: [couch_server,couch_primary_services,couch_server_sup, cb_couch_sup,ns_server_cluster_sup,<0.65.0>] messages: [] links: [] dictionary: [] trap_exit: true status: running heap_size: 987 stack_size: 24 reductions: 268 neighbours: [ns_server:error,2012-11-12T12:27:21.692,ns_1@192.168.0.21:ns_heart<0.408.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [ns_server:error,2012-11-12T12:27:21.703,ns_1@192.168.0.21:ns_heart<0.408.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [error_logger:error,2012-11-12T12:51:27.866,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Connection attempt from disallowed node 'ns_1@192.168.0.20' ** [error_logger:error,2012-11-12T12:51:27.880,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_report:72] =========================SUPERVISOR REPORT========================= Supervisor: {local,ns_bucket_sup} Context: shutdown_error Reason: normal Offender: [{pid,<0.472.0>}, {name,buckets_observing_subscription}, {mfargs,{ns_bucket_sup,subscribe_on_config_events,[]}}, {restart_type,permanent}, {shutdown,1000}, {child_type,worker}] [error_logger:error,2012-11-12T12:51:27.891,ns_1@192.168.0.21:error_logger<0.5.0>:ale_error_logger_handler:log_msg:76]** Connection attempt from disallowed node 'ns_1@192.168.0.22' ** [ns_server:error,2012-11-12T12:51:28.186,ns_1@192.168.0.21:ns_heart<0.4481.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_info,2}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [ns_server:error,2012-11-12T12:51:28.190,ns_1@192.168.0.21:ns_heart<0.4481.0>:ns_heart:grab_samples_loading_tasks:298]Failed to grab samples loader tasks: {exit, {noproc, {gen_server,call, [samples_loader_tasks,get_tasks, 2000]}}, [{gen_server,call,3}, {ns_heart,grab_samples_loading_tasks,0}, {ns_heart,current_status,0}, {ns_heart,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} [ns_server:error,2012-11-12T13:45:03.144,ns_1@192.168.0.21:<0.20562.3>:ns_orchestrator:rebalance_progress:176]Couldn't talk to orchestrator: {exit, {noproc, {gen_fsm,sync_send_event, [{global,ns_orchestrator}, rebalance_progress,2000]}}}