Couchbase Server

(node crashes under set/get load while running large scale testing ( SQL logic error or missing database)

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Won't Fix
  • Affects Version/s: 1.8.0
  • Fix Version/s: None
  • Component/s: bucket-engine, ns_server
  • Security Level: Public
  • Labels:
    None

Description

diags attached.

according to diags node was running fine but after 00:02 we had issues writing to disk

memcached<0.10886.0>: sqlite error: disk I/O error


ns_log: logging ns_port_server:0:Port server memcached on node 'ns_1@10.82.238.5' exited with status 1. Restarting. Messages: sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database
sqlite error: SQL logic error or missing database

  1. Text File
    log.txt
    30/Dec/11 10:31 AM
    3.64 MB
    Farshid Ghods

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Farshid Ghods added a comment -
i dont have diags from other nodes . i rebalanced and failed over a node to recover the cluster.

will grab more diags later and attach it here but need to know why only this need seemed to crash ( we had 6 nodes in the cluster )
Show
Farshid Ghods added a comment - i dont have diags from other nodes . i rebalanced and failed over a node to recover the cluster. will grab more diags later and attach it here but need to know why only this need seemed to crash ( we had 6 nodes in the cluster )
Hide
Permalink
Aleksey Kondratenko added a comment -
Node seemingly ran out of disk space. I don't think we're going to handle it in any way.
Show
Aleksey Kondratenko added a comment - Node seemingly ran out of disk space. I don't think we're going to handle it in any way.
Hide
Permalink
Farshid Ghods added a comment -
okay but what i dont understand is did ns_server actually delete these data files ?

when i logged in to this machine in the morning the bucket was gone
Show
Farshid Ghods added a comment - okay but what i dont understand is did ns_server actually delete these data files ? when i logged in to this machine in the morning the bucket was gone
Hide
Permalink
Farshid Ghods added a comment -
more diags :


Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.100.254.36-diag.txt.zip
Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.118.159.174-diag.txt.zip
Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.36.191.171-diag.txt.zip
Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.77.35.248-diag.txt.zip
Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.82.238.5-diag.txt.zip
Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.82.239.221-diag.txt.zip
Show
Farshid Ghods added a comment - more diags : Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.100.254.36-diag.txt.zip Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.118.159.174-diag.txt.zip Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.36.191.171-diag.txt.zip Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.77.35.248-diag.txt.zip Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.82.238.5-diag.txt.zip Public URL of the object is: http://bugdb.s3.amazonaws.com/jira/MB-4602/10.82.239.221-diag.txt.zip
Hide
Permalink
Aleksey Kondratenko added a comment -
Somebody restarted service around 1 hour at night. And it was failed over during that time. So data files were deleted.
Show
Aleksey Kondratenko added a comment - Somebody restarted service around 1 hour at night. And it was failed over during that time. So data files were deleted.

People

  • Assignee:
    Bin Cui
    Reporter:
    Farshid Ghods
Vote (0)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: