Home | Forums | Membase | Membase Server 1.6.x

Membase crashes every few seconds

4 replies [Last post]
Tue, 02/22/2011 - 09:33
roller123
Offline
Joined: 02/22/2011
Groups: None

I've installed a fresh 1.6.5 two-node cluster for testing.

Everything was fine in the beginning. Then I wanted to test what happenes when I take down a node. The first few tries worked well (using the membase.server init.d script to start/stop).

However, after 2-3 restarts, one node started to issue "Service memcached exited on node 'ns_1@ipaddress' in 1.35s - supervisor_cushion001" every 6-7 seconds, and refused connections. Restart didn't help. I tried to restart the other node.... and from then on, even that other node showed the same symptoms.

In total, I stored only a few test items. One of the items I stored with an expire time, and occasionally I also got this error:
"Port server memcached on node 'ns_1@ipaddress' exited with status 134. Restarting. Messages: Item was expired at load: test
memcached: stored-value.hh:974: add_type_t HashTable::add(const Item&, bool, bool): Assertion `v->isDirty() == isDirty' failed."

So now I have a totally corrupted cluster, and I'm very glad it's only a test cluster. However, it seems to be a VERY bad idea to use a database which gets corrupted so easily in production. Is this some known bug?
 

Top
Tue, 02/22/2011 - 09:48
perry
Offline
Joined: 10/11/2010
Groups:

 Hey there, I'm going to file a bug and have this looked at...seems like the software ran into something it wasn't expecting.

Your database actually isn't corrupted, there's just some issue with reading it into the software.  If you need to, you should be able to restore the data using the mbrestore script: http://wiki.membase.org/display/membase/Membase+Server+version+1.6.4+and+up

 

Perry

__________________

Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!

Top
Tue, 02/22/2011 - 09:48
perry
Offline
Joined: 10/11/2010
Groups:

 Can you send the output of /opt/membase/bin/ns_server/collect_info <output_file> to 'perry -at- membase -dot- com'?

 

Thanks

 

Perry

__________________

Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!

Top
Fri, 03/25/2011 - 16:52
perry
Offline
Joined: 10/11/2010
Groups:

Terry, we just released Membase Server Enterprise Edition version 1.6.5.3 which has resolved these and many other stability issues.

Would you please download it and verify that your issues are fixed?

Thanks!

Perry

__________________

Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!

Top
Fri, 03/25/2011 - 16:53
perry
Offline
Joined: 10/11/2010
Groups:

We just released Membase Server Enterprise Edition version 1.6.5.3 which has resolved these and many other stability issues.

Would you please download it and verify that your issues are fixed?

Thanks!

Perry

__________________

Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!

Top