Home | Forums | Couchbase | Couchbase Server 2.0

ALL Nodes (3) Crash CB ENT 2.0

7 replies [Last post]
Tue, 01/22/2013 - 07:46
michaelt
Offline
Joined: 11/06/2012
Groups: None

I've been running 2.0 beta for months w/o issue. I upgraded to enterprise 2.0 GA last week and this morning I log in and all of my nodes are crashed with the following error in /var/log/messages

Jan 22 14:15:04 couch01 kernel: [411073.960652] do_general_protection: 24 callbacks suppressed
Jan 22 14:15:04 couch01 kernel: [411073.960658] beam.smp[1918] general protection ip:7f4bfcabf8c3 sp:7f4c0c2d15e8 error:0 in libstdc++.so.6.0.16[7f4bfca62000+e2000]

Jan 22 14:17:52 couch02 kernel: [1496388.967379] do_general_protection: 12 callbacks suppressed
Jan 22 14:17:52 couch02 kernel: [1496388.967384] beam.smp[10934] general protection ip:7f5f61d548b0 sp:7f5f795665e8 error:0 in libstdc++.so.6.0.16[7f5f61cf7000+e2000]

Jan 22 14:12:17 couch03 kernel: [1496631.103918] beam.smp[24037] general protection ip:7f8a3864b8b0 sp:7f8a431fe5e8 error:0 in libstdc++.so.6.0.16[7f8a385ee000+e2000]

I find it very odd that all 3 servers would crash within 5 minutes of each other. Something is up with the timing of this .

Top
Tue, 01/22/2013 - 08:13
rbin
Offline
Joined: 01/10/2013
Groups: None

Hi Michael,

For us to be able to diagnose this issue properly, could you please provide me with some more information? Namely:

- Which OS are you running your cluster on?
- What are the general hardware specs running your cluster?
- Briefly what type of usage is your application receiving?

Finally, I appreciate your logs above, but if you could supply your log file to us, it would help greatly.

Info on how to do that can be found at the bottom of this page: http://www.couchbase.com/wiki/display/couchbase/Working+with+the+Couchba...

Kind regards,

Robin J.

Top
Tue, 01/22/2013 - 08:18
michaelt
Offline
Joined: 11/06/2012
Groups: None

OS: Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-31-generic x86_64)

HW: 8 Core Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz, 16 GB Ram (VMware)

Usage: LOW (this is presently in development, so there is almost no load)

Top
Tue, 01/22/2013 - 15:56
tgrall
Offline
Joined: 09/05/2012
Groups: None

Hello Michael,

Do you mind sending us your log? as documented here
http://www.couchbase.com/wiki/display/couchbase/Working+with+the+Couchba...

This will allow us to analyze the issue and see what is happening.

Regards

__________________

Tug
@tgrall

Top
Tue, 01/22/2013 - 17:11
alkondratenko
alkondratenko's picture
Offline
Joined: 12/01/2010
Groups: None

If possible we'd like either core dumps or backtraces.

This appears to be some crash in erlang. We'll need more information to say more.

BTW check your ram. We've found erlang to often be "first to crash" in a box with broken ram.

Top
Sun, 02/17/2013 - 01:20
tgrall
Offline
Joined: 09/05/2012
Groups: None

Hello,

Do you stil have a problem with your installation?
Let us know if you have uploaded log files.

Thanks

__________________

Tug
@tgrall

Top
Mon, 02/18/2013 - 18:15
michaelt
Offline
Joined: 11/06/2012
Groups: None

-Which OS are you running your cluster on?
A: Ubuntu 12.10

- What are the general hardware specs running your cluster?
A: 8core 2.0ghz, 64GB RAM, 6 Nodes

- Briefly what type of usage is your application receiving?
A: Right now this is just a dev environment to test how powerful couchbase might be for my application. Usage is very Low

Top
Mon, 02/18/2013 - 18:16
michaelt
Offline
Joined: 11/06/2012
Groups: None

I think I need to figure out how to get the logs you need. I'll try to get that done tomorrow.

Top