Home | Forums | Membase | Membase Server 1.7.x

Membase fails to connect when one node is down

4 replies [Last post]
Thu, 05/31/2012 - 13:56
mayank_connectem
Offline
Joined: 05/31/2012
Groups: None

Hi
i have two membase servers (192.168.0.150 and 192.168.0.152) and have added some implementation in my code so that if one of them is down or not working then the other one should take its place and also i have added that failover part and rebalanced them so that we have the replicas in both membase.
But i am not getting the result as expected as if one of the membase which comes first in the list is down then the system tries to reconnect with it again and again rather then using the other one in the serverlist.
i have mentioned the implemented code and the exception that i am getting please let me know if i am going wrong.

thanx.

COde:

URI l_server1 = new URI(uris[0]);
URI l_server2 = new URI(uris[1]);
ArrayList serverList = new ArrayList();
serverList.add(l_server1);
serverList.add(l_server2);
m_client = new MemcachedClient(serverList, "default", "");

Exception

2012-05-30 22:04:30.072 WARN net.spy.memcached.vbucket.ConfigurationProviderHTTP: Connection problems with URI http://192.168.0.150:8091/pools ...skipping
java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
at sun.net.www.http.HttpClient.(HttpClient.java:233)
at sun.net.www.http.HttpClient.New(HttpClient.java:306)
at sun.net.www.http.HttpClient.New(HttpClient.java:323)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:970)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:911)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:836)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1172)
at net.spy.memcached.vbucket.ConfigurationProviderHTTP.readToString(ConfigurationProviderHTTP.java:255)
at net.spy.memcached.vbucket.ConfigurationProviderHTTP.readPools(ConfigurationProviderHTTP.java:106)
at net.spy.memcached.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:87)
at net.spy.memcached.MemcachedClient.(MemcachedClient.java:255)
at net.spy.memcached.MemcachedClient.(MemcachedClient.java:216)
at net.spy.memcached.MemcachedClient.(MemcachedClient.java:340)

net.spy.memcached.vbucket.ConfigurationException: Configuration for bucket default was not found.
at net.spy.memcached.vbucket.ConfigurationProviderHTTP.readPools(ConfigurationProviderHTTP.java:151)
at net.spy.memcached.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:87)
at net.spy.memcached.MemcachedClient.(MemcachedClient.java:255)
at net.spy.memcached.MemcachedClient.(MemcachedClient.java:216)
at net.spy.memcached.MemcachedClient.(MemcachedClient.java:340)

Top
Mon, 06/04/2012 - 18:16
ingenthr
Offline
Joined: 03/16/2010
Groups:

The log there indicates you do not have a default bucket. Can you confirm you have a default bucket defined on the server/cluster?

Top
Tue, 06/05/2012 - 05:14
mayank_connectem
Offline
Joined: 05/31/2012
Groups: None

Hi thanks for your response,
I have some how resolved with this problem i upgraded my membase version to 1.8 and now i am using couchbased client but still i am having some exception while the client tries to make connection to couchbase.

here is the exception which i receive
2012-06-05 17:24:58.048 DEBUG : Connecting to Membase [http://192.168.0.150:8091/pools, http://192.168.0.122:8091/pools, http://192.168.0.151:8091/pools]
2012-06-05 17:25:11.188 WARN com.couchbase.client.vbucket.ConfigurationProviderHTTP: Connection problems with URI http://192.168.0.150:8091/pools ...skipping
java.net.ConnectException: Connection timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
at sun.net.www.http.HttpClient.(HttpClient.java:233)
at sun.net.www.http.HttpClient.New(HttpClient.java:306)
at sun.net.www.http.HttpClient.New(HttpClient.java:323)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:970)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:911)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:836)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1172)
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readToString(ConfigurationProviderHTTP.java:322)
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readPools(ConfigurationProviderHTTP.java:144)
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:126)
at com.couchbase.client.CouchbaseConnectionFactory.getVBucketConfig(CouchbaseConnectionFactory.java:154)
at com.couchbase.client.CouchbaseClient.(CouchbaseClient.java:156)
at com.couchbase.client.CouchbaseClient.(CouchbaseClient.java:125)
at com.couchbase.client.CouchbaseClient.(CouchbaseClient.java:77)

after 30 seconds client tries to connect to other URL's
2012-06-05 17:25:20.353 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.122:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-06-05 17:25:20.356 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.151:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-06-05 17:25:29.492 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@ef137d
2012-06-05 17:25:29.495 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@141b571

Top
Fri, 06/08/2012 - 16:09
mayank_connectem
Offline
Joined: 05/31/2012
Groups: None

BTW, to answer the question on default bucket, yes the default is defined on the server/cluster.

Top
Wed, 06/13/2012 - 09:38
ingenthr
Offline
Joined: 03/16/2010
Groups:

Connection timing out there seems to indicate some kind of connectivity problem between the client and the servers. This could be, for example, a firewall.

I'd recommend using just a plain old HTTP client to verify you can get to /pools on each of the systems you're putting in your List. I bet that'll turn up either an incorrect IP address or an incorrect network configuration.

Top