Cannot write on Apache AWS Couchbase
Hi
I have following problem:
I try to write on a Amazon AWS VM with the Java code from
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/hello-world.html
using the Amazon Domain Name.
import java.net.URI; import java.util.LinkedList; import java.util.List; import java.util.concurrent.TimeUnit; import com.couchbase.client.CouchbaseClient; import net.spy.memcached.internal.GetFuture; import net.spy.memcached.internal.OperationFuture; public class Main { public static final int EXP_TIME = 10; public static final String KEY = "spoon"; public static final String VALUE = "Hello World!"; public static void main(String args[]) { // Set the URIs and get a client List<URI> uris = new LinkedList<URI>(); Boolean do_delete = false; // // Connect to localhost or to the appropriate URI uris.add(URI.create("http://ec2-23-20-202-21.compute-1.amazonaws.com:8091/pools")); CouchbaseClient client = null; try { client = new CouchbaseClient(uris, "default", ""); } catch (Exception e) { System.err.println("Error connecting to Couchbase: " + e.getMessage()); System.exit(0); } // Do a synchrononous get Object getObject = client.get(KEY); // Do an asynchronous set OperationFuture<Boolean> setOp = client.set(KEY, EXP_TIME, VALUE); // Do an asynchronous get GetFuture getOp = client.asyncGet(KEY); // Do an asynchronous delete OperationFuture<Boolean> delOp = null; if (do_delete) { delOp = client.delete(KEY); } // Shutdown the client client.shutdown(3, TimeUnit.SECONDS); // Now we want to see what happened with our data // Check to see if our set succeeded try { if (setOp.get().booleanValue()) { System.out.println("Set Succeeded"); } else { System.err.println("Set failed: " + setOp.getStatus().getMessage()); } } catch (Exception e) { System.err.println("Exception while doing set: " + e.getMessage()); } // Print the value from synchronous get if (getObject != null) { System.out.println("Synchronous Get Suceeded: " + (String) getObject); } else { System.err.println("Synchronous Get failed"); } // Check to see if ayncGet succeeded try { if ((getObject = getOp.get()) != null) { System.out.println("Asynchronous Get Succeeded: " + getObject); } else { System.err.println("Asynchronous Get failed: " + getOp.getStatus().getMessage()); } } catch (Exception e) { System.err.println("Exception while doing Aynchronous Get: " + e.getMessage()); } // Check to see if our delete succeeded if (do_delete) { try { if (delOp.get().booleanValue()) { System.out.println("Delete Succeeded"); } else { System.err.println("Delete failed: " + delOp.getStatus().getMessage()); } } catch (Exception e) { System.err.println("Exception while doing delete: " + e.getMessage()); } } } }
I'm then getting the following errors:
2012-04-16 11:19:41.972 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.209.194.24:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue 2012-04-16 11:19:42.961 WARN com.couchbase.client.CouchbaseConnection: Node exepcted to receive data is inactive. This could be due to afailure within the cluster. Will check for updated configuration. Key without a configured node is: spoon. 16.04.2012 11:19:42 com.couchbase.client.CouchbaseConnectionFactory checkConfigUpdate WARNUNG: No reconnect required, though check requested. Current config check is 1 out of a threshold of 10. Exception in thread "main" net.spy.memcached.OperationTimeoutException: Timeout waiting for value at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1003) at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1018) at Main.main(Main.java:33) Caused by: net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: 10.209.194.24/10.209.194.24:11210 at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:93) at net.spy.memcached.internal.GetFuture.get(GetFuture.java:62) at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:997) ... 2 more 2012-04-16 11:20:57.471 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@a39ab89 2012-04-16 11:20:57.472 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to failure to connect to {QA sa=10.209.194.24/10.209.194.24:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0} java.net.ConnectException: Operation timed out at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:369) at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:242) at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:229) 2012-04-16 11:20:57.472 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=10.209.194.24/10.209.194.24:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0}, attempt 1. 2012-04-16 11:21:01.475 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=10.209.194.24/10.209.194.24:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0}
telnet ec2-23-20-202-21.compute-1.amazonaws.com 11211
-> stats
works as well as sshing ec2-23-20-202-21.compute-1.amazonaws.com and performing
telnet localhost 11211
-> stats
Ports (TCP, UDP, ICMP) should be open.
What I'm doing wrong? Is it an authentication problem? I thought the default bucket doesn't need this
Best regards
Matthias
Hi,
I'm trying to remote connect. I'm using the AWS depicted public IP (see screenshot).
ping gives me:
PING ec2-23-20-114-71.compute-1.amazonaws.com (23.20.114.71): 56 data bytes 64 bytes from 23.20.114.71: icmp_seq=0 ttl=47 time=540.222 ms 64 bytes from 23.20.114.71: icmp_seq=1 ttl=47 time=332.899 ms
Connecting by browser works (http://[DNS name supplied by AWS]:8091); using the public IP which ping is delivering as well. Using the public DNS name and IP in the code is yielding to the already addressed errors.
Hi, sorry to bring up an old post but I am having the same issue, were you able to resolve it?
No sorry
Re-looking at the thread, it appears that there is something blocking the connection. The client is saying the initial connection wasn't made. This could be AWS configuration or OS level firewall.
Can you verify with the cbcstats tool (see the docs at couchbase.com/docs) for usage info that you can connect to port 11210? Or use telnet to verify.
Thanks, I am not sure how to use the cbcstats tool, but using telnet I could add a document to the default bucket thought port 11211 (I had to create the default bucket first since the AMI installation dosent create one) and it did connect to the 11210 port - but I dont know which commands I could run there.
The lack of a bucket may well have been affecting the Java client too. Now that you've created a bucket, does the Java client work as expected?
On your telnet session, you wouldn't be able to issue commands to port 11210, but just being able to connect is a good sign. Still, I'd like you to try cbstats if you can.
See: http://www.couchbase.com/docs/couchbase-manual-1.8/couchbase-admin-cmdli...
I already setup a different bucket when I tried only not the default one, and tried to connect to it from the java client, only using telnet I needed to create the default bucket for it to work, since thats the bucket it connects to. so adding the default bucket didn't make any difference.
cbstats connects and displays all information without any problem.
in the Java app, I am getting this in the console :
2012-07-20 23:05:20.705 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to failure to connect to {QA sa=10.202.17.203/10.202.17.203:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 5 Opaque: 1 Key: UserId Amount: 1 Default: 1 Exp: 0, toWrite=0, interested=0}
java.net.ConnectException: Operation timed out
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:369)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:242)
at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:229)
2012-07-20 23:05:20.706 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=10.202.17.203/10.202.17.203:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 5 Opaque: 1 Key: UserId Amount: 1 Default: 1 Exp: 0, toWrite=0, interested=0}, attempt 298.
2012-07-20 23:05:50.708 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=10.202.17.203/10.202.17.203:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 5 Opaque: 1 Key: UserId Amount: 1 Default: 1 Exp: 0, toWrite=0, interested=0}
looping endlessly.
Maybe there is something wrong with the AMI ? I tried it on various types of instances with the same results
It really looks like a connectivity issue. Can you possibly start a packet trace on ports 8091 and port 11210, then send us those?
Did running cbstats from the client system work? That'd be an easier way to verify connectivity.
running bstats from the client system worked without a problem. I did :
$ cbstats [host:11210] all
accepting_conns: 1
auth_cmds: 1
auth_errors: 0
bucket_active_conns: 1
bucket_conns: 3
bytes_read: 7408256
bytes_written: 1331541017
cas_badval: 0
cas_hits: 0
cas_misses: 0
cmd_flush: 0
cmd_get: 1
cmd_set: 1
...
I'll do a packet trace and post the data
Ok dump file is here :
Dump File
Thanks for the help
I did some more tests :
Installed cb 1.8 enterprise edition on aws - same result
Installed cb 2.0 dp4 on aws and replaced the java client - same
Installed cb 2.0 dp4 locally - works without a problem
So I still can't figure out what's wrong with my aws setup
Will have a look at the packet trace and let you know what comes out of it.
I am having a similar problem with AWS. Are there no updates on this problem?
2012-10-03 17:16:25.774 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.48.94.158:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-10-03 17:16:25.776 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=ec2-176-34-204-15.eu-west-1.compute.amazonaws.com/176.34.204.15:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-10-03 17:16:25.780 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-10-03 17:16:25.811 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@aa37a6
2012-10-03 17:16:26.121 WARN com.couchbase.client.CouchbaseConnection: Node exepcted to receive data is inactive. This could be due to a failure within the cluster. Will check for updated configuration. Key without a configured node is: spoon.
Exception in thread "main" net.spy.memcached.OperationTimeoutException: Timeout waiting for value
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1014)
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1029)
at Main.main(Main.java:33)
Caused by: net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: 10.48.94.158/10.48.94.158:11210
at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:159)
at net.spy.memcached.internal.GetFuture.get(GetFuture.java:62)
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1008)
... 2 more
2012-10-03 17:16:34.775 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@4cee32
2012-10-03 17:16:34.775 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to exception on {QA sa=10.48.94.158/10.48.94.158:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0}
java.net.SocketException: Network is unreachable
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:369)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:242)
at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:229)
2012-10-03 17:16:34.776 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=10.48.94.158/10.48.94.158:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0}, attempt 1.
2012-10-03 17:16:38.778 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=10.48.94.158/10.48.94.158:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0}
2012-10-03 17:16:59.778 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@11671b2
2012-10-03 17:16:59.779 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to failure to connect to {QA sa=10.48.94.158/10.48.94.158:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=Cmd: 0 Opaque: 1 Key: spoon, toWrite=0, interested=0}
java.net.ConnectException: Connection timed outOk, changing the login data for the actual password I now get this error:
Error connecting to Couchbase: null
Where are you physically running the application code? If you are running from outside of the Amazon environment, this won't work out of the box. The problem is that the Couchbase node is identifying itself with the private IP address of the Amazon instance which is not routable from the outside:
2012-04-16 11:19:41.972 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.209.194.24:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
In order to get this to work, you'll either have to run the code from within Amazon or use these instructions to have the node identify itself with an IP address or hostname that is externally available: http://www.couchbase.com/docs/couchbase-manual-1.8/couchbase-bestpractic... (this is typically done with an EIP, or the public hostname of the instance but any address that is resolvable both internally and externally to Amazon will work)
This is interesting, it should look up fine.
Where is the client code running? On the same server? Are you using the public DNS name?
There had been some issues with AWS which can be set up slightly differently, but I believe they'd all been resolved.
Two things to try are to use the public EC2 hostname or the public EC2 IP. Do let me know how things are set up.