Enyim Membase client receives POOLFail periodically
Hi,
We've managed to get a cluster of Membase cache servers up and running on Amazon EC2 and managed to get the Enyim Membase to connect to them successfully. But weirdly we get these error messages periodically when the the pool url points to a Load Balancer:
2010-07-28 13:44:22,340 [23] WARN Membase.Store.MessageStreamListener [(null)] - Current pool [url]http://xxx:8080/pools/default/bucketsStreaming/default[/url] has failed.
2010-07-28 13:44:22,340 [23] WARN Membase.Store.MessageStreamListener [(null)] - All pools are dead, sleeping a while.
2010-07-28 13:45:28,920 [23] ERROR Membase.Store.MessageStreamListener [(null)] - POOLFAIL
System.IO.IOException: Unable to read data from the transport connection: The connection was closed.
at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadLine()
at Membase.Store.MessageStreamListener.MessageReader.Start(Uri uri)
at Membase.Store.MessageStreamListener.Work(Object state)
This goes away when I replace the Load Balancer url with one of the nodes in the cluster, but I'm just curious as to why this can be a problem when the url points to a load balancer.
Thanks,
Hi Perry,
Thanks for getting back so quickly! We're just using the standard Amazon Load Balancer, which works fine for any other operations we put through the load balancer.
We want to use the load balancer url so we're not susceptible to losing the cluster if the nodes in the url pool are lost in the unlikely event of say, the virtual machines are lost due to problem/maintenance on the underlying physical machines, which we have come across before and would like to guard against.
On a side note, when a server is added to the cluster, existing data are not redistributed across the cluster, right? I'm assuming the only change will be happening in the hashing algorithm the client uses to decide where to get/set a given key?
Cheers,
Thanks, I'll have to see if we can reproduce that in Amazon. Does the client eventually recover from this or does it impact data operations?
To answer your question, you are correct that existing data is not redistributed across the cluster. In fact, with our Membase product we WILL redistribute the data across the cluster when you add a node.
Hope that helps.
Perry
What kind of load balancer are you using? Looks like it's closing the connection between your client and the server.
Also, what's the use case for using a load balancer in this situation? Ideally you should just be able to provide a few IP's to the Enyim client. The actual load balancing of data will take place through the client anyway (not going through the load balancer).
Perry
Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Membase: http://www.membase.com/products-and-services/overview
Call or email "sales -at- membase -dot- com" today!