Moxi and DNS
Hi, we are using membase from the Git repo on EC2 instances with EBS volumes. We have internal DNS to register our instances. DNS resolve to the private EC2 IP. We setup our membase cluster to use those DNS. When we need to restart an instance, we get a new IP. Our DNS get updated but moxi seems to keep reference to the old IP even if the OS resolve correctly. We generally end up restarting the service on all the nodes...
Does moxi has some internal DNS caching ?
Regards,
Nicolas
We did read this post when setting up our clusters. The following is from our dev cluster.
For the Erlang process :
/usr/lib/erlang/erts-5.7.4/bin/beam.smp -A 16 -- -root /usr/lib/erlang -progname erl -- -home /home/nicolas -- -pa ./deps/menelaus/deps/erlwsh/ebin ./deps/menelaus/deps/mochiweb/ebin ./deps/menelaus/ebin ./ebin -setcookie nocookie -run ns_bootstrap -noshell -noinput -ns_server error_logger_mf_dir "/opt/membase/storage/logs" -ns_server error_logger_mf_maxbytes 10485760 -ns_server error_logger_mf_maxfiles 10 -ns_server dont_suppress_stderr_logger true -ns_server path_prefix "/opt/membase/storage" -ns_server pidfile "/opt/membase/storage/membase.pid" -ns_server ns_server_config "/etc/opt/membase/config" -mnesia dir "/opt/membase/storage/Mnesia.ns_1@dev-rtb-membase01.us-east-1b.private" -kernel inet_dist_listen_min 21100 inet_dist_listen_max 21199 -noshell -name ns_1@dev-rtb-membase01.us-east-1b.private --
For Moxi :
./bin/moxi/moxi -Z port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200 -z url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming -p 0 -Y y -O stderr
And where is your application code running?
Our application use the spy-memcached client and is running on a different server.
Are you using the latest spy-memcached client? We wrote the original and have updated it to be a "smart" client for Membase: http://techzone.couchbase.com/wiki/display/membase/prerelease+spymemcach...
With that one, you don't need to go through Moxi at all.
I'm also not quite sure I fully understand the problem here because the default Moxi that runs on each node should work correctly in the situation that you've described.
PErry
Actually, I'm saying Moxi but I think it's the Erlang process. After changing DNS it seem that when we try to add back the node to the cluster it still try to resolve to the old ip and see the node down. I could do more testing and post some logs when we see this again. I should be able to reproduce it on our dev cluster too.
Moxi should be picking this up correctly, can you show me your command-line options to Moxi?
Also, have you followed these instructions to have the server properly identify itself with a DNS entry instead of an IP: http://techzone.couchbase.com/wiki/display/membase/Using+Membase+in+the+...
Perry
Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!