Memcached Server cannost start on Win Server 2008 SP2 64 bit

Hi Michal,
It looks like we're having a problem getting the list of IP addresses on the system. Can you try following these instructions to manually register the IP address that we should listen on:

1. from "INSTALLDIR\Memcached Server\bin"
2. run "service_stop.bat"
3. run "service_unregister.bat"
4. run "service_register.bat ns_1@"
5. run "service_start.bat"

In the field, please put in the IP address on this server that you want to use our service to bind to.

Let me know how that goes, thanks.

Perry

Hi Perry.
I tried it, no success. What is strange (from my point of view) is, that file "c:\Program Files\NorthScale\Memcached Server\config\ns_1\config.dat" has size = 0 bytes.

I tried to uninstall it (a delete all rest folders), reinstall, again your instructions and same result.
Log & crash_dump files are attached

Can I get the ouput of "ipconfig /all" on your system?

What are the differences between this system and the one that it worked successfully on? Is it possible to uninstall the VMware server temporarily just to make sure?

Hi Perry. I cannot uninstall VMWare, sorry.

aa.bb.ccc.* is real internet address, 10.10.100.* is IP on second network cast, rest of interfaces are from VMWare and incoming VPN.

IPconfig:

Windows IP Configuration

Host Name . . . . . . . . . . . . : xxx
Primary Dns Suffix . . . . . . . : yyyyy.local
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : Yes
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : yyyy.local

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Broadcom NetXtreme Gigabit Ethernet #2
Physical Address. . . . . . . . . : 00-26-9E-02-D1-F6
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::5cd5:7b34:255d:2c07%18(Preferred)
IPv4 Address. . . . . . . . . . . : aa.bb.ccc.131(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
IPv4 Address. . . . . . . . . . . : aa.bb.ccc.132(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : aa.bb.ccc.1
DHCPv6 IAID . . . . . . . . . . . : 419440286
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-AC-0B-2B-00-1E-68-86-A8-6F
DNS Servers . . . . . . . . . . . : aa.bb.ccc.14
8.8.8.8
NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter 1G Internal:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) PRO/1000 PT Server Adapter
Physical Address. . . . . . . . . : 00-15-17-8F-F3-E3
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::458f:276d:a955:8fb1%11(Preferred)
IPv4 Address. . . . . . . . . . . : 10.10.100.103(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
DHCPv6 IAID . . . . . . . . . . . : 285218071
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-AC-0B-2B-00-1E-68-86-A8-6F
DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
fec0:0:0:ffff::2%1
fec0:0:0:ffff::3%1
NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet1:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet1
Physical Address. . . . . . . . . : 00-50-56-C0-00-01
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::3575:1dc5:9a40:f373%13(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.142.1(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
DHCPv6 IAID . . . . . . . . . . . : 369119318
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-AC-0B-2B-00-1E-68-86-A8-6F
DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
fec0:0:0:ffff::2%1
fec0:0:0:ffff::3%1
NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet8:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet8
Physical Address. . . . . . . . . : 00-50-56-C0-00-08
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::3ca9:eb69:599e:be89%15(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.233.1(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
DHCPv6 IAID . . . . . . . . . . . : 402673750
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-AC-0B-2B-00-1E-68-86-A8-6F
DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
fec0:0:0:ffff::2%1
fec0:0:0:ffff::3%1
NetBIOS over Tcpip. . . . . . . . : Enabled

PPP adapter RAS (Dial In) Interface:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : RAS (Dial In) Interface
Physical Address. . . . . . . . . :
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 192.168.223.10(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.255
Default Gateway . . . . . . . . . :
NetBIOS over Tcpip. . . . . . . . : Disabled

Tunnel adapter Local Area Connection* 8:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : isatap.{7C880928-1CE6-40DE-A867-8AEFFF76F573}
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 9:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 11:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : 6TO4 Adapter
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2002:4d5d:d083::4d5d:d083(Preferred)
IPv6 Address. . . . . . . . . . . : 2002:4d5d:d084::4d5d:d084(Preferred)
Default Gateway . . . . . . . . . : 2002:c058:6301::c058:6301
DNS Servers . . . . . . . . . . . : aa.bb.ccc.14
8.8.8.8
NetBIOS over Tcpip. . . . . . . . : Disabled

Tunnel adapter Local Area Connection* 12:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft ISATAP Adapter #3
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 15:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft ISATAP Adapter #6
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 16:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft ISATAP Adapter #7
Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Thanks Michal,

What are the differences between this system and the one that it worked successfully on?

Perry

Second one (where memcached is working) is WinServer 2008 R2 (not SP2), don't have VMWare and its virtualized server running on XEN.
Problematic server is physical server with VMWare.

Both are 4Quad core CPU, 8GB Ram.

I have a similar issue. We have has MCDS installed on a production machine for the last few months. This morning we did a restart on the server after a windows update but MCDS wouldn't start back up.

We have added additional IP addresses to the server since the install which may be related to this issue. Like the OP, our ns_1/config.dat file is empty. I tried running the instructions in this thread, and I'm receiving the following errors:
C:\Program Files\NorthScale\Memcached Server>"C:\Program Files\NorthScale\Memcached Server\bin\ip_addr.bat"

When I run the ip_addr.bat script I receive the following error:
C:\Program Files\NorthScale\Memcached Server>"C:\Program Files\NorthScale\Memcac
hed Server\bin\erlang\erts-5.7.4\bin\erl.exe" -pa bin\ns_server\ebin -eval "io:f
ormat(\"~s\", [addr_util:get_my_address()])." -noshell -s init stop
{"init terminating in do_boot",{{badmatch,{error,efault}},[{addr_util,list_all_a
ddresses,0},{addr_util,get_my_address,1},{erl_eval,do_apply,5},{erl_eval,expr,5}
,{erl_eval,expr_list,6},{erl_eval,expr,5},{init,start_it,1},{init,start_em,1}]}}

Crash dump was written to: erl_crash.dump
init terminating in do_boot ()

Can you try following the instructions in post #2 of this thread to register the IP address and report back? It would also be good if you could try installing beta 4 of our Membase server (which includes support for memcached bucket types) to see if that resolves this issue?

Thanks

Perry

Tried running instructions in comment #2 and no luck. Looks to be an Erlang crash when running "service_register.bat ns_1@192.168.238.66" Output below:

[CODE]C:\Program Files\NorthScale\Memcached Server\bin>service_stop.bat

C:\Program Files\NorthScale\Memcached Server\bin>set NS_BIN=C:\Program Files\Nor
thScale\Memcached Server\bin\

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ROOT=C:\Program Files\No
rthScale\Memcached Server\bin\..

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ERTS=C:\Program Files\No
rthScale\Memcached Server\bin\erlang\erts-5.7.4\bin

C:\Program Files\NorthScale\Memcached Server\bin>"C:\Program Files\NorthScale\Me
mcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.exe" stop NorthScaleServer
C:\Program Files\NorthScale\Memcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.ex
e: Failed to stop service NorthScaleServer.
Error: The service has not been started.

C:\Program Files\NorthScale\Memcached Server\bin>service_unregister.bat

C:\Program Files\NorthScale\Memcached Server\bin>set NS_BIN=C:\Program Files\Nor
thScale\Memcached Server\bin\

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ROOT=C:\Program Files\No
rthScale\Memcached Server\bin\..

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ERTS=C:\Program Files\No
rthScale\Memcached Server\bin\erlang\erts-5.7.4\bin

C:\Program Files\NorthScale\Memcached Server\bin>"C:\Program Files\NorthScale\Me
mcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.exe" remove NorthScaleServer
C:\Program Files\NorthScale\Memcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.ex
e: Service NorthScaleServer removed from system.

C:\Program Files\NorthScale\Memcached Server\bin>service_register.bat ns_1@192.1
68.238.66

C:\Program Files\NorthScale\Memcached Server\bin>set NS_BIN=C:\Program Files\Nor
thScale\Memcached Server\bin\

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ROOT=C:\Program Files\No
rthScale\Memcached Server\bin\..

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ERTS=C:\Program Files\No
rthScale\Memcached Server\bin\erlang\erts-5.7.4\bin

C:\Program Files\NorthScale\Memcached Server\bin>for /F "tokens=1* delims=" %a i
n ('cmd /q/c "C:\Program Files\NorthScale\Memcached Server\bin\ip_addr.bat"') do
set IP_ADDR=%a

Crash dump was written to: erl_crash.dump
init terminating in do_boot ()

C:\Program Files\NorthScale\Memcached Server\bin>set IP_ADDR={"init terminating
in do_boot",{{badmatch,{error,efault}},[{addr_util,list_all_addresses,0},{addr_u
til,get_my_address,1},{erl_eval,do_apply,5},{erl_eval,expr,5},{erl_eval,expr_lis
t,6},{erl_eval,expr,5},{init,start_it,1},{init,start_em,1}]}}

C:\Program Files\NorthScale\Memcached Server\bin>set NS_NAME=ns_1@{"init termina
ting in do_boot",{{badmatch,{error,efault}},[{addr_util,list_all_addresses,0},{a
ddr_util,get_my_address,1},{erl_eval,do_apply,5},{erl_eval,expr,5},{erl_eval,exp
r_list,6},{erl_eval,expr,5},{init,start_it,1},{init,start_em,1}]}}

C:\Program Files\NorthScale\Memcached Server\bin>if not "ns_1@192.168.238.66" ==
"" set NS_NAME=ns_1@192.168.238.66

C:\Program Files\NorthScale\Memcached Server\bin>cd "C:\Program Files\NorthScale
\Memcached Server\bin\.."

C:\Program Files\NorthScale\Memcached Server>"C:\Program Files\NorthScale\Memcac
hed Server\bin\erlang\erts-5.7.4\bin\erlsrv.exe" add NorthScaleServer -name ns_1
@192.168.238.66 -onfail restart -workdir "C:\Program Files\NorthScale\Memcached
Server\bin\.." -args "-pa bin/ns_server/ebin -pa bin/ns_server/deps/emoxi/ebin -
pa bin/ns_server/deps/menelaus/ebin -pa bin/ns_server/deps/menelaus/deps/mochiwe
b/ebin -pa bin/ns_server/deps/gen_smtp/ebin -setcookie nocookie -sasl sasl_error
_logger false -sasl error_logger_mf_dir \\\"logs\\\" -sasl error_logger_mf_maxby
tes 10485760 -sasl error_logger_mf_maxfiles 10 -kernel inet_dist_listen_min 2110
0 inet_dist_listen_max 21199 -run ns_bootstrap override_resolver -ns_server rest
art_script \\\".\\bin\\service_reregister.bat\\\" -- "
C:\Program Files\NorthScale\Memcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.ex
e: Service NorthScaleServer added to system.

C:\Program Files\NorthScale\Memcached Server>service_start.bat
'service_start.bat' is not recognized as an internal or external command,
operable program or batch file.

C:\Program Files\NorthScale\Memcached Server>cd bin

C:\Program Files\NorthScale\Memcached Server\bin>service_start.bat

C:\Program Files\NorthScale\Memcached Server\bin>set NS_BIN=C:\Program Files\Nor
thScale\Memcached Server\bin\

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ROOT=C:\Program Files\No
rthScale\Memcached Server\bin\..

C:\Program Files\NorthScale\Memcached Server\bin>set NS_ERTS=C:\Program Files\No
rthScale\Memcached Server\bin\erlang\erts-5.7.4\bin

C:\Program Files\NorthScale\Memcached Server\bin>"C:\Program Files\NorthScale\Me
mcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.exe" start NorthScaleServer
C:\Program Files\NorthScale\Memcached Server\bin\erlang\erts-5.7.4\bin\erlsrv.ex
e: Service NorthScaleServer started.

C:\Program Files\NorthScale\Memcached Server\bin>[/CODE]

Thanks Karl, looks like some problem with trying to understand the list of IP addresses. Can you try installing our 1.6.0 beta4 server and see if the behavior is better?

Thanks.

Perry

Hi Perry,

This is a production server so we would prefer not to install Membase Server unless it had been tested on our staging servers. Also, we have only worked with Memcached Server up to now and would prefer not to begin working with another product at the moment.

Is there any other way to get to the bottom of this?

Thanks
Karl

Makes sense Karl. I certainly understand your desire to resolve this issue. Membase server actually provides complete out-of-the-box compatibility with memcached and is very similar (though enhanced) to our 1.0.3 release in terms of configuration and management. 1.6 is a rollup of many fixes and enhancements to 1.0.3, we will be retiring that version eventually.

It would be good to know whether your particular issue exists in 1.6 as we have much more enhanced debugging and troubleshooting capabilities. I can't at this point guarantee that we can get this configuration working properly with the current release of 1.0.3 and we would like to avoid making any changes on that code branch if possible.

Let's keep digging on this though, we've had very good success with the workaround in the past. Can you attempt uninstalling the current server, making sure all the files have been deleted and then reinstall? You'll probably have to run through those steps again.

Thanks Karl.

Perry