Running on windows x64, use of select event demultiplexing in libevent?
Mon, 02/28/2011 - 12:23
Does anyone have any experience with membase running on x64 windows scaling to hundreds of machines? I'm concerned it won't.
I installed it and started poking around memcached.exe's threads, and found that it appears to be waiting on network events using select. Same with moxi. After looking through the source code, it appears to be based on libevent, which, though not listed on their website, has source code that includes windows IO completion port use. I think this is with libevent2.
Does the prebuilt windows installers use libevent2? If not, is there a plan to move that direction, or would I be on my own?
Aaron
Update: Found this snippet in the latest libevent distribution (in whatsnew-2.0.txt):
Unfortunately, the main Windows backend is still select()-based: from
testing the IOCP backends on the mailing list, it seems that there isn't
actually a way to tell for certain whether a socket is writable with IOCP.
Libevent 2.1 may add a multithreaded WaitForMultipleEvents-based
backend for better performance with many inactive sockets and better
That sucks. I would still be interested in hearing if anyone else has run this on decent sized windows clusters (hundreds of cores, hundreds of connections per machine).
My experience is that this select based backend quickly consumes a sizable amount of the CPU just doing demultiplexing, and I need every bit of it for actual processing work.