Link error for memcached: undefined reference to `evthread_use_pthreads'

dhoblem1 · September 12, 2017, 10:18am

I am trying to build the latest Couchbase source (master) on Ubuntu16.04 with ppc64le platform.
At the build level its failing with below error :

[ 75%] Linking CXX executable …/memcached
…/libmemcached_daemon.a(libevent_locking.cc.o): In function setup_libevent_locking()': /root/meghali/couchbase/kv_engine/daemon/libevent_locking.cc:27: undefined reference toevthread_use_pthreads’
collect2: error: ld returned 1 exit status
kv_engine/daemon/CMakeFiles/memcached.dir/build.make:119: recipe for target ‘kv_engine/memcached’ failed
make[2]: *** [kv_engine/memcached] Error 1
CMakeFiles/Makefile2:11718: recipe for target ‘kv_engine/daemon/CMakeFiles/memcached.dir/all’ failed
make[1]: *** [kv_engine/daemon/CMakeFiles/memcached.dir/all] Error 2
Makefile:138: recipe for target ‘all’ failed
make: *** [all] Error 2

I found a similar bug being raised at the couchbase issues page, however the fix it mentions is already merged at master still its failing.
The issue link for reference is : https://issues.couchbase.com/browse/MB-23836?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aworklog-tabpanel

Can someone please help me understand the issue and provide some pointers around it?

Thanks,
Meghali Dhoble

drigby · September 12, 2017, 11:01am

It sounds like the build of libevent you have isn’t build with pthreads support - see libevent documentation at: http://www.wangafu.net/~nickm/libevent-2.0/doxygen/html/thread_8h.html

What version / build of libevent are you using? Given ppc isn’t a supported platform currently, you’ll likely need to manually build libevent - the flags we normally build with can be seen at:

github.com

couchbase/tlm/blob/master/deps/packages/libevent/CMakeLists.txt

# Downloads the declared version of libevent source code and builds it.

if (APPLE OR WIN32)
  # We bundle OpenSSL on MacOS X and Windows, and we want support
  # for OpenSSL in libevent so we need to install the OpenSSL Package
  SET (_openssl_install "${CMAKE_CURRENT_BINARY_DIR}/dep-install")
  SET (_orig_install_prefix "${CMAKE_INSTALL_PREFIX}")
  SET (CMAKE_INSTALL_PREFIX "${_openssl_install}")
  DECLARE_DEP(openssl VERSION 1.0.2k-cb1 PLATFORMS windows_msvc windows_msvc2015 macosx)
  SET (CMAKE_INSTALL_PREFIX "${_orig_install_prefix}")
  SET (_libevent_extra_options -D OPENSSL_ROOT_DIR=${_openssl_install})
endif (APPLE OR WIN32)

include(ExternalProject)

### Download, configure and build libevent ####################################
ExternalProject_Add(libevent
  GIT_REPOSITORY ${_git_repo}
  GIT_TAG ${_git_rev}

This file has been truncated. show original

dhoblem1 · September 12, 2017, 11:20am

Thanks for your reply @drigby!
I have installed libevent directly using “apt-get install” as its supported on ppc64le, hence I have not build it from source.
The version I have is libevent-2.0-5 and I see the libevent_pthread libraries also listed under the path: /usr/lib/powerpc64le-linux-gnu,

/usr/lib/powerpc64le-linux-gnu/libevent_pthreads-2.0.so.5
/usr/lib/powerpc64le-linux-gnu/libevent_pthreads.a
/usr/lib/powerpc64le-linux-gnu/libevent_pthreads-2.0.so.5.1.9
/usr/lib/powerpc64le-linux-gnu/libevent_pthreads.so

I am not sure if there are more files expected to be present here. If you still think that building manually with those flags should help then I will give it a try.
Thanks,
Meghali

drigby · September 12, 2017, 11:34am

Note we use libevent 2.18 (see https://github.com/couchbase/tlm/blob/master/deps/packages/CMakeLists.txt#L164) so you might need a newer version - I don’t know when the missing function you’re encountering was introduced.

dhoblem1 · September 15, 2017, 12:32pm

Hi @drigby,
As per your suggestion, I have installed libevent 2.18 and tried the build again. However I am still getting the same error.
I installed libevent 2.18 from source and followed below steps:

$ git clone https://github.com/couchbasedeps/libevent.git
$ cd libevent && git checkout release-2.1.8-stable-cb
$ ACLOCAL=“aclocal -I /usr/share/aclocal” autoreconf -if
$ ./autogen.sh && ./configure && make && sudo make install

Please correct me if I have missed something.

dhoblem1 · September 20, 2017, 12:46pm

Hi @drigby
I tried re-building the libevent with an additional flag -D EVENT__DISABLE_THREAD_SUPPORT=OFF
It generated the libevent_pthreads.so and all relevant files still the error is observed.

Also I tried to see more detailed logs with verbose output for the specific component make command, which gives below logs:
$ make -ddd
No need to remake target ‘/usr/lib/powerpc64le-linux-gnu/libssl.so’.
Considering target file ‘/usr/lib/powerpc64le-linux-gnu/libcrypto.so’.
Looking for an implicit rule for ‘/usr/lib/powerpc64le-linux-gnu/libcrypto.so’.
Trying pattern rule with stem ‘libcrypto.so’.
Trying implicit prerequisite ‘/usr/lib/powerpc64le-linux-gnu/libcrypto.so,v’.
Trying pattern rule with stem ‘libcrypto.so’.
…
Trying implicit prerequisite ‘/usr/lib/powerpc64le-linux-gnu/SCCS/s.libcrypto.so’.
No implicit rule found for ‘/usr/lib/powerpc64le-linux-gnu/libcrypto.so’.
Finished prerequisites of target file ‘/usr/lib/powerpc64le-linux-gnu/libcrypto.so’.
No need to remake target ‘/usr/lib/powerpc64le-linux-gnu/libcrypto.so’.
Finished prerequisites of target file ‘kv_engine/memcached’.
Must remake target ‘kv_engine/memcached’.
Putting child 0x10000b5ac80 (kv_engine/memcached) PID 8304 on the chain.
Live child 0x10000b5ac80 (kv_engine/memcached) PID 8304
[ 94%] Linking CXX executable …/memcached
Reaping winning child 0x10000b5ac80 PID 8304
Live child 0x10000b5ac80 (kv_engine/memcached) PID 8306
…/libmemcached_daemon.a(libevent_locking.cc.o): In function setup_libevent_locking()': /root/meghali/couchbase/kv_engine/daemon/libevent_locking.cc:28: undefined reference toevthread_use_pthreads’

Looking at the logs I am unable to figure out what is missing. Any pointers around this would be helpful.

drigby · September 20, 2017, 12:50pm

From that it still sounds like the evthread_use_pthreads symbol is missing from your build of pthreads. If you examine the list of symbols in your libevent shared libraries using nm -D does that symbol appear? e.g.

nm -D libevent_pthreads.so | grep evthread_use_pthreads

?

The other thing you could check is which libevent libraries are you linking against - if you run make VERBOSE=1 it will print out the full link line.

dhoblem1 · September 21, 2017, 9:01am

Hi @drigby,
Yes, the nm -D command lists the symbols for libevent shared libraries for me.

~/meghali/couchbase/libevent# nm -D /usr/local/lib/libevent_pthreads.so |grep evthread_use_pthreads
0000000000001558 T evthread_use_pthreads

Here is the output with VERBOSE=1 option with make :

[ 94%] Linking CXX executable …/memcached
cd /root/meghali/couchbase/build/kv_engine/daemon && /root/meghali/cmake-3.7.1/bin/cmake -E cmake_link_script CMakeFiles/memcached.dir/link.txt --verbose=1
/usr/bin/c++ -std=c++11 -g -Wall -Wredundant-decls -Wmissing-braces -fno-strict-aliasing -pedantic -fvisibility=hidden -pthread -Werror=switch -O3 -DNDEBUG -g CMakeFiles/memcached.dir/main.cc.o -o …/memcached -Wl,-rpath,/root/meghali/couchbase/build/kv_engine:/root/meghali/couchbase/build/platform/cbcompress:/root/meghali/couchbase/build/platform:/root/meghali/couchbase/build/phosphor:/usr/local/lib: -rdynamic …/libmemcached_daemon.a …/libauditd.so.0.1.0 …/libmcd_time.so.1.0.0 …/libmcbp.a …/libmcd_util.so.1.0.0 …/…/platform/cbcompress/libcbcompress.so …/libengine_utilities.so …/…/platform/libJSON_checker.so.1.0.0 …/libmemcached_rbac.so.1.0.0 …/libcbsasl.so.1.1.1 …/…/platform/libdirutils.so.0.1.0 …/libcbcrypto.so.1.0.0 …/…/subjson/lib/libsubjson.a …/libxattr.so.0.1.0 …/…/platform/libcJSON.so.1.1.0 …/…/platform/libplatform.a …/…/platform/libplatform_so.so.0.1.0 …/…/phosphor/libphosphor.so -ldl -lpthread -lrt -lm -lsnappy /usr/local/lib/libjemalloc.so -levent_core -levent_extra -lssl -lcrypto
…/libmemcached_daemon.a(libevent_locking.cc.o): In function setup_libevent_locking()': /root/meghali/couchbase/kv_engine/daemon/libevent_locking.cc:28: undefined reference toevthread_use_pthreads’
collect2: error: ld returned 1 exit status
kv_engine/daemon/CMakeFiles/memcached.dir/build.make:119: recipe for target ‘kv_engine/memcached’ failed
make[2]: *** [kv_engine/memcached] Error 1
make[2]: Leaving directory '/root/meghali/couchbase/build’
CMakeFiles/Makefile2:11718: recipe for target ‘kv_engine/daemon/CMakeFiles/memcached.dir/all’ failed
make[1]: *** [kv_engine/daemon/CMakeFiles/memcached.dir/all] Error 2
make[1]: Leaving directory '/root/meghali/couchbase/build’
Makefile:138: recipe for target ‘all’ failed
make: *** [all] Error 2

drigby · September 21, 2017, 9:44am

Note you don’t have libevent_pthreads included in the set of libraries you are linking against - you only have core and extra:

You probably want to look at the contents of tlm/cmake/Modules/FindCouchbaseLibevent.cmake and the matching CMake log output - you likely have some configuration issue where the build system isn’t including libevent_pthread in LIBEVENT_LIBRARIES.

dhoblem1 · September 22, 2017, 9:36am

Thanks @drigby, that was a perfect catch! My code doesn’t have the library added for libevent_pthreads. Your pointers helped me understand the issue.

I had added the paths at tlm/cmake/Modules/FindCouchbaseLibevent.cmake for “find_library (LIBEVENT_THREAD_LIB” however I figured out that the code is not entering the conditional statement “if(NOT _supported_platform)” where the library paths are set for pthread libraries.

I am now going through the code and trying to understand more as to why this conditional doesn’t include architecture “_arch” check and which un-supported platforms this part of code might have been written for.