Sdk compilation issue on high sierra

Python SDK
1

Just wondering if anyone has successfully installed/built the python sdk on MacOS High Sierra?

I’m running into this error:

gcc -bundle -undefined dynamic_lookup -L/Users/jonernster/anaconda2/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.6-x86_64-2.7/src/exceptions.o build/temp.macosx-10.6-x86_64-2.7/src/ext.o build/temp.macosx-10.6-x86_64-2.7/src/result.o build/temp.macosx-10.6-x86_64-2.7/src/opresult.o build/temp.macosx-10.6-x86_64-2.7/src/callbacks.o build/temp.macosx-10.6-x86_64-2.7/src/cntl.o build/temp.macosx-10.6-x86_64-2.7/src/convert.o build/temp.macosx-10.6-x86_64-2.7/src/bucket.o build/temp.macosx-10.6-x86_64-2.7/src/store.o build/temp.macosx-10.6-x86_64-2.7/src/constants.o build/temp.macosx-10.6-x86_64-2.7/src/multiresult.o build/temp.macosx-10.6-x86_64-2.7/src/miscops.o build/temp.macosx-10.6-x86_64-2.7/src/typeutil.o build/temp.macosx-10.6-x86_64-2.7/src/oputil.o build/temp.macosx-10.6-x86_64-2.7/src/get.o build/temp.macosx-10.6-x86_64-2.7/src/counter.o build/temp.macosx-10.6-x86_64-2.7/src/http.o build/temp.macosx-10.6-x86_64-2.7/src/htresult.o build/temp.macosx-10.6-x86_64-2.7/src/ctranscoder.o build/temp.macosx-10.6-x86_64-2.7/src/observe.o build/temp.macosx-10.6-x86_64-2.7/src/iops.o build/temp.macosx-10.6-x86_64-2.7/src/connevents.o build/temp.macosx-10.6-x86_64-2.7/src/pipeline.o build/temp.macosx-10.6-x86_64-2.7/src/views.o build/temp.macosx-10.6-x86_64-2.7/src/n1ql.o build/temp.macosx-10.6-x86_64-2.7/src/fts.o build/temp.macosx-10.6-x86_64-2.7/src/ixmgmt.o -L/usr/local/lib -L/Users/jonernster/anaconda2/lib -lcouchbase -o build/lib.macosx-10.6-x86_64-2.7/couchbase/_libcouchbase.so
    ld: file not found: /usr/lib/system/libsystem_coretls.dylib for architecture x86_64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/Users/jonernster/anaconda2/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/nz/wbv7ph057zzcqtt5h6vy06qh0000gp/T/pip-build-Impg6F/couchbase/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/nz/wbv7ph057zzcqtt5h6vy06qh0000gp/T/pip-9u9_M2-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/nz/wbv7ph057zzcqtt5h6vy06qh0000gp/T/pip-build-Impg6F/couchbase/

Thanks

2

Hi @jmernster,

This looks to be due to the MacOS CoreTLS library being relocated in High Sierra. There is a workaround posted below:

i.e.:

sudo ln -s /usr/lib/libcoretls.dylib /usr/lib/libsystem_coretls.dylib

This seems to be a transitive dependency, i.e. libcouchbase.2.dylib->libSystem.B.dylib ->libsystem_coretls.dylib - just going to talk to the libcouchbase maintainer to determine if this is a known issue, so we can arrive at a permanent fix to this problem in the next release.

Is this workaround satisfactory for the time being?

Many thanks,

Ellis

3

Hi Ellis,

Thanks for the response. The would be sufficient for my needs but unfortunately I seem to have an issue creating that link:

jonernster$ sudo ln -s /usr/lib/libcoretls.dylib /usr/lib/libsystem_coretls.dylib
ln: /usr/lib/libsystem_coretls.dylib: Operation not permitted

Same thing if I directly run as root…seems odd?

4

Ah, possibly it requires you to disable System Integrity Protection temporarily to achieve this… However, on inspection of my Sierra machine, it appears that the renaming has already occurred, and I can compile/link with no issues. Have you just upgraded from an older OS? Perhaps there is something cached. If you have been trying to build from a git checkout, could you try:
rm -rf build/ && python setup.py --buildext --inplace && python setup.py install ?

If this fails, perhaps try building libcouchbase from source by checking out from the couchbase/libcouchbase repository on GitHub, as follows:

git clone http://github.com/couchbase/libcouchbase.git
cd libcouchbase && export LCB_DIR=$PWD && cmake -G 'Unix Makefiles' && sudo make install

Then check out and build couchbase-python-client:

cd .. && git clone http://github.com/couchbase/couchbase-python-client.git
cd couchbase-python-client
python setup.py clean && python setup.py build_ext --inplace --library-dir $LCB_DIR/lib --include-dir $LCB_DIR/include && sudo python setup.py install

Hope that helps.

Many thanks,

Ellis

5

Hi Ellis,

I was able to compile libcouchbase after installing cmake using the steps provided, but unfortunately the couchbase-python-client fails the same way as previously mentioned, even after creating a new build directory with no reference to a previously attempted install/compile.

The upgrade to high sierra was from the previous latest version so it wasn’t a massive jump up to the latest or anything.

6

Hi @jmernster,

Thanks for the reply. That’s interesting… Perhaps there was some transitional code in Sierra that allows it to work. Will try to get this fixed permanently, but in the meantime, I can only suggest disabling SIP, making the symlink, enabling SIP, rebooting and trying to build again.

I suppose this should have the advantage that it will work with the stock PyPi packages, but hopefully we will come up with a proper solution in the very near future.

Many thanks,

Ellis

7

Hi Ellis,

I’ll give that a shot next time I plan to reboot and post back; thanks for your help!

1 Like
8

Disabled SIP and ran sudo ln -s /usr/lib/libsystem_coretls.dylib /usr/lib/system/libsystem_coretls.dylib which allowed me to compile the couchbase-python-client.

Thanks again for the help!

9

You’re welcome - I assume you were able to reenable SIP and reboot and retain the changes? Obviously I would not recommend keeping SIP disabled.

Anyway, will raise a bug about this on Monday to test this on High Sierra and come up with a proper fix.

Thanks,

Ellis

10

As promised: https://issues.couchbase.com/browse/PYCBC-455

Many thanks,

Ellis

11

Thanks Ellis. Confirming that the python client still works (just doing a simple from couchbase.bucket import Bucket test) after re-enabling SIP.

Thanks

1 Like
12

Hi @jmernster, we have been trying to reproduce this on our High Sierra installs for a while but not come across the same issue. Have you seen it on any other High Sierra systems?

Many thanks,

Ellis

13

Unfortunately I only have access to this one laptop that I experienced the issue on. It’s not a brand new laptop/fresh install so perhaps a combination of previous sdk installs, using brew and/or compiled installations created some sort of conflict/problem along the way. If you guys aren’t seeing the same issue then I wouldn’t spend too much time on it, especially since you were able to find a solution/workaround.

Thanks

1 Like
14

OK, many thanks @jmernster - I will put this issue to sleep for now then.