Couchbase PHP SDK - Persistent Connections
Hi,
We are currently looking to deploy Couchbase for a high traffic large site.
Everything is working with no issues - however across the set-up we are seeing network issues between our php-fpm server and the Couchbase servers.
This largely seems to be caused by each php thread creating a new connection to (1) server:8091 and then (2) server:11210.
Reading the documentation:
3.1.1.1.1.1. Performance Considerations Note that calling new Couchbase($url) will make a REST call to Couchbase Server to determine the cluster topology. This is a relatively expensive call and that's the reason why we use persistent connections by default, so that this call doesn't have to be made more often then necessary, and most importantly not on every user-request. That means, if you have any code that calls new Memcached(); in a tight loop, you want to move the call to new Couchbase(); to outside of that loop. If you compare performance between ext/memcached and ext/couchbase and find ext/couchbase to run at 1/2 the speed of ext/memcached, look for places where new Couchbase(); is called in a loop or otherwise repeatedly.
I have read this several times - and it seems to be saying that ext/couchbase will maintain connections - and have them persistent.
This is a relatively expensive call and that's the reason why we use persistent connections by default, so that this call doesn't have to be made more often then necessary, and most importantly not on every user-request.
My initial question is - does the PHP SDK use persistent connections across user requests?
e.g. for every HTTP request that goes through to php-fpm should there be calls to the cluster on 8091 and then specific server on 11210?
I can not decide if I am reading the documentation wrong - of if I have got something configured wrong.
Regards
David
Any update ?
Same problem here, the connection start with each request.
The recent commit may solve your problems: https://github.com/couchbase/php-ext-couchbase/commit/e959001ce680862ee1...
Just to follow up on what daschl said, I have bits available now and if you'd like to help me test them, that'd be great. We're going to go through a full testing cycle and have an updated release here soon as well, likely within the next week.
If you're interested, post on this thread what OS and architecture and I'll get you some bits.
Hi,
Our environment is Ubuntu x86_64.
If we could get access to PHP1.1 that would be very useful.
Regards
David
The fix has been applied to the 1.0 branch, but I'm currently not aware if this issue has been removed in 1.1
You can download the 1.1-dp2 preview from here: http://www.couchbase.com/develop/php/next Note that view support is currently not working in there, but it should be possible to validate your issue.
To test the fix in the 1.0 branch, you need to clone the repository from here: https://github.com/couchbase/php-ext-couchbase/tree/1.0.x . If you haven't done this yet, the readme provides the instructions to do so (it's not that hard on ubuntu, just make sure you have the php5-dev packages installed so that the "phpize" command is available.
Hope that helps,
Michael
David:
Did you get what you needed from daschl's reply? The fix is currently only on the 1.0 branch, but I can probably merge it up for you since we'll be doing so anyway. Also, would be useful to know Ubuntu 10 or 11. We build to the dependencies in the OS.
Hello,
I installed these packages :
http://packages.couchbase.com/ubuntu/pool/lucid/main/libv/libvbucket/lib...
http://packages.couchbase.com/ubuntu/pool/lucid/main/libc/libcouchbase/l...
http://packages.couchbase.com/ubuntu/pool/lucid/main/libc/libcouchbase/l...
And installed the php extension from :
https://github.com/couchbase/php-ext-couchbase/zipball/1.0.x
I still get the segfault, in the Apache error log "child pid ... exit signal Segmentation fault (11)", when I try to use persistent connections with this code :
$cb = new Couchbase("127.0.0.1:8091", '', '', 'default', true);
$cb->set("a", 1);
And I'm very sad (of course). The canary mismatch has gone but the segfault is still here.
I'm not sure if this is helpful to you, but it seems in the current 1.1.0-dp2 it is fixed (it works for me at least).
I'm running libcouchbase 1.1.0.dp6 and php-ext-couchbase on 1.1.0-dp2. The following code works for me without segfaults:
php -r '$cb = new Couchbase("127.0.0.1:8091", "", "", "default", true); $cb->set("a", 1); echo $cb->get("a")."\n";'
===> 1
So if upgrading to a developer-preview is an option for you, this may lead to a quick win.
I'll see if I can get it to run with the versions you provided here.
I would like to try :) but I cannot find the libcouchbase 1.1.0.dp6 :(
I tried with the new extension but (of course) I get an error : /usr/sbin/apache2: symbol lookup error: /usr/lib/php5/20090626/couchbase.so: undefined symbol: libcouchbase_set_couch_complete_callback
Please could you tell me where I could download the libcouchbase 1.1.0.dp6 ?
Thank you
Julien
Of course, there is a "preview" repo for that available here:
deb http://packages.couchbase.com/preview/ubuntu oneiric oneiric/main
If you update afterwards, the packages should be updated to the new version automatically!
Hi,
I tried adding the unstable repo but it's not compatible with the stable Debian (squeeze) I have on my test server. libevent and libstdc++ are needed in a new version, I cannot install them so I cannot install the latest version of libcouchbase.
Any idea of what I could do ? Apart from waiting ? :)
Thx
In that case, the most reliable thing would be to build libvbucket and libcouchbase from the tarballs associated with the release. couchbase.com/develop/c/next has them listed.
After untarring, just do ./configure && make && make install
After that, it should be possible to build the PHP from source.
This is more reliable than cloning the git repo, since all of the autoconf/automake checking has been built in to the source release tarballs.
yay it's working :) and it's 2.5 times faster than Memcache !
I installed from these packages :
http://packages.couchbase.com/preview/ubuntu/pool/lucid/main/libc/libcou...
http://packages.couchbase.com/ubuntu/pool/lucid/main/libv/libvbucket/lib...
http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-dp2-ub...
Did not need to compile anything.
Glad to hear that it worked for you! Also, awesome that it's so much faster.
Thanks for your feedback!
Hello,
now it's not working, I have segfaults again as soon as I enable persistent connections.
It's working randomly :(
Should I open a bug ticket ?
Thanks
Can you tell me what OS/Architecture you're using? I'd like to send you a build to see if it resolves things in your environment. If it doesn't, then we'll need to open a ticket and get some information.
SDK version 1.0.4 is doing the same to me as well. When i originally tested it with just the default bucket and no auth with persistent connections it seemed fine. I removed the default bucket, created a new one with a different name and the connection with server,user,pass,bucket,persistent segfaults on the second connection attempt.
Hi there, I don't know if I am being silly, but I just signed up and can't see the PHP 1.1 SDK for Windows x86 64-bit...
Please let me know where (or if) this exists. I am rather excited to get started with this - 'Big Data' is all the rave these days :-)
Thanks and regards.
Martin
Hello,
you are not silly - the SDK is currently only available on *NIX platforms. As far as I know, there are plans to provide a windows build in the near future, but we'll need ingenthr for a explicit answer on this.
Upgraded to 1.0.5 which seemed promising according to the changelog.. Alas... Issue still persists. Persistent connections seg faults still with php extension 1.0.5
That's definitely unexpected. Any chance you can just run this in a simple script and generate a core or get a stack trace on it? Let me know if you need more help.
I've tried a direct PM and a direct reply here and was flagged as spam..? Please contact me if you want a core dump etc. This is still and issue in my stack.
Okay, I'll drop you an email. Sorry for the spam issue.
Unable to create a small proof thus far to trigger the issue but it flat out bombs every time in production using persistant connections. Can you PM me what you need from me to assist? (I sent you a PM right after you posted this already fyi)
I receive segfault in apache error log when trying to instance a persistent connection more than once.
Here's some sample php code:
<?php
$user = NULL;
$password = NULL;
$bucket = 'charlie';
$hostAndPort = 'localhost:8091';
$persistent = TRUE;
$cb = new \Couchbase($hostAndPort, $user, $password, $bucket, $persistent);
$cb = new \Couchbase($hostAndPort, $user, $password, $bucket, $persistent);
$cb->set('food', 'tacos');
var_dump($cb, $cb->get('food'));results in : [notice] child pid 2194 exit signal Segmentation fault (11)
Comment out one of the calls, or set $persistent to FALSE on the second call, and it works as expected.
I'm guessing additional calls for persistent connections results in the segfault.
Any thoughts why?
We've recently fixed a couple of memory handling bugs in the PHP extension and are working to test everything and send along a release.
What OS/Architecture are you on? I may be able to send you a pre-release so you can help us verify the fix (and have working persistent connections)
Here is my environment:
PHP 5.3.10-1ubuntu3.4 with Suhosin-Patch
Ubuntu Server 12.04 64-bit.
Couchbase: version 1.8.0 community edition (build-55)
Apache Server version: Apache/2.2.22 (Ubuntu)
I have tested on a few different machines with similar setups, and have same segfault result on all.
Debian Lenny x86_64, Latest Zend Server build running PHP 5.3.9
1.0.5-10 prerelease seems to be better. No segfaults in staging thus far but i am a bit curious as to what the expected behavior is. We're using a single node with all couch buckets and simply refreshing a page increased the connection count to the server until it hit 104 connections..and seems to be stable there.. does that seem correct?
If you're using persistent connections, you should expect to see one connection per process, so that 104 could well be right. That's more dependent on how your PHP is deployed.
The server has a total of 25 apache processes so that really doesn't seem right..
Are you using Apache Prefork with mod_php or php-fpm?
Prefork mod_php
We may have to check that out. If you don't mind, can you run a netstat -a at the server side? It'd be interesting to see what state those connections are in. Perhaps we're counting CLOSE_WAIT state connections. We don't have any open issues with a leak.
Hello, I am trying to use the client and I am receiving the segmentation fault as well. Mountain Lion.
These are the steps I have followed.
(1) wget http://packages.couchbase.com/releases/2.0.0-beta/couchbase-server-community_x86_64_2.0.0-beta.zip
(2) Unpacked and copied CouchBase Server.app in my Applications folder
(3) Installed libcouchbase. I have tried both the upcoming version, like
brew install https://github.com/couchbase/homebrew/raw/preview/Library/Formula/libcouchbase.rb
, and also the stable version
brew install https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouchbase.rb
(4) Downloaded binary and unpacked it:
wget <a href="http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-dp5-macosx-x86_64.tar.gz;<br /> tar" title="http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-dp5-macosx-x86_64.tar.gz;<br /> tar">http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-dp5-ma...</a> xzf php-ext-couchbase-1.1.0-dp5-macosx-x86_64.tar.gz
(5) Copied couchbase.so to the extensions folder
sudo cp php-ext-couchbase/couchbase.so /usr/lib/php/extensions/no-debug-non-zts-20090626(6) Restarted Apache
sudo apachectl restart
A simple script like
$cb = new Couchbase('127.0.0.1:8091', 'Administrator', 'mypassword', 'beer-sample', false); $cb->set("a", 1);
gives me
[Mon Dec 10 22:44:52 2012] [notice] child pid 40575 exit signal Segmentation fault (11)
P.S. The forum does not help me in posting code without making html code for links on my behalf.
Hi,
can you verify with the stable release out now: http://www.couchbase.com/develop/php/current and the corresponding new version of libcouchbase (http://www.couchbase.com/develop/c/current) - 2.0.1
Thanks,
Michael
Thank you for the quick reply.
I will try the new suggested libraries and come back to you with the results.
I have followed your instructions and used the new libraries, using the steps described below. I confirm everything is working ok now. Thanks!
➜ brew unlink libcouchbase Unlinking /usr/local/Cellar/libcouchbase/2.0.0... 23 links removed ➜ brew install <a href="https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouchbase.rb<br /> ########################################################################" title="https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouchbase.rb<br /> ########################################################################">https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouc...</a> 100.0% ==> Downloading <a href="http://packages.couchbase.com/clients/c/libcouchbase-2.0.1.tar.gz<br /> ########################################################################" title="http://packages.couchbase.com/clients/c/libcouchbase-2.0.1.tar.gz<br /> ########################################################################">http://packages.couchbase.com/clients/c/libcouchbase-2.0.1.tar.gz<br /> ######...</a> 100.0% ==> ./configure --prefix=/usr/local/Cellar/libcouchbase/2.0.1 --disable-examples --disable-couchbasemock ==> make install /usr/local/Cellar/libcouchbase/2.0.1: 39 files, 556K, built in 20 seconds ➜ wget <a href="http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz<br /> --2012-12-12" title="http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz<br /> --2012-12-12">http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-macosx...</a> 12:56:28-- <a href="http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz<br /> Resolving" title="http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz<br /> Resolving">http://packages.couchbase.com/clients/php/php-ext-couchbase-1.1.0-macosx...</a> packages.couchbase.com... 205.251.242.196 Connecting to packages.couchbase.com|205.251.242.196|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 59758 (58K) [application/x-tar] Saving to: ‘php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz’ 100%[==============================================================================================>] 59,758 55.6KB/s in 1.0s 2012-12-12 12:56:30 (55.6 KB/s) - ‘php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz’ saved [59758/59758] ➜ tar xzf php-ext-couchbase-1.1.0-macosx-x86_64.tar.gz ➜ cp php-ext-couchbase/couchbase.so /usr/lib/php/extensions/no-debug-non-zts-20090626 ➜ sudo apachectl restart
Hello David,
well I was going to post the exact same question. Using non-persistent connections makes the application run slower (about 15 ms) because of the time needed by the client to get the cluster topology.
I tried to start a persistent connection :
$cb = new Couchbase("127.0.0.1:8091", '', '', '', true); $cb->set("a", 1); echo $cb->get('a') . '<br>';and, on the second call, I get a suhosin error message (and Apache resets the connection) with "ALERT - canary mismatch on efree() - heap overflow detected....."
I don't think the php extension handles correctly persistent connections :(
I don't deploy Couchbase because of this reason... Still waiting for some update, let's keep this thread active :)