http://www.couchbase.com/issues This file is an XML representation of an issue en-us 5.2.4 845 26-12-2012 http://www.couchbase.com/issues/browse/PCBC-141 Couchbase PHP client library This affects all php binaries which do not have the symbol &quot;php_json_encode&quot;. This includes EL5 and EL6 based linux distributions <br/> <br/> The library cannot load because the symbol is not found. Disabling using this symbol also means preventing views from functioning (as well as JSON serialization). <br/> <br/> I&#39;d like to note that this is not our bug and not our fault. We are probably not the only extension relying on php_json* functions, and package creators (specifically redhat) should expose this symbol. <br/> <br/> Additionally, our configure script checks to see if the PHP_JSON_* constants are defined (and if not, compiles out the relevant code from views - this would make views not work, but not prevent the library from loading). <br/> <br/> In any event, we should find a way to work around this, as this bug has been seen quite a bit. <br/> <br/> One possible solution would be to call the php-level json encoding function. It may incur a bit of overhead from calling into php, but the function itself is more likely to be there. PCBC-141

1.1 dp releases not working on phps which do not export the symbol php_json_encode

Bug Blocker Resolved Duplicate Public Matt Ingenthron Mark Nunberg Wed, 7 Nov 2012 17:55:57 -0600 Fri, 10 May 2013 02:43:07 -0500 Fri, 10 May 2013 02:43:07 -0500 1.1.0-dp5 1.1.0 library 0 2 Currently the only workaround is to compile your own php One other solution that I&#39;d be okay with is to simply uplevel our minimum version of CentOS/RHEL. Unless there&#39;s significant pushback, I really don&#39;t see any reason to keep people living with ancient code for longer and longer periods of time. <br/> <br/> Dissenting opinions accepted, but at some point you have to say &quot;we don&#39;t support that because it&#39;s broken&quot;, right? <br/> <br/> Do you think it&#39;s likely that some minimum 5.x EL is fixed enough? Or there&#39;s some patchlevel that is fixed enough? <br/> <br/> If on the other hand, it&#39;s just conscious decision to not expose that symbol and it&#39;s only supported through PHP land, then calling up there as a workaround seems okay by me. Assigning this to the subject matter expert for now. Will hopefully pick it back up soon. One other question, I don&#39;t think so, but is this possibly related to need to put the ini for loading the extension after the php_json as an extension? <br/> <br/> Another possible workaround is that if the license is permissive enough, we could just suck in the parts we need? Possible API/ABI issues here I guess, but hard to say. Duh, I see you proactively addressed the &quot;which release&quot; in the original description. Thanks for that. I&#39;ve explored that possibility. The problem is that on rhel and centos don&#39;t export that symbol. EPEL actually contains a json.so (php-pecl-json), but much to our my chagrin, it does not export that symbol either. <br/> <br/> I was thinking about sticking the significant bits inside our own code.. it&#39;d probably be easier to just call the php level function though.. <br/> <br/> This does seem to be a conscious decision by redhat - as this happens in CentOS 6.3 (EL6) as well. So this is obviously not fixed anywhere. <a href="http://review.couchbase.org/#/c/22425/">http://review.couchbase.org/#/c/22425/</a> The problem is a bit more intricate it seems :) <br/> <br/> Apparently some of the json.so modules I&#39;ve seen *do* export the php_json_* functions. HOWEVER, apparently php loads these modules with a line similar to dlopen(mod, RTLD_LOCAL..) making their symbols unavailable for other modules to use. <br/> <br/> A more complex workaround: <br/> <br/> static void (*_json_encode)(smart_str*, zval*, int TSRMLS_DC) = NULL; <br/> static void (*_json_decode)(zval*, char*, int, zend_bool, long TSRMLS_DC) = NULL; <br/> <br/> static void _init_json_symbols(void) <br/> { <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;zend_module_entry *m_ent; <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;int ret; <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if (_json_encode) { <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return; <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ret = zend_hash_find(&amp;module_registry, &quot;json&quot;, sizeof(&quot;json&quot;), (void**)&amp;m_ent); <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if (ret == FAILURE) { <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fprintf(stderr, &quot;Couldn&#39;t load extension..\n&quot;); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;abort(); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_json_encode = DL_FETCH_SYMBOL(m_ent-&gt;handle, &quot;php_json_encode&quot;); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_json_decode = DL_FETCH_SYMBOL(m_ent-&gt;handle, &quot;php_json_decode&quot;); <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if (_json_encode == NULL || _json_decode == NULL) { <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fprintf(stderr, &quot;Coudln&#39;t find JSON handles!: %s\n&quot;, dlerror()); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;abort(); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} <br/> } <br/> <br/> void php_json_encode(smart_str *buf, zval *value, int options TSRMLS_DC) <br/> { <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_init_json_symbols(); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_json_encode(buf, value, options TSRMLS_CC); <br/> } <br/> <br/> void php_json_decode(zval *out, char *buf, int len, zend_bool assoc, long depth TSRMLS_DC) <br/> { <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_init_json_symbols(); <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_json_decode(out, buf, len, assoc, depth TSRMLS_CC); <br/> } <br/> Very interesting. Why is this done only on RHEL/CentOS? We should check with some PHP internals folks before we go too far here. Maybe there&#39;s some subtlety we don&#39;t understand. hi! <br/> <br/> Not sure what RHEL5/6 does but PHP 5.3.2 has these APIs exported since the very 1st day: <br/> <br/> <a href="https://github.com/php/php-src/blob/php-5.3.2/ext/json/php_json.h">https://github.com/php/php-src/blob/php-5.3.2/ext/json/php_json.h</a> <br/> <br/> Also json has always been a default extension in PHP, again something wrong at RHEL. <br/> <br/> I would suggest to add a configure check to see if it is exposed and refused to compile if it is missing. To me it is a bug in RHEL (not the 1st weird one :) and would not begin to duplicate code around (especially not for json, to ensure 100% compatibility and get all fixes in time). The real problem is in the first line I mentioned; and it might be a change in the arguments of dlopen. If the json extension is truly loaded before the couchbase one, then the couchbase ext *should* find the appropriate symbols to use. <br/> <br/> However it seems that for RHEL (I haven&#39;t checked on debian.. debian builds the json ext into the php executable itself, rhel provides it as a loadable module) even though the json extension is already loaded, because there is something weird with either the way it uses dlopen (default php header macro for DL_LOAD uses RTLD_GLOBAL, i.e. make the symbols global to the entire app.. but it might be that RHEL uses RTLD_LOCAL...). <br/> <br/> But we&#39;re not duplicating code here and I personally like the call-to-php solution (even though it&#39;s not the most elegant, it&#39;s the most predictable/reliable. For example, I don&#39;t know how the snippet in my last post would work on a php that doesn&#39;t have json as a DSO) So it seems we&#39;re (or is it only me) looking at it from multiply wrong ways :) <br/> <br/> (1) RHEL (and debian) modify the default php DL_LOAD from using RTLD_LAZY (which would have avoided giving us this error) to RTLD_NOW <br/> <br/> (2) On Debian this is not a problem (at least not for JSON) since it&#39;s compiled into the php binary <br/> <br/> (3) Even on RHEL, placing extension=json.so before extension=couchbase.so seems to do the trick. This is mentioned in the documentation <br/> <br/> (4) For tests, run-tests.php will not work. Unless a specifically crafted ini file is written, run-tests.php will not load the json extension at all. Defining an extra -d parameter will load the json extension; however the script counter-intuitively *appends* this option to the commandline, so one thus effectively has: <br/> <br/> -d extension=couchbase.so -d extension=json.so <br/> <br/> Coupled with RTLD_NOW, this fails. <br/> <br/> I&#39;d still like to fix this in code; as this is a fairly confusing matter; if only to be able to print out a more meaningful user message. 1) Not sure that should cause this error, as long as the loading order is respected. <br/> <br/> 2) Debian does it right. JSON is a default builtin extension and should always be available, not optionally. <br/> <br/> 3) Yes, loading order, that should be documented as such in the couchbase install documentation <br/> <br/> 4) yes, it is expected, also using a php.ini is always a good thing, to avoid tests failure due to random php.ini being caught <br/> <br/> There is nothing to fix in couchbase, calling user land function is horribly slow and should really not be done for such thing. The fix here would be to make users understand this a bit better. <br/> <br/> The point isn&#39;t that something is broken, but rather that something is confusing. This issue is hinted in the couchbase ext documentation, but seems to be passed over by quite a few people who have run into this. <br/> <br/> This seems like a very subtle configuration aspect which users shouldn&#39;t have to encounter; or at least something in which we&#39;d be able to show the user an error message. <br/> <br/> So without going into a debate of terminology about what&#39;s &quot;broken&quot; and whose &quot;fault&quot; it is, this is something that lots of users are seeing, and shouldn&#39;t. Such fine minutiae really isn&#39;t easy to pick up. <br/> <br/> Another option would have been to load the symbols dynamically from either the binary or from the loaded library (depending on the configuration) - however &#39;php_json_decode&#39; is now no longer an exported symbol (it&#39;s defined in the header as an inline wrapper around php_json_decode_ex) in some versions. ext dependencies are a very common issue in php, this is well documented. I won&#39;t change anything in the code (causing more arms) and clearly document that the json ext, for old broken (it is broken, as it must be builtin and always enabled) distribution packages. Thanks for all the help here Pierre. <br/> <br/> According to Mark, this doesn&#39;t affect only old, broken distribution packages. It&#39;s even in recent RHEL6*. <br/> <br/> Just to fix this with the simplest, most supportable solution perhaps we should test the function at startup time and if it&#39;s not there log and exit appropriately? If the distro is loading that lazily, and people want to use that distro, then they need to configure their PHP .ini&#39;s correctly, right? <br/> <br/> Our log message can be explicit even, saying &quot;make sure json whatever is loaded first with your php.ini&quot;. <br/> <br/> Thoughts? <br/> <br/> * which maybe is arguably old, broken, but it&#39;s also arguably not since it&#39;s the most recent thing shipped on this particular distro fork Pierre: one other question-- any &#39;prior art&#39; here? In other words, surely some other extension must rely on json and has an approach? Unfortunately there is no way to configure RHEL&#39;s php to load things lazily. <br/> <br/> RHEL Specifically mangles this. <br/> <br/> I know the patch says 5.0.4, but this is still in php-5.3.3-14.el6_3.src.rpm: <br/> <br/> --- php-5.0.4/Zend/zend.h.dlopen <br/> +++ php-5.0.4/Zend/zend.h <br/> @@ -102,11 +102,11 @@ <br/> &nbsp;# endif <br/> <br/> &nbsp;# if defined(RTLD_GROUP) &amp;&amp; defined(RTLD_WORLD) &amp;&amp; defined(RTLD_PARENT) <br/> -# define DL_LOAD(libname) dlopen(libname, RTLD_LAZY | RTLD_GLOBAL | RTLD_GROUP | RTLD_WORLD | RTLD_PARENT) <br/> +# define DL_LOAD(libname) dlopen(libname, RTLD_NOW | RTLD_GLOBAL | RTLD_GROUP | RTLD_WORLD | RTLD_PARENT) <br/> &nbsp;# elif defined(RTLD_DEEPBIND) <br/> -# define DL_LOAD(libname) dlopen(libname, RTLD_LAZY | RTLD_GLOBAL | RTLD_DEEPBIND) <br/> +# define DL_LOAD(libname) dlopen(libname, RTLD_NOW | RTLD_GLOBAL | RTLD_DEEPBIND) <br/> &nbsp;# else <br/> -# define DL_LOAD(libname) dlopen(libname, RTLD_LAZY | RTLD_GLOBAL) <br/> +# define DL_LOAD(libname) dlopen(libname, RTLD_NOW | RTLD_GLOBAL) <br/> &nbsp;# endif <br/> &nbsp;# define DL_UNLOAD dlclose <br/> &nbsp;# if defined(DLSYM_NEEDS_UNDERSCORE) <br/> ~ <br/> <br/> <br/> So basically by the time our library loads, if it utilizes a bare &#39;json_decode&#39; that reference is resolved immediately; our module has no chance to warn. Given there is no good solution, for now we&#39;ll need to document this one very, very well. I don&#39;t do the release notes for individual SDK libraries. This isn&#39;t a release note item, this is a PHP documentation item. This needs to be added to the getting started guide with a really clear description of what the issue is and how to work around it for RHEL/CentOS. <br/> <br/> I think you&#39;re still able to help us with PHP documentation, right? I see. Do you want this to go as a note during the platform-specific install section under RHEL/Centos? I&#39;ll follow the docs team&#39;s guidance on how best to present the info. <br/> <br/> Just be aware that it&#39;s something we&#39;ve frequently hit and nearly every CentOS/RHEL user will hit it too. It&#39;s not our bug really that we can&#39;t make it simpler. It&#39;s a limitation in current PHP extension loading. <br/> <br/> That&#39;s why I think it needs to be covered in the appropriate section of the getting started guide, which is both on the web site (<a href="http://www.couchbase.com/develop/php/next">http://www.couchbase.com/develop/php/next</a>) and in our documentation (<a href="http://www.couchbase.com/docs/couchbase-sdk-php-1.1/download.html">http://www.couchbase.com/docs/couchbase-sdk-php-1.1/download.html</a> and <a href="http://www.couchbase.com/docs/couchbase-sdk-php-1.1/installation-verification.html)">http://www.couchbase.com/docs/couchbase-sdk-php-1.1/installation-verification.html)</a>. Most people will hit it when following either of those. <br/> <br/> Our web pages currently have the following note: <br/> Note: With the PHP packages on many Red Hat/CentOS distributions (and possibly others), PHP&#39;s JSON encoding is not available to other extensions by default. As a result, you may see an error resolving the php_json_encode symbol. The solution is to edit ini file that loads the JSON extension (typically /etc/php.d/json.ini) to add the Couchbase extension after the JSON extension. <br/> <br/> Something along these lines (but improved, if you think need be) should be added to the getting started guide in the documentation. Note, Pierre replied via email to my &#39;prior art&#39; question: <br/> <br/> No, ext dep manager is a long due todo but much easier to document than to implement. <br/> <br/> Other core exts have this, exif and mbstring, pdo exts and the sin pdo ext (which should be builtin but rhel and defiant made it wrong 1st) Hi, <br/> <br/> I&#39;m adding this. Do we have an example of the php_json_encode error that you will get? Need to add this to the section. Added to getting started/install as: <br/> <br/> &quot;If you are using the PHP SDK on a Linux distribution such as Red Hat/CentOS, be aware that JSON encoding for PHP is by default not available to other extensions. As a result you will receive an error resolving the php_json_encode symbol. The solution is to edit the .ini file that loads the JSON extension to add the Couchbase extension after the JSON extension. For instance, if your JSON extension is at /etc/php.d/json.ini, add the following line to the file under extensions: <br/> <br/> extension=/path/to/couchbase.so&quot; Added to getting started and install for PHP 1.1:<br/> <br/> If you are using the PHP SDK on a Linux distribution such as Red Hat/CentOS, be aware that JSON encoding for PHP is by default not available to other extensions. As a result you will receive an error resolving the php_json_encode symbol. The solution is to edit the .ini file that loads the JSON extension to add the Couchbase extension after the JSON extension. For instance, if your JSON extension is at /etc/php.d/json.ini, add the following line to the file under extensions:<br/> <br/> extension=/path/to/couchbase.so Karen: per the email thread with Mark Nunberg the other day, can we update this to have the two line recommendation? Perhaps that&#39;s what you&#39;ve already done? Oh yes, I did update per Mark&#39;s email several days ago. It is now:<br/> <br/> Depending on the platform you are using, you may also need to reference the JSON library in your PHP configuration file.<br/> <br/> If you are using the Couchbase PHP SDK on Red Hat/CentOS or their derivatives, be aware that JSON encoding for PHP is by default not available to other extensions. As a result you will receive an error resolving the php_json_encode symbol. The solution is to edit the php.ini file to load the JSON library and also load the Couchbase library. For instance, if your extensions are at /etc/php.ini, add the following two lines to the file:<br/> <br/> extension=/path/to/json.so<br/> extension=/path/to/couchbase.so<br/> The reference to the two extensions must be in this specific order.<br/> <br/> <br/> You can see it here:<br/> <br/> <a href="http://www.couchbase.com/docs/couchbase-sdk-php-1.1/download.html">http://www.couchbase.com/docs/couchbase-sdk-php-1.1/download.html</a> Oh yes, I did update per Mark&#39;s email several days ago. It is now:<br/> <br/> Depending on the platform you are using, you may also need to reference the JSON library in your PHP configuration file.<br/> <br/> If you are using the Couchbase PHP SDK on Red Hat/CentOS or their derivatives, be aware that JSON encoding for PHP is by default not available to other extensions. As a result you will receive an error resolving the php_json_encode symbol. The solution is to edit the php.ini file to load the JSON library and also load the Couchbase library. For instance, if your extensions are at /etc/php.ini, add the following two lines to the file:<br/> <br/> extension=/path/to/json.so<br/> extension=/path/to/couchbase.so<br/> The reference to the two extensions must be in this specific order.<br/> <br/> <br/> You can see it here:<br/> <br/> <a href="http://www.couchbase.com/docs/couchbase-sdk-php-1.1/download.html">http://www.couchbase.com/docs/couchbase-sdk-php-1.1/download.html</a> Use version 1.1.1 of client lib and still get this error even when couchbase.so loaded after json.so <br/> <br/> CentOS 5.6, PHP 5.2.17, Couchbase 1.8 Hi Matt,<br/> <br/> This was reported as an technical issue still with PHP SDK:<br/> <br/> &nbsp;&nbsp;&nbsp;[ <a href="http://www.couchbase.com/issues/browse/PCBC-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=46067#comment-46067">http://www.couchbase.com/issues/browse/PCBC-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=46067#comment-46067</a> ]<br/> <br/> Andrey Nikishaev commented on <a href="http://www.couchbase.com/issues/browse/PCBC-141" title="1.1 dp releases not working on phps which do not export the symbol php_json_encode"><strike>PCBC-141</strike></a>:<br/> ---------------------------------------<br/> <br/> Use version 1.1.1 of client lib and still get this error even when couchbase.so loaded after json.so what&#39;s the exact error you&#39;re getting? <br/> <br/> Maybe the json module needs to be installed as well? (I&#39;ll need to check this) -- <br/> Andrey: We do not support PHP 5.2, so you&#39;ll want to try 5.3 or later. CentOS 5.6 does have, if I recall correctly, a &quot;php53&quot; package. There are multiple bugs reported for this issue Rank 9798 Time In Status