Couchbase

[CCBC-193] libcouchbase build is not checking install properly

Fri, 22 Mar 2013 19:56:36 -0500

I was trying to package the PHP client, which depends on libcouchase, and it is was refusing to build on some systems.

After looking into it a bit, there may be a couple of problems, but the big one is in the summary.

One problem is that it seems the dpkg -i doesn't actually get run. I added an echo of the label in there, and see that in the console output in Jenkins, but I don't see the lines under the label run. I'm stumped there.

[CCBC-208] libvbucket CRC32 hashing function returns wrong hashes

Thu, 9 May 2013 14:02:28 -0500

It seems that the CRC32 hashing function in libvbucket (hash_crc32()) returns an invalid hash.

For instance, hashing the key '1234567890' returns 9757 (0x261D), while it should return 639479525 (0x261DAEE5).

Every key I tried (on both win and linux) returns a 16bit hash.

The problem seems to be on line 85 in crc32.c.

Changing the return expression to 'return crc ^ ~0;' fixes the problem and computes the correct hash.

[CCBC-184] There are a lot of connections are established after restarting Couchbase server

Mon, 25 Feb 2013 01:08:00 -0600

1) Start Couchbase Server
2) Start to recieve some key from Couchbase server using a client based on libcouchbase library. We use about 250 threads for receiving key
3) Restart Couchbase server
4) After that a lot of connections are created (all of then in ESTABLISHED state). And soon quota for handlers is over.

[CCBC-162] API reference uses old names, not current lcb_* names

Thu, 31 Jan 2013 03:11:36 -0600

The API reference for all functions uses libcouchbase_* names (libcouchbase_create, etc.) instead of the correct lcb_* names (lcb_create, etc.).

[CCBC-209] Make Windows builds work with jom (via CMake)

Fri, 7 Jun 2013 09:40:55 -0500

Jom supports parallel-make for windows' nmake (the equivalent to normal make's '-j' option). Right now compiling on windows is painfully slow.

Another alternative would be to make a solution file.

http://qt-project.org/wiki/jom

[CCBC-200] lcb_make_http_request doesn't respect timeouts

Fri, 5 Apr 2013 06:17:51 -0500

Run the attached testcase on Windows (after fixing the host and such). The lcb_wait after calling lcb_make_http_request will hang for a very long time (perhaps forever) if the server is no longer accessible.

I tested this by starting the client. When it prints "sleeping" I pull the ethernet cable on my machine. If the lcb_make_http_request is replaced by lcb_get then you'll get a timeout which is expected.

[CCBC-199] Problem with a newly created memcached bucket

Tue, 2 Apr 2013 06:28:35 -0500

See: http://www.couchbase.com/forums/thread/libcouchbase-problem-newly-created-memcached-bucket

[CCBC-195] config cache will crash client due to relocating buffers

Tue, 2 Apr 2013 03:30:20 -0500

See ~~CCBC-190~~

During a cluster configuration change, the buffers are relocated while they are being processed. This causes crashes.

A solution would be to "request" a configuration refresh and have that take place once it is safe to move buffers around (i.e. once we are out of the buffer processing call chain).

[CCBC-194] Please add support for setsockopt() SO_KEEPALIVE & TCP_KEEPIDLE

Mon, 25 Mar 2013 12:25:05 -0500

Please add support for setsockopt() options SO_KEEPALIVE & TCP_KEEPIDLE through (suggestion)
lcb_connect() or some other such api so that I can control the connection's ability to not timeout
and have a firewall cause a disconnection from lack of inactivity...without such control, receive
NETWORK_ERROR from couchbase apis.

I cannot adjust the firewall, so I need to keep the connection alive.

[CCBC-187] Set TCP_NODELAY flag

Mon, 25 Feb 2013 01:14:48 -0600

We think it is good idea to set TCP_NODELAY flag to switch off the Nagle mode
(don't wait for sending a short packets)
Couchbase server, as we think, use this flag

[CCBC-186] Request to provide concrete examples in the documentation for OBSERVE functionality

Thu, 21 Feb 2013 15:17:58 -0600

http://support.couchbase.com/tickets/2608

Customer are interested in using OBSERVE functionality to allow them to check whether items have been replicated successfully, as they are worried about unreplicated items.

https://github.com/couchbase/libcouchbase/blob/master/man/man3couchbase/lcb_observe.3couchbase.txt
has basic info on the OBSERE functionality in libcouchbase, but we need to provide concrete examples on how to check multiple keys having been replicated successfully (they do a bulk write, so should be more efficient to wait for all of them, rather than wait for them individually)

Examples provided to the customer :

You will find the below example very useful for simple operations :

https://github.com/couchbase/php-ext-couchbase/blob/master/simple_observe.c

The below example would provide more insights when you have any performance or reliability issues :

https://github.com/couchbase/php-ext-couchbase/blob/master/observe.c

The basic idea here is to constantly "poll" the keys (sleeping within a given interval) until all of them are declared as "OK". The structures involved are the "observe_expectation" (persistence/replication requirements), the "pollprefs" (i.e. timeout, interval, etc.) and the "keystate" - this keeps tabs on how the actual status of each key matches up with the specified criteria. The lcb_observe callback is called for each "status report" for each key from each server; so it is necessary to check and see if the requirements have been met during each individual update.

[CCBC-183] Implement approach to specify behaviour for lcb_get_replica()

Fri, 15 Feb 2013 12:10:15 -0600

Implement approach to specify behaviour for lcb_get_replica()

[CCBC-177] Docs: Reference installation

Mon, 11 Feb 2013 05:36:17 -0600

Customers have lots of choices at their fingertips, and many of those choices will result in a non-functioning installation, or at least a lot of time spent figuring out what the right pieces are.

Can we please provide as much details as possible about a reference installation that we can be sure will work for a customer who is just getting started? This is outside of our support for any one operatin system, but more about telling the customer "if you install with these versions and packages, this code will work"

-OS
-other packages/modules required and/or tested with

[CCBC-174] Error handling documentation

Tue, 5 Feb 2013 02:39:18 -0600

Please create some documentation specifying possible error/failures to operations, what they "look" like in the logs/exceptions/stack traces and what our recommendation is on how to handle them.

i.e. tmp_oom, timeouts (connection/operation/java-internal/etc), "get miss" (it's technically a failure, let's make it overly obvious what it means), CAS failure, add() failure, replace() failure,

Some of this should be covered in the API reference, but this bug is specifically for a single page where this information is aggregated that a customer/user could read about how to handle errors.

[CCBC-170] After upgrading to 2.0.2 with CBC163, I now get timesouts again!!!

Wed, 6 Feb 2013 15:43:10 -0600

I was using 2.0.0 beta3 with the patch for TIMEOUTS manually applied....this was working fine but I had high CPU. This ran for weeks (if not a month) without any
timeouts at all...I was the happiest with lcb as I had been in a long time. However, after getting some complaints from my server admins and NOC, I upgraded to
2.0.2 with the fix for high CPU...

I rebuilt all modules and rolled to production. It's now been about 4 hours and I have received 2 timeouts as to where I had NONE for weeks without this change.

The load is now normal though but somehow something has broken because of this fix.

I looked at the change with CBC-163, but I don't see how this could have broken the code...but, it has - I promise. Can someone please take a look?

HELP!

[CCBC-152] Provide lcb_check to (weakly) check for the existence of a key

Fri, 21 Dec 2012 13:27:18 -0600

This API will be a variant of OBSERVE and shall do a simple probe against the master node to see if the key exists in cache.

We need to have this in libcouchbase as normal observe is implemented by 'broadcasting' to all the nodes

[CCBC-135] lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed

Fri, 14 Dec 2012 13:04:43 -0600

Cannot reproduce, this is all the info I have

FAIL GetMulti - MgetOrdered [tests/phpt/GetMulti/MgetOrdered.phpt]

mnunberg@csure:/sources/lcb2.0-repo/php$ cat tests/phpt/GetMulti/*.out
PHP_COUCHBASE_OK
php: src/server.c:799: lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed.

*Might* be related to ~~CCBC-120~~

To reiterate again, I cannot reproduce this - I just happened to notice it

[CCBC-111] Make iops functions 'private', and discourage calls to run_event_loop/stop_event_loop

Mon, 10 Dec 2012 07:47:33 -0600

There's quite a bit of user code which calls io->run_event_loop and io->stop_event_loop.

These functions are largely private and are intended for use by libcouchbase itself. The fact that they are public API is only accidental, due to the fact that they are part of the iops structure.

These functions should never be called by user code:

1) They are a source of confusion (especially when mixed with lcb_wait)
2) They prevent any kind of wrapping/sanity logic done by lcb_wait (some already exists, and some can be implemented in future versions)

The following should be done

1) Remove the iops structure definition from being included by the main header file. Thus, #include <libcouchbase/couchbase.h> should not allow a dereference of the iops structure. Rather the iops structure should be defined in a new header file (iops.h) - so that users who want to explicitly manipulate and write event loop plugins can include it, but should otherwise not be present in 'user' code. types.h should include a predeclaration of the structure so that it may still be passed around as a simple pointer value.

2) Document the purpose of run_event_loop/stop_event_loop. Make it very clear that lcb_wait should be used in all synchronous use cases. If we have any examples using run_event_loop/stop_event_loop, remove them.

3) [ Optional? ] - In order to further discourage users from using these functions, throw a compiler warning/error for those functions unless LIBCOUCHBASE_INTERNAL is defined - tbd

[CCBC-110] Implement lcb_reconnect() which will reconnect all dead sockets

Tue, 20 Nov 2012 14:40:01 -0600

This functionality very important for long-living connection handles

[CCBC-108] Allow per-command retries for not-my-vbucket and other transient errors

Mon, 10 Dec 2012 07:47:32 -0600

We should avoid the possibility of 'infinite' errors and allow the user the ability of seeing the 'real' error which caused the command to fail..

http://review.couchbase.org/13177

[CCBC-127] libvbucket RPM is missing /usr/lib64/libvbucket.la file

Fri, 16 Nov 2012 17:51:51 -0600

The libvbucket1 rpm does not include the libvbucket.la file. The libcouchbase.la file references libvbucket.la as a dependency.

Linking via libtool against libcouchbase fails because of this. For example, compiling the php-ext-couchbase package fails like:

/bin/sh /root/code/php-ext-couchbase/libtool --mode=link cc -DPHP_ATOM_INC -I/root/code/php-ext-couchbase/include -I/root/code/php-ext-couchbase/main -I/root/code/php-ext-couchbase -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -I/usr/include/ -DHAVE_CONFIG_H -g -O2 -o couchbase.la -export-dynamic -avoid-version -prefer-pic -module -rpath /root/code/php-ext-couchbase/modules compress.lo couchbase.lo fastlz/fastlz.lo -lz -lcouchbase
grep: /usr/lib64/libvbucket.la: No such file or directory
/bin/sed: can't read /usr/lib64/libvbucket.la: No such file or directory
libtool: link: `/usr/lib64/libvbucket.la' is not a valid libtool archive
make: *** [couchbase.la] Error 1

I created a fake libvbucket.la by copying /usr/lib64/libcouchbase.la and editing it, and it allowed the compile to succeed.

[CCBC-116] Heavy errors during short network loss

Wed, 14 Nov 2012 11:55:10 -0600

During network loss to the nodes, there are heavy network errors - a combination of both timed-out operations as well as operations which fail with a network error. Might we want to provide an option to retry these operations?

[CCBC-83] Update the getting started to match the Beer sample DB

Thu, 18 Oct 2012 11:53:21 -0500

Once the sample DB has been integrated into the server, update the getting started (both on the web page and in the documentation) to use the sample database.

[CCBC-84] Updated screencast for /develop pages

Thu, 12 Jul 2012 00:56:01 -0500

Create an updated screencast to ship with the new 2.0 developer SDKs

[CCBC-49] API reference is complete

Mon, 10 Dec 2012 07:47:32 -0600

API reference is complete

[CCBC-205] Provide stricter semantics/behavior for configuration cache.

Wed, 1 May 2013 16:43:06 -0500

We may want to create a 'cache-only' mode in which creation of the client fails if the cache cannot be created (e.g. a bad path was passed, or the file is not writeable, etc.)

Currently, the cache is mostly a transparent extra, but there is no way for a user to determine whether the mechanism is at all working.

[CCBC-207] Failed connection attempts will hang until timeout interval is reached on win32

Wed, 8 May 2013 15:55:16 -0500

Failed connection errors are not detected in the select() call. This is because the select call does not add the socket to the exception fdset.

MSDN documentation states that failed connections are delivered through exceptfds, and that only successful connections are delivered as writability notifications. See http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141(v=vs.85).aspx

[CCBC-198] Instance should close REST connection upon receiving valid config cache

Sat, 11 May 2013 13:28:28 -0500

If a valid config cache configuration is received, there is no point in keeping the REST socket open; (one of the reasons someone would even want to use a cached config is in order to reduce load from the server - so we should try to minimize connections in such cases).

Close the socket immediately (or eventually) if a valid configuration is obtained. Re-open as necessary.

[CCBC-137] libcouchbase test improvements

Sat, 11 May 2013 13:31:12 -0500

Generic improvements and fixes for libcouchbase test suite

[CCBC-139] add timeout tests for libcouchbase

Sat, 8 Dec 2012 00:11:17 -0600

add timeout tests for libcouchbase

[CCBC-142] test libcouchbase observe functionality

Sat, 8 Dec 2012 00:11:17 -0600

test libcouchbase observe functionality

[CCBC-214] Re-design internal operations in a queue/linked list structure

Dependency — Tue, 21 May 2013 11:46:07 -0500

Re-design internal operations in a queue/linked list structure

[CCBC-213] Unify internal connect() routines

Dependency — Tue, 21 May 2013 23:12:50 -0500

This will unify the connection routines in terms of error handling and callback invocations.

[CCBC-215] Test whether server gets reconnected when user idle for longer than timeout interval

Wed, 22 May 2013 00:47:27 -0500

see inline discussion here: http://review.couchbase.org/#/c/26462/2/src/timeout.c

Basically, the semantics of the timeouts aren't clear - the timeout will unconditionally purge the server and disconnect if the timeout hits, even if there is nothing waiting.

In practice this might not be a bug, but it means needless connection code to the server (and SASL auth) is being executed each time.

[CCBC-211] libcouchbase may touch a failed node after failover

Tue, 21 May 2013 10:24:35 -0500

Library version: v2.0.6 (compiled from the source code, on x64)
Server version: v2.0.1
Number of nodes in my test setup: 3 (2 x CentOS 6.2 x64, and 1 x CentOS 5.8 x86, if it's of any relevance)

Steps to reproduce:

1. Create a new cluster
2. Print the returned value by lcb_get_server_list function (there shoud be 3 servers printed)
3. Bring down network on one of servers
4. Try to put several values, until one of them is routed to the failed node. The library should return an error, which is expected.
5. Repeat 2
6. Manually fail over the failed node.
7. Print the returned value by lcb_get_server_list function (there should be 2 servers printed)
8. Try to put several values, until one of them is routed to the failed node. This behaviour is not expected.
9. Start rebalance.
10. While rebalancing takes place, a request might be routed to the failed node. It is not expected as well.
11. Once rebalance is completed, no requests are routed to the failed node. It is as expected.

Thanks in advance for your support.

[CCBC-216] Change 'lcb_wait' semantics

Dependency — Wed, 22 May 2013 10:53:51 -0500

Currently we have 'wait' and 'breakout' which is optimized and termed for a synchronous model.

We should rename these functions (though we can still maintain aliases) to something like:

lcb_operations_flush:

Which indicates that the user has finished sending requests, and that packets should be sent

And:
lcb_operations_done_callback

Which will be called when the operations have been completed.

Effectively this is the same as lcb_wait and the stop_event_loop callback, but the usage and purpose of those two existing functions aren't clear.

[CCBC-212] IO Enhancements

Dependency — Wed, 22 May 2013 10:53:51 -0500

This super bug is an umbrella task for refactoring our IO system to be more streamlined, readable, and compatible with other I/O models.

[CCBC-129] Add libevent support for win32

Wed, 21 Nov 2012 12:53:20 -0600

I created a NMakefile and made changes to the code to support libevent on windows.

Files affected:
iofactory.c
plugin-libevent.c

New file:
NMakefile.libevent

I uploaded a zipfile of the changes if you are interested in merging them.

[CCBC-101] allow google test suite to be preserved between invocations of make distclean and friends

Tue, 18 Sep 2012 11:35:28 -0500

additionally we should make it into a DSO and link the tests against that.. would make it somewhat quicker..

This is obviously not a major issue.. but would be nice :)

[CCBC-78] Improved Sample Code for "Simple Example"

Thu, 5 Jul 2012 16:25:15 -0500

Hi,

I received a variety of content corrections from a customer for the C SDK. I made all possible, however some require changes in the sample code the SDK Team provides. The existing sample code is here:

http://www.couchbase.com/docs/couchbase-sdk-c-1.0/simple_example.html

The customer requests:

- The tutorial example bails from callbacks with exit(), rather than
passing control back to main (after setting a global flag, for example,
or one passed in via the cookie, etc.).
- The tutorial should also show use of libcouchbase_remove() and what to
expect from libcouchbase_mget() afterwards (the latter is in the Python
docs, but not the C docs).
- The tutorial should use libcouchbase_destroy().
- Some mention of whether a disconnect is needed or not would be
appropriate.

Let me know if you are able to demonstrate use of these other methods in the sample code and send the code. I will add the new sample and document.

Also let me know if a C client needs to be destroyed and if so how.

[CCBC-161] libcouchbase_flush wrongly documented to "flush entire cluster"

Thu, 2 May 2013 16:32:40 -0500

The docs say that libcouchbase_flush() will "Flush the entire cluster" and "libcouchbase_flush may be used to remove all key/value pairs from the entire cluster."

It actually flushes a bucket, not the entire cluster.

[CCBC-206] Improve libcouchbase rpm instructions

Thu, 2 May 2013 16:35:16 -0500

There is currently no documentation about installing libcouchbase from the rpm or deb files.
The only documentation is for installation with yum or apt-get.

Since some packages are optional, more instructions would be nice.