Couchbase

[CCBC-216] Change 'lcb_wait' semantics

Dependency — Wed, 22 May 2013 10:53:51 -0500

Currently we have 'wait' and 'breakout' which is optimized and termed for a synchronous model.

We should rename these functions (though we can still maintain aliases) to something like:

lcb_operations_flush:

Which indicates that the user has finished sending requests, and that packets should be sent

And:
lcb_operations_done_callback

Which will be called when the operations have been completed.

Effectively this is the same as lcb_wait and the stop_event_loop callback, but the usage and purpose of those two existing functions aren't clear.

[CCBC-215] Test whether server gets reconnected when user idle for longer than timeout interval

Wed, 22 May 2013 00:47:27 -0500

see inline discussion here: http://review.couchbase.org/#/c/26462/2/src/timeout.c

Basically, the semantics of the timeouts aren't clear - the timeout will unconditionally purge the server and disconnect if the timeout hits, even if there is nothing waiting.

In practice this might not be a bug, but it means needless connection code to the server (and SASL auth) is being executed each time.

[CCBC-214] Re-design internal operations in a queue/linked list structure

Dependency — Tue, 21 May 2013 11:46:07 -0500

Re-design internal operations in a queue/linked list structure

[CCBC-213] Unify internal connect() routines

Dependency — Tue, 21 May 2013 23:12:50 -0500

This will unify the connection routines in terms of error handling and callback invocations.

[CCBC-212] IO Enhancements

Dependency — Wed, 22 May 2013 10:53:51 -0500

This super bug is an umbrella task for refactoring our IO system to be more streamlined, readable, and compatible with other I/O models.

[CCBC-211] libcouchbase may touch a failed node after failover

Tue, 21 May 2013 10:24:35 -0500

Library version: v2.0.6 (compiled from the source code, on x64)
Server version: v2.0.1
Number of nodes in my test setup: 3 (2 x CentOS 6.2 x64, and 1 x CentOS 5.8 x86, if it's of any relevance)

Steps to reproduce:

1. Create a new cluster
2. Print the returned value by lcb_get_server_list function (there shoud be 3 servers printed)
3. Bring down network on one of servers
4. Try to put several values, until one of them is routed to the failed node. The library should return an error, which is expected.
5. Repeat 2
6. Manually fail over the failed node.
7. Print the returned value by lcb_get_server_list function (there should be 2 servers printed)
8. Try to put several values, until one of them is routed to the failed node. This behaviour is not expected.
9. Start rebalance.
10. While rebalancing takes place, a request might be routed to the failed node. It is not expected as well.
11. Once rebalance is completed, no requests are routed to the failed node. It is as expected.

Thanks in advance for your support.

[CCBC-210] segfault from lcb_wait and libev

Thu, 16 May 2013 04:52:14 -0500

When running a test with a broken cluster, I happened to have PHP segfault after a long, long wait.

#0 0xffffffffffffffff in ?? ()
#1 0x00002aab69d1fab4 in ?? () from /usr/lib64/libcouchbase.so.2
#2 0x00002aab69d16421 in ?? () from /usr/lib64/libcouchbase.so.2
#3 0x00002aab6b271e73 in ev_invoke_pending () from /usr/lib64/libev.so.4
#4 0x00002aab6b276688 in ev_run () from /usr/lib64/libev.so.4
#5 0x00002aab6b06dd86 in ?? () from /usr/lib64/libcouchbase_libev.so
#6 0x00002aab69d1e698 in lcb_wait () from /usr/lib64/libcouchbase.so.2
#7 0x00002aab69af22b3 in php_couchbase_view_impl (ht=2,
    return_value=0x178de850, return_value_ptr=0x0, this_ptr=0x178dcf28,
    return_value_used=1, oo=1, uri_only=0)
    at /tmp/tmpwlG54H/couchbase-1.1.5/views.c:347
#8 0x00002aab69adfe86 in zim_couchbase_view (ht=395387424, return_value=0x1,
    return_value_ptr=0x11, this_ptr=0x0, return_value_used=16)
    at /tmp/tmpwlG54H/couchbase-1.1.5/apidecl.c:768
#9 0x00000000005e78e9 in ?? ()
#10 0x00000000005bd9ab in execute ()
#11 0x0000000000599f45 in zend_execute_scripts ()
#12 0x000000000054ad48 in php_execute_script ()
#13 0x0000000000622cde in ?? ()
#14 0x0000003d16a1d994 in __libc_start_main () from /lib64/libc.so.6
#15 0x0000000000421789 in _start ()

[CCBC-209] Make Windows builds work with jom

Fri, 10 May 2013 01:29:45 -0500

Jom supports parallel-make for windows' nmake (the equivalent to normal make's '-j' option). Right now compiling on windows is painfully slow.

Another alternative would be to make a solution file.

http://qt-project.org/wiki/jom

[CCBC-208] libvbucket CRC32 hashing function returns wrong hashes

Thu, 9 May 2013 14:02:28 -0500

It seems that the CRC32 hashing function in libvbucket (hash_crc32()) returns an invalid hash.

For instance, hashing the key '1234567890' returns 9757 (0x261D), while it should return 639479525 (0x261DAEE5).

Every key I tried (on both win and linux) returns a 16bit hash.

The problem seems to be on line 85 in crc32.c.

Changing the return expression to 'return crc ^ ~0;' fixes the problem and computes the correct hash.

[CCBC-207] Failed connection attempts will hang until timeout interval is reached on win32

Wed, 8 May 2013 15:55:16 -0500

Failed connection errors are not detected in the select() call. This is because the select call does not add the socket to the exception fdset.

MSDN documentation states that failed connections are delivered through exceptfds, and that only successful connections are delivered as writability notifications. See http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141(v=vs.85).aspx

[CCBC-206] Improve libcouchbase rpm instructions

Thu, 2 May 2013 16:35:16 -0500

There is currently no documentation about installing libcouchbase from the rpm or deb files.
The only documentation is for installation with yum or apt-get.

Since some packages are optional, more instructions would be nice.

[CCBC-205] Provide stricter semantics/behavior for configuration cache.

Wed, 1 May 2013 16:43:06 -0500

We may want to create a 'cache-only' mode in which creation of the client fails if the cache cannot be created (e.g. a bad path was passed, or the file is not writeable, etc.)

Currently, the cache is mostly a transparent extra, but there is no way for a user to determine whether the mechanism is at all working.

[CCBC-204] configuration cache does not work for memcached buckets

Wed, 8 May 2013 06:31:57 -0500

Currently, the configuration cache relies on not_my_vbucket responses as an indicator that the configuration needs to be updated. In an environment with memcached buckets only though, there will not be such a response and thus we need another way to reliably update the configuration.

One thought would be to simply ensure the configuration cache is not more than NN seconds old. The process who has the configuration connection would simply need to touch it every NN-1 seconds if the configuration connection is still active.

If that process exits, then after NN seconds, processes will race to create a new configuration cache file.

The only downside I can see is that certain failure modes (stale TCP socket) would still need to be handled.

[CCBC-202] Initial connect timeout not being observed

Mon, 6 May 2013 02:25:13 -0500

Steps:

1) Set the timeout to something small (say, 1 second)
2) Suspend the mock server
3) Try to connect to the mock

This will hang the client until an actual connection error is received (or possibly is never received). The perl tests die with a SIGALRM in this case.

    note "Suspending mock server";
    $mock->suspend_process();
    $cbo->timeout(2);
    ok(!$cbo->connect(), "Connect failed");

Needless to say, $cbo->connect() stalls.

[CCBC-201] Pass Host HTTP header when doing bootstrap

Mon, 6 May 2013 02:26:05 -0500

According to HTTP/1.1 Host header is required.

Either pass Host header or use HTTP/1.0

[CCBC-200] lcb_make_http_request doesn't respect timeouts

Fri, 5 Apr 2013 06:17:51 -0500

Run the attached testcase on Windows (after fixing the host and such). The lcb_wait after calling lcb_make_http_request will hang for a very long time (perhaps forever) if the server is no longer accessible.

I tested this by starting the client. When it prints "sleeping" I pull the ethernet cable on my machine. If the lcb_make_http_request is replaced by lcb_get then you'll get a timeout which is expected.

[CCBC-199] Problem with a newly created memcached bucket

Tue, 2 Apr 2013 06:28:35 -0500

See: http://www.couchbase.com/forums/thread/libcouchbase-problem-newly-created-memcached-bucket

[CCBC-198] Instance should close REST connection upon receiving valid config cache

Sat, 11 May 2013 13:28:28 -0500

If a valid config cache configuration is received, there is no point in keeping the REST socket open; (one of the reasons someone would even want to use a cached config is in order to reduce load from the server - so we should try to minimize connections in such cases).

Close the socket immediately (or eventually) if a valid configuration is obtained. Re-open as necessary.

[CCBC-197] configuration socket reconnects infinitely when using cached config

Thu, 16 May 2013 10:39:00 -0500

The connection to the REST configuration socket is infinitely retried when a cached configuration is used. This is because the refresh_config_cache function calls apply_vbucket_config; which changes the vbucket configuration and relocates the buffers, but does not tell the instance to stop trying to connect to the REST node.

The fix is to make it call the function which is called whenever a new configuration is received over the socket, and optionally close the socket and/or timers when the configuration is properly received.

[CCBC-196] Handle -1 return value from found_incorrect_master

Mon, 6 May 2013 02:29:21 -0500

If vbucket_found_incorrect_master returns a negative value, the code in parse_single will fail:

assert((lcb_size_t)idx < instance->nservers);

This should be handled and the command should be failed out accordingly.

[CCBC-195] config cache will crash client due to relocating buffers

Tue, 2 Apr 2013 03:30:20 -0500

See CCBC-190

During a cluster configuration change, the buffers are relocated while they are being processed. This causes crashes.

A solution would be to "request" a configuration refresh and have that take place once it is safe to move buffers around (i.e. once we are out of the buffer processing call chain).

[CCBC-194] Please add support for setsockopt() SO_KEEPALIVE & TCP_KEEPIDLE

Mon, 25 Mar 2013 12:25:05 -0500

Please add support for setsockopt() options SO_KEEPALIVE & TCP_KEEPIDLE through (suggestion)
lcb_connect() or some other such api so that I can control the connection's ability to not timeout
and have a firewall cause a disconnection from lack of inactivity...without such control, receive
NETWORK_ERROR from couchbase apis.

I cannot adjust the firewall, so I need to keep the connection alive.

[CCBC-193] libcouchbase build is not checking install properly

Fri, 22 Mar 2013 19:56:36 -0500

I was trying to package the PHP client, which depends on libcouchase, and it is was refusing to build on some systems.

After looking into it a bit, there may be a couple of problems, but the big one is in the summary.

One problem is that it seems the dpkg -i doesn't actually get run. I added an echo of the label in there, and see that in the console output in Jenkins, but I don't see the lines under the label run. I'm stumped there.

[CCBC-192] Failure to handle host that has been removed from a cluster

Wed, 20 Mar 2013 21:38:41 -0500

We're encountering an issue where both the Ruby and Perl clients get() method will consistently, repeatedly just return an error. On the Perl client it is "Temporary error. try again later", and on Ruby it is "Bucket not found".

This occurs when we have several couchbase servers, and have connected using the node list option to supply all of them to the library.

We remove the first server in the node list via the Couchbase GUI, trigger a rebalance, and wait for it to complete.
Once the rebalance has completed, we start a script that simply attempts to connect and retrieve some keys we previously inserted. Instead of success, we find that it repeatedly gives the errors mentioned above.

note that if we remain connected to the cluster, then everything seems fine. The errors only start occurring when a client connects.

note also that the issue only occurs if the first server (or servers) in the list are down; if the first server listed is currently actively in the cluster, then we're OK.

It seems like the problem is that the now-removed server is still accepting connections on port 8091, and thus the client library thinks that this means it is a valid server. However because it's removed from the pool, it causes confusion and errors to the client.

Obviously, the correct/desired behaviour would be for the client library to behave as if the removed-server was not accepting connections, and to move onto the next server in the node list.

[CCBC-191] Segfault when quering view with spaces

Tue, 19 Mar 2013 10:38:14 -0500

Segfault when quering view with spaces

[CCBC-190] Conncache configuration not passing integration tests

Dependency — Mon, 1 Apr 2013 12:39:36 -0500

There are two primary issues which occur during server configuration changes (i.e. failover, rebalance):

1)
[SDKD(WARNING) 56.55 cbsdk.sdkd.remote remote.py:266] sdkd_lcb: src/event.c:211: parse_single: Assertion `(lcb_size_t)idx < c->instance->nservers' failed.

2)
The client may just hang.

[CCBC-189] Create a compat mode that cache the configuration objects

Tue, 26 Feb 2013 13:23:21 -0600

Create a compat mode that cache the configuration objects

[CCBC-188] library codedumped during rebalance process

Tue, 23 Apr 2013 06:19:49 -0500

How to reproduce:
1) Attach second node to a Couchbase claster, start rebalance process and in that time call lcb_get function many times. As a result application go down (assert in 0).

assert(ringbuffer_read(&src->pending_cookies, &ct, sizeof(ct)) == sizeof(ct));

src->pending_cookies->nbytes == 0 in that case, so ringbuffer_read return 0 so assert is 0
and application is over.
We think, that nbytes == 0 is a common situation, than src->connected == 0 and this situation must not interpretated as error.

Below you can find additional information
Stack:
#0 0xc000000000211ab0:0 in _lwp_kill+0x30 () from /lib/hpux64/libpthread.so.1
#1 0xc000000000178810:0 in pthread_kill ()
at /ux/core/libs/threadslibs/src/common/pthreads/signal.c:207
#2 0xc0000000003f8140:0 in raise ()
at ../../../../../core/libs/libc/shared_em_64_perf/../core/gen/raise.c:33
#3 0xc000000000508c50:0 in abort ()
at ../../../../../core/libs/libc/shared_em_64_perf/../core/gen/abort_em.c:84
#4 0xc0000000003c3f90:0 in _assert ()
at ../../../../../core/libs/libc/shared_em_64_perf/../core/gen/assert.c:57
#5 0xc0000000076a9080:0 in relocate_packets (src=0x6000000001c63c00,
dst_instance=0x60000000064f6800) at src/instance.c:542
#6 0xc0000000076a9b40:0 in lcb_update_serverlist (instance=0x60000000064f6800)
at src/instance.c:606
#7 0xc0000000076ac7e0:0 in vbucket_stream_handler (sock=31, which=2,
arg=0x60000000064f6800) at src/instance.c:996
#8 0xc000000007d6db60:0 in event_base_loop (base=0x6000000006381780,
flags=<not available>) at event.c:1346
#9 0xc000000000a3aed0:0 in lcb_io_run_event_loop (iops=0x60000000061bff20)
at plugins/io/libevent/plugin-libevent.c:324
#10 0xc0000000076c2750:0 in lcb_wait (instance=0x60000000064f6800)
at src/wait.c:60
#11 0xc0000000109c5e00:0 in Cache_Storage::CCouchbaseStorage::do_usefull_work (
---Type <return> to continue, or q <return> to quit---
this=0x6000000006224280) at cache_source/ucouchbasestorage.cpp:524
#12 0xc0000000109c7850:0 in Cache_Storage::CCouchbaseStorage::execute (
this=0x6000000006224280) at cache_source/ucouchbasestorage.cpp:585
#13 0xc00000000782c0f0:0 in threads::thread_proc (thr_ptr=0x60000000062242a0)
at has_common_source/source/cpp/threads.cpp:225
#14 0xc00000000013fb20:0 in __pthread_bound_body ()
at /ux/core/libs/threadslibs/src/common/pthreads/pthread.c:4875

(gdb) list
537 }
538 dst = dst_instance->servers + (lcb_size_t)idx;
539 if (src->connected) {
540 assert(ringbuffer_read(&src->output_cookies, &ct, sizeof(ct)) == sizeof(ct));
541 } else {
542 assert(ringbuffer_read(&src->pending_cookies, &ct, sizeof(ct)) == sizeof(ct));
543 }
544
545 assert(ringbuffer_ensure_capacity(&dst->cmd_log, npacket));
546 assert(ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes));
(gdb) p src->pending_cookies
$5 = {root = 0x6000000001e3d200 "", read_head = 0x6000000001e3d200 "",
write_head = 0x6000000001e3d200 "", size = 512, nbytes = 0}
(gdb) p src->connected
$7 = 0

lcb_size_t ringbuffer_read(ringbuffer_t *buffer, void *dest, lcb_size_t nb)
{
char *d = dest;
lcb_size_t nr = 0;
lcb_size_t space;
lcb_size_t toRead;

if (buffer->nbytes == 0) {
return 0;
}

[CCBC-187] Set TCP_NODELAY flag

Mon, 25 Feb 2013 01:14:48 -0600

We think it is good idea to set TCP_NODELAY flag to switch off the Nagle mode
(don't wait for sending a short packets)
Couchbase server, as we think, use this flag

[CCBC-186] Request to provide concrete examples in the documentation for OBSERVE functionality

Thu, 21 Feb 2013 15:17:58 -0600

http://support.couchbase.com/tickets/2608

Customer are interested in using OBSERVE functionality to allow them to check whether items have been replicated successfully, as they are worried about unreplicated items.

https://github.com/couchbase/libcouchbase/blob/master/man/man3couchbase/lcb_observe.3couchbase.txt
has basic info on the OBSERE functionality in libcouchbase, but we need to provide concrete examples on how to check multiple keys having been replicated successfully (they do a bulk write, so should be more efficient to wait for all of them, rather than wait for them individually)

Examples provided to the customer :

You will find the below example very useful for simple operations :

https://github.com/couchbase/php-ext-couchbase/blob/master/simple_observe.c

The below example would provide more insights when you have any performance or reliability issues :

https://github.com/couchbase/php-ext-couchbase/blob/master/observe.c

The basic idea here is to constantly "poll" the keys (sleeping within a given interval) until all of them are declared as "OK". The structures involved are the "observe_expectation" (persistence/replication requirements), the "pollprefs" (i.e. timeout, interval, etc.) and the "keystate" - this keeps tabs on how the actual status of each key matches up with the specified criteria. The lcb_observe callback is called for each "status report" for each key from each server; so it is necessary to check and see if the requirements have been met during each individual update.

[CCBC-185] Incorrect parsing of URIs

Tue, 5 Mar 2013 16:07:55 -0600

The current parsing of the hostname believes that the entire thing specified is the hostname rather than stripping of potential http:// and paths

[CCBC-184] There are a lot of connections are established after restarting Couchbase server

Mon, 25 Feb 2013 01:08:00 -0600

1) Start Couchbase Server
2) Start to recieve some key from Couchbase server using a client based on libcouchbase library. We use about 250 threads for receiving key
3) Restart Couchbase server
4) After that a lot of connections are created (all of then in ESTABLISHED state). And soon quota for handlers is over.

[CCBC-183] Implement approach to specify behaviour for lcb_get_replica()

Fri, 15 Feb 2013 12:10:15 -0600

Implement approach to specify behaviour for lcb_get_replica()

[CCBC-182] The library stops iterating backup nodes list if next one isn't accessible

Tue, 5 Mar 2013 16:07:55 -0600

Because of typo in loop condition[1], it won't continue trying the backup nodes

https://github.com/couchbase/libcouchbase/blob/4053418dbdffa8e2315c35c433768d0fa19e9fd0/src/instance.c#L834

[CCBC-181] enhance observe man page with a more complete example

Tue, 2 Apr 2013 01:36:45 -0500

The current observe man page doesn't really go into how to interpret results from the request. I needs to have that.

Ideally it should also show how one would do a regular poll to show checking for replication or persistence.

[CCBC-180] Segfault when in case server name resolved to several addresses, where first of them rejects couchbase connection

Tue, 5 Mar 2013 16:07:54 -0600

I get a lot of segfaults using the client when trying to connect to _external_ couchbase servers (i.e. when there is no couchbase server on localhost).

I've tested this on a couple different machines, with both ruby REE and 1.9.3 - same results every time.

To reproduce:
1) setup couchbase server / cluster on machine-1
2) setup ruby development environment on machine-2
-- I'm using rvm
-- I've install the couchbase library
3) run simple ruby program to test connections:
#simple_test.rb
require 'couchbase'
Couchbase.connection_options = {:bucket => 'rubytest', :node_list => ['123.45.67.89:8091'], :password => 'password'}
Couchbase.connect
4) watch it segfault when you run it

Essentially, it seems that setting the 'connection_optoins' has no effect on creating new buckets, they keep trying to connect to the built-in default of 'localhost'

This is especially problematic because of what it means trying to use couchbase in production: If I work around this by passing connection options to a new bucket, I will eventually run into this same segfault issue whenever the cluster I am connecting to is rebalancing - I've gotten quite a few segfaults when adding / removing nodes and then rebalancing.

[CCBC-179] Provide and example and clear statement that for usual data connection user should specify bucket credentials instead of admin

Mon, 6 May 2013 02:48:55 -0500

https://github.com/couchbase/libcouchbase/blob/master/man/man3couchbase/lcb_create.3couchbase.txt

This doc lacks of example with bucket level credentials

[CCBC-178] Build error on solaris/sparc: -Werror=cast-align

Tue, 5 Mar 2013 16:07:54 -0600

CC src/libcouchbase_la-server.lo
src/server.c: In function 'lcb_failout_observe_request':
src/server.c:47:8: error: cast increases required alignment of target type [-Werror=cast-align]

[CCBC-177] Docs: Reference installation

Mon, 11 Feb 2013 05:36:17 -0600

Customers have lots of choices at their fingertips, and many of those choices will result in a non-functioning installation, or at least a lot of time spent figuring out what the right pieces are.

Can we please provide as much details as possible about a reference installation that we can be sure will work for a customer who is just getting started? This is outside of our support for any one operatin system, but more about telling the customer "if you install with these versions and packages, this code will work"

-OS
-other packages/modules required and/or tested with

[CCBC-176] libcouchbase does not work on systems with EWOULDBLOCK != EAGAIN

Duplicate — Wed, 6 Feb 2013 13:48:21 -0600

Some systems define EWOULDBLOCK, but have a different value for EAGAIN and returns EAGAIN instead of EWOULDBLOCK in some cases.

We found this "the hard way" on HP-UX where libcouchbase wouldn't work at all.

[CCBC-175] Network Error on HP-UX "Resource temporarily unavailable"

Duplicate — Wed, 6 Feb 2013 13:48:22 -0600

Network Error on HP-UX "Resource temporarily unavailable"

[CCBC-174] Error handling documentation

Tue, 5 Feb 2013 02:39:18 -0600

Please create some documentation specifying possible error/failures to operations, what they "look" like in the logs/exceptions/stack traces and what our recommendation is on how to handle them.

i.e. tmp_oom, timeouts (connection/operation/java-internal/etc), "get miss" (it's technically a failure, let's make it overly obvious what it means), CAS failure, add() failure, replace() failure,

Some of this should be covered in the API reference, but this bug is specifically for a single page where this information is aggregated that a customer/user could read about how to handle errors.

[CCBC-173] Small typo in C tutorial

Mon, 6 May 2013 11:25:29 -0500

From http://www.couchbase.com/forums/thread/string-typo-c-sdk-tutorial-sample-code

In the couchbase-sdk-c-2.0.pdf tutorial sample code, there's a typo in an error string:
{
lcb_store_cmd_t cmd;
const lcb_store_cmd_t *commands[1];
commands[0] = &cmd;
memset(&cmd, 0, sizeof(cmd));
cmd.v.v0.operation = LCB_SET;
cmd.v.v0.key = "foo";
cmd.v.v0.nkey = 3;
cmd.v.v0.bytes = "bar";
cmd.v.v0.nbytes = 3;
err = lcb_store(instance, NULL, 1, commands);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to get: %s\n", lcb_strerror(NULL, err));
return 1;
}
}
The err code there is for a set operation, so the error string in the fprintf call should say "Failed to set" not "Failed to get".

[CCBC-172] When connection aborted in GetError abort() function is called

Wed, 6 Feb 2013 13:48:22 -0600

How to reproduce:
1) Start writing keys to a bucket.
2) Delete bucket from admin panel

As I understand problem is in getError function (plugin-win32.c)
In that case function WSAGetLastError() returns WSAECONNABORTED (10053)
But this value doesn't present in switch-case.
So executes code in default label and abort() function is called...

[CCBC-171] Invalid read in plugin-libevent.c

Wed, 6 Feb 2013 13:48:22 -0600

In plugin-libevent.c, event_new() mallocs a "struct event", which is not initialized, then passes it to event_assign(), which passes it into event_base_set(), causing an invalid read. We picked it up during a valgrind run of our program.

See lines 47 - 78 here: https://github.com/couchbase/libcouchbase/blob/master/plugins/io/libevent/plugin-libevent.c

Proposed patch:

Index: libcouchbase/src/plugin-libevent.c
===================================================================
--- libcouchbase/src/plugin-libevent.c (revision 16)
+++ libcouchbase/src/plugin-libevent.c (working copy)
@@ -47,7 +47,6 @@
              event_callback_fn callback,
              void *arg)
{
- event_base_set(base, ev);
     ev->ev_callback = callback;
     ev->ev_arg = arg;
     ev->ev_fd = fd;
@@ -56,6 +55,7 @@
     ev->ev_flags = EVLIST_INIT;
     ev->ev_ncalls = 0;
     ev->ev_pncalls = NULL;
+ event_base_set(base, ev);

     return 0;
}

[CCBC-170] After upgrading to 2.0.2 with CBC163, I now get timesouts again!!!

Wed, 6 Feb 2013 15:43:10 -0600

I was using 2.0.0 beta3 with the patch for TIMEOUTS manually applied....this was working fine but I had high CPU. This ran for weeks (if not a month) without any
timeouts at all...I was the happiest with lcb as I had been in a long time. However, after getting some complaints from my server admins and NOC, I upgraded to
2.0.2 with the fix for high CPU...

I rebuilt all modules and rolled to production. It's now been about 4 hours and I have received 2 timeouts as to where I had NONE for weeks without this change.

The load is now normal though but somehow something has broken because of this fix.

I looked at the change with CBC-163, but I don't see how this could have broken the code...but, it has - I promise. Can someone please take a look?

HELP!

[CCBC-169] Handle 30x HTTP response code when chosen node doesn't have active vbuckets

Dependency — Tue, 5 Mar 2013 16:08:38 -0600

Handle 30x HTTP response code when chosen node doesn't have active vbuckets

[CCBC-168] Query on multiset and multiget libcouchbase

Wed, 6 Feb 2013 13:48:20 -0600

As per a query from the customer on multiset and multiget regarding libcouchbase 2.0.1 client

"When you calculate number of operation, do you consider the set of keys in Multiset ? Or is it just one operation?
I am trying to calculate the writes per second. The client would send 15k writes per second, each of them is a multiset with 20 keys. So is this 15k writes per second ?
Or 15*20= 300k writes per second ? "

Please help answer this.

[CCBC-167] lcb_wait() waits infinitely

Duplicate — Wed, 6 Feb 2013 13:48:22 -0600

From http://www.couchbase.com/forums/comment/reply/1002656

> Usually problem happens after creating bucket (I use curl for it).
> After that I try to connect to created bucket... and call lcb_wait function.
> And it executes infinitly...
> I investigate it a little and found, what the problem occures when in Couchbase server bucket loaded some times (in log there is note something like "bucket Name loader (3 times)").
> And why lcb_wait works infinitly???
> I add debug output and found that select function (on which event loop is based) returns 0 (that means that timeout happend) and after that timer is deleted... and than next time select calls, timeout is 0... that means that wait event infinitly, but events doesn't occurs.
> What do you think about that problem?
> P.S. Then I create many buckets in parallel, this bug occurs very often

[CCBC-166] make distcheck may file depending on how the dist is built

Wed, 6 Feb 2013 13:48:21 -0600

The problem is how we're building the dist file. Currently we include all of the .txt files into the bundle and only a subset of the generated files (man pages). Instead we should list the files we want to include in the dist file, and that should be all of the _GENERATED_ files.

[CCBC-164] Concurrent access to continuum can cause invalid memory reference when servers are removed

Wed, 6 Feb 2013 13:48:21 -0600

In libvbucket/vbucket.c lines 121/122, function update_ketama_continuum first updates the array pointer, and then the counter.

In lines 615/616, function vbucket_map reads both the array pointer and the counter to determine the boundaries of the memory to scan.

In the (rare) event that a server is removed and thread A executes update_ketama_continuum line 121 with a smaller array and then blocks, a second thread B could execute vbucket_map, read the new array pointer and the (larger) size of the old array, and access memory that has been freed.

A similar issue exists in (some versions of) libmemcached code. To fix this, one could change the code to use a pointer to a struct which contains both the array and its size, or place both the array pointer and the size adjacent in memory and use an atomic operation to update both simultaneously ( less portable )

[CCBC-163] High CPU spikes for operations

Wed, 6 Feb 2013 13:48:22 -0600

High CPU spikes are observed in the latest HEAD on the 2.0.2 pre-release version. This is due to the LCB_WRITE_EVENT not being disabled

[CCBC-162] API reference uses old names, not current lcb_* names

Thu, 31 Jan 2013 03:11:36 -0600

The API reference for all functions uses libcouchbase_* names (libcouchbase_create, etc.) instead of the correct lcb_* names (lcb_create, etc.).

[CCBC-161] libcouchbase_flush wrongly documented to "flush entire cluster"

Thu, 2 May 2013 16:32:40 -0500

The docs say that libcouchbase_flush() will "Flush the entire cluster" and "libcouchbase_flush may be used to remove all key/value pairs from the entire cluster."

It actually flushes a bucket, not the entire cluster.

[CCBC-160] Request timeout while connecting to protected bucket

Wed, 6 Feb 2013 13:48:24 -0600

The bug was introduced in this fix http://www.couchbase.com/issues/browse/CCBC-159

To reproduce just try to connect to SASL protected bucket

[CCBC-159] Scheduling large list of commands leads to timeouts

Dependency — Wed, 6 Feb 2013 13:48:23 -0600

Scheduling large list of commands leads to timeouts

[CCBC-158] Unable to store 1M objects in a loop ( Internal error (0x6), Unknown implicit send message op=1 , eventually crash )

Wed, 6 Feb 2013 13:48:21 -0600

When executing 1M lcb_store calls in a loop, storing items like "user0000001 = <16 hex bytes>" in the default bucket, I get a number of error callbacks saying "Internal error (0x6), Unknown implicit send message op=1" ( opcode 0x1 = PROTOCOL_BINARY_CMD_SET ). Finally, after a last "Internal error (0x6), Unknown implicit send message op=46" the process gets killed
.
Unfortunately I cannot tell where the crash happens, as running with gdb results in:
Couchbase event loop starting/build/buildd/gdb-7.4-2012.04/gdb/infrun.c:3189: internal-error: handle_inferior_event: Assertion `inf' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.

After a number of attempts in both debug and release mode, I ended up with almost 350K out of 1M items entered. However, I don't manage to run the loop to its completion; something always breaks before it completes

[CCBC-157] libvbucket should validate vbucket indices before performing lookup

Wed, 6 Feb 2013 13:48:25 -0600

Otherwise, this will cause a segfault and buffer overrun;
e.g. vb=44444 should raise an error rather than crash

[CCBC-156] don't retry OBSERVE on NOT_MY_VBUCKET

Wed, 6 Feb 2013 13:48:24 -0600

Retrying observe on NOT_MY_VBUCKET might lead to an infinite network loop as NOT_MY_VBUCKET is received if any of the vbuckets are not belonging to the server.

This could happen if an OBSERVE command happens during a configuration change.

[CCBC-155] Observe will malfunction in the case of multiple keys and server failure

Wed, 6 Feb 2013 13:48:24 -0600

Observe requests and responses may contain multiple keys, but this is not handled during the server failure phase in the code; therefore a user will not get callbacks for all the keys.

We need to make a common handler for observe requests and responses

[CCBC-154] Missing documentation for some error codes

Wed, 6 Feb 2013 13:48:24 -0600

LCB_SERVER_BUG and LCB_PLUGIN_VERSION_MISMATCH is not documented with lcb_strerror

[CCBC-153] core dump whenever CouchBase connection attempt fails

Dependency — Wed, 6 Feb 2013 13:48:24 -0600

We are seeing a core dump whenever CouchBase connection attempt fails for the first time and we try to reconnect.
This is a case when the client comes up before the Couchbase server.

[CCBC-152] Provide lcb_check to (weakly) check for the existence of a key

Fri, 21 Dec 2012 13:27:18 -0600

This API will be a variant of OBSERVE and shall do a simple probe against the master node to see if the key exists in cache.

We need to have this in libcouchbase as normal observe is implemented by 'broadcasting' to all the nodes

[CCBC-151] Cancellation of the HTTP request might lead to segfault or memory leaks

Wed, 6 Feb 2013 13:48:22 -0600

* request_valid() and lcb_cancel_http_request() must not dereference request pointer until it makes sure that the pointer is valid

* lcb_cancel_http_request() must not remove the request from the hashset, because it will prevent it from destroying

* on_data callback should check if the request has been cancelled and finish it without on_complete notification.

[CCBC-150] Observe (and possibly other) callbacks receiving NULL callback more than once when batched

Dependency — Wed, 6 Feb 2013 13:48:20 -0600

See here http://paste.scsys.co.uk/219902 for an example.

This primarily affects observe operation in 'multi' mode; where multiple observe requests are queued up. This fails when the request distribution is asymmetric.

This also affects stats and version commands if coupled with other commands in an asynchronous mode.

This will definitely affect node.js.

In summary, the function lcb_lookup_server_with_packet tries to find 'associated' packets; but it does it wrongly. It assumes associated packets will always be at the tail (or pop-end) of the command log.

However this will only be the case in a purely synchronous mode where there are no subsequent commands. The following scenario demonstrates this rather well:

With three servers and two keys (and one replica), you have something like this for the vb/server map

"K1" = servers[1], servers[2]
"K2" = servers[0], servers[1]

When the commands are sent out, the command look for the server looks something like:

-- order in the buffer
BUFFERS:
[ OLD .. NEW]

0: "K2",
1: "K1", "K2"
2: "K1",

LOGICAL PRECEDENCE:
[ OLD .. NEW ]

"K1", "K2"

The idea to keep in mind is that even though we've scheduled the request for "K1" first, we may very well receive the response for "K2" first - this is especially true if server[0] is quicker than the other servers.

So what happens when "K2" is received first? - the code checks to see the older commands on all the other servers (only the oldest command), and if not, it assumes this is the last command to be received and notifies the user that the command has finished.

So now the buffers look like:

0: "K3"
1: "K1", "K2"
2: "K1"

Now, server[1] responds with "K1". Since the request for "K1" is still at the pop-end of server[2] nothing special happens here

The buffers look like:

0:
1: "K2"
2: "K1"

Then server[1] responds again, this time with the next response for "K2". It delivers the callback again. In a typical use case a user maintains a cookie tied to the command, and when the command is done, the cookie is freed. Plainly speaking, this double-singalling (i.e. telling the user a command has finished twice) segfaults the app.

[CCBC-149] Docs: typo under Mac installation instructions

Fri, 8 Feb 2013 06:01:26 -0600

http://www.couchbase.com/develop/c/current

The text for the Mac installation instructions needs to be cleaned up a bit.

[CCBC-148] intermittent test failures in MockUnitTest.testPurgedBody

Wed, 6 Feb 2013 13:48:20 -0600

Currently, we see intermittent test failures all in the same test from sdk-repo. This has been at least since build #866, and probably earlier.

http://sdkbuilds.couchbase.com/job/sdk-repo/

[----------] Global test environment tear-down
[==========] 80 tests from 17 test cases ran. (49277 ms total)
[ PASSED ] 79 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] MockUnitTest.testPurgedBody

[CCBC-147] function lcb_wait works infinitly after trying to connect (by calling lcb_connect) to Couchbase server that is on Pending state

Duplicate — Tue, 5 Mar 2013 16:07:54 -0600

Start server with one node.
Create many buckets, that uses all allowed memory -> Server have to chnage status on Pending (In my situation server stay in Pending for all time and can't change it to Up)

After that try to connect to one of the bucket by calling lcb_connect.
Call lcb_wait to wait for connection is done.
As a result lcb_wait work infinitly and timeout doesn't happen

Below I share call stack in Red Hat Linux:
#0 0x00000034350d4473 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1 0x00002ad81b1dccc9 in ?? () from ./lib/libevent-2.0.so.5
#2 0x00002ad81b1c9cdc in event_base_loop () from ./lib/libevent-2.0.so.5
#3 0x00002ad8174c1dc6 in lcb_wait () from ./lib/libcouchbase.so.2
#4 0x000000000043c0e1 in Couchbase::connect (this=0x60df508) at couchbase_loader_source/couchbase.cpp:152
#5 0x000000000043c8b7 in connect_to_bucket (cbase=..., config=..., bucket_name=...) at couchbase_loader_source/couchbase.cpp:478
#6 0x0000000000440924 in couchbase_loader::writer_thread::writer_thread (this=0x60df480, config=..., bucket_name=..., dbh=...) at couchbase_loader_source/writer_thread.cpp:25
#7 0x000000000042168c in main (argc=1, argv=0x7fff22162028) at couchbase_loader_source/couchbase_loader.cpp:175

[CCBC-146] configuration.h not copied during installation

Wed, 6 Feb 2013 13:48:21 -0600

configuration.h not copied during installation

[CCBC-145] Provide API for durability operations

Dependency — Fri, 21 Dec 2012 18:59:38 -0600

This feature will provide a callback-oriented API for durability (aka observe-poll) operations. Considering that the logic of implementing these operations has proved somewhat more complex than intended, it would be better to centralize this in libcouchbase.

The API will provide a "durability set" which is a group of keys to be monitored, a "per-key" callback for when each key reaches its persistence/durability requirements, a "set callback" for when all keys have been endured (or have encountered a condition in which no further retries can be made)

[CCBC-144] Standardize view option handling

Thu, 10 Jan 2013 15:47:26 -0600

Centralize view option conversion, validation, and construction so dependent code does not have to repeat the same boilerplate all the time. This makes it easier for client libraries to adapt to new view options, as well as provide common expected behavior with current options.

[CCBC-143] cbc version reports that uses 2.0.0, but really installed with 2.0.1

Wed, 6 Feb 2013 13:48:20 -0600

$ cbc version
cbc built from: libcouchbase 2.0.1
using libcouchbase: 2.0.0

[CCBC-142] test libcouchbase observe functionality

Sat, 8 Dec 2012 00:11:17 -0600

test libcouchbase observe functionality

[CCBC-141] test libcouchbase lock functionality

Sat, 8 Dec 2012 00:11:17 -0600

test libcouchbase lock functionality

[CCBC-140] add http tests to libcouchbase

Wed, 6 Feb 2013 13:48:24 -0600

add http tests to libcouchbase

[CCBC-139] add timeout tests for libcouchbase

Sat, 8 Dec 2012 00:11:17 -0600

add timeout tests for libcouchbase

[CCBC-138] allow libcouchbase tests to be executed against a real cluster

Wed, 6 Feb 2013 13:48:23 -0600

allow libcouchbase tests to be executed against a real cluster

[CCBC-137] libcouchbase test improvements

Sat, 11 May 2013 13:31:12 -0500

Generic improvements and fixes for libcouchbase test suite

[CCBC-136] abort from libcouchbase_purge_single_server at timeout.c:93

Dependency — Wed, 6 Feb 2013 13:48:23 -0600

#0 0x40022424 in __kernel_vsyscall ()
#1 0x402bc1ef in raise () from /lib/i386-linux-gnu/libc.so.6
#2 0x402bf835 in abort () from /lib/i386-linux-gnu/libc.so.6
#3 0x40be9b7b in libcouchbase_purge_single_server (server=0x42043d58, tmo=2500000000, now=1354688386324058000, error=LIBCOUCHBASE_ETIMEDOUT) at src/server.c:234
#4 0x40bec9bc in libcouchbase_purge_timedout (instance=0x42003af8) at src/timeout.c:93
#5 0x40beca2c in libcouchbase_timeout_handler (sock=-1, which=0, arg=0x42003af8) at src/timeout.c:41
(trimmed...)

[CCBC-135] lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed

Fri, 14 Dec 2012 13:04:43 -0600

Cannot reproduce, this is all the info I have

FAIL GetMulti - MgetOrdered [tests/phpt/GetMulti/MgetOrdered.phpt]

mnunberg@csure:/sources/lcb2.0-repo/php$ cat tests/phpt/GetMulti/*.out
PHP_COUCHBASE_OK
php: src/server.c:799: lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed.

*Might* be related to ~~CCBC-120~~

To reiterate again, I cannot reproduce this - I just happened to notice it

[CCBC-134] Repo refuses to download packages: "Package does not match intended download"

Thu, 10 Jan 2013 15:57:47 -0600

Doing this:

wget -O/etc/yum.repos.d/couchbase.repo http://packages.couchbase.com/rpm/couchbase-centos55-x86_64.repo
yum clean all
yum install -y libcouchbase2.x86_64 libcouchbase-devel

Results in a failed install/update with these errors:

[root@host ~]# yum install -y libcouchbase2.x86_64 libcouchbase-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.sonic.net
* centosplus: mirror.hmc.edu
* epel: mirrors.solfo.com
* extras: mirror.stanford.edu
* updates: mirrors.arpnetworks.com
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libcouchbase-devel.x86_64 0:2.0.0-1 set to be updated
--> Processing Dependency: libcouchbase2-core for package: libcouchbase-devel
--> Processing Dependency: libcouchbase.so.2()(64bit) for package: libcouchbase-devel
---> Package libcouchbase2.x86_64 0:2.0.0-1 set to be updated
--> Processing Dependency: libcouchbase2-backend for package: libcouchbase2
--> Running transaction check
---> Package libcouchbase2-core.x86_64 0:2.0.0-1 set to be updated
---> Package libcouchbase2-libev.x86_64 0:2.0.0-1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================
Package Arch Version Repository Size
=======================================================================
Installing:
libcouchbase-devel x86_64 2.0.0-1 couchbase 52 k
libcouchbase2 x86_64 2.0.0-1 couchbase 26 k
Installing for dependencies:
libcouchbase2-core x86_64 2.0.0-1 couchbase 82 k
libcouchbase2-libev x86_64 2.0.0-1 couchbase 31 k

Transaction Summary
=======================================================================
Install 4 Package(s)
Upgrade 0 Package(s)

Total download size: 191 k
Downloading Packages:
(1/4): libcouchbase2-2.0.0-1.x86_64.rpm | 26 kB 00:00
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase2-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.
(2/4): libcouchbase2-libev-2.0.0-1.x86_64.rpm | 31 kB 00:00
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase2-libev-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.
(3/4): libcouchbase-devel-2.0.0-1.x86_64.rpm | 52 kB 00:01
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase-devel-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.
(4/4): libcouchbase2-core-2.0.0-1.x86_64.rpm | 82 kB 00:00
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase2-core-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.

Error Downloading Packages:
  libcouchbase2-2.0.0-1.x86_64: failure: libcouchbase2-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.
  libcouchbase2-core-2.0.0-1.x86_64: failure: libcouchbase2-core-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.
  libcouchbase-devel-2.0.0-1.x86_64: failure: libcouchbase-devel-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.
  libcouchbase2-libev-2.0.0-1.x86_64: failure: libcouchbase2-libev-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.

[CCBC-133] HTTP Request hangs during failover

Wed, 6 Feb 2013 13:48:23 -0600

lcb_wait hangs..

Code:

    lcb_http_request_t htreq;
    lcb_error_t lcb_err;
    lcb_err = lcb_make_http_request(handle->getLcb(),
                                    this,
                                    LCB_HTTP_TYPE_VIEW,
                                    htcmd,
                                    &htreq);

    if (lcb_err != LCB_SUCCESS) {
        goto GT_ERR;
    }

    lcb_err = lcb_wait(handle->getLcb());
    printf("Wait returned..\n");

GDB:

$45 = {sock = -1275067896, io = 0x7f76b4000208, server = 0x0, instance = 0x4,
  url = 0x7f76ffffffff <Address 0x7f76ffffffff out of bounds>, nurl = 0, url_info = {field_set = 65535,
    port = 65535, field_data = {{off = 0, len = 0}, {off = 19968, len = 476}, {off = 0, len = 0}, {off = 65535,
        len = 65535}, {off = 65535, len = 65535}, {off = 337, len = 0}}}, host = 0x7f76b4000000 " ", port = 0x0,
  path = 0x8000000000 <Address 0x8000000000 out of bounds>, npath = 36, password = 0x7f76b4000b90 "\020",
  method = LCB_HTTP_METHOD_GET, parser = 0x0, parser_settings = {on_message_begin = 0x25,
    on_url = 0x3132312e332e3031, on_header_field = 0x3231313a3830322e, on_header_value = 0x3031,
    on_headers_complete = 0x25, on_body = 0x3132312e332e3031, on_message_complete = 0x323131003830322e},
  root_ai = 0x3031, curr_ai = 0x35, event = 0x312f2f3a70747468, chunked = 775106096,
  on_complete = 0x323930383a383032, on_data = 0x746c75616665642f, output = {root = 0x0,
    read_head = 0x35 <Address 0x35 out of bounds>, write_head = 0x3 <Address 0x3 out of bounds>, size = 7,
    nbytes = 8}, input = {root = 0x7f76b400e0c0 "", read_head = 0x0,
    write_head = 0x55 <Address 0x55 out of bounds>, size = 0, nbytes = 0}, result = {root = 0x0,
    read_head = 0x0, write_head = 0x0, size = 0, nbytes = 0}, command_cookie = 0x0, cancelled = 0,
  completed = 0, headers_list = 0x25, headers = 0x3132312e332e3031, nheaders = 4120855481762722350}
(gdb) frame 4
#4 0x00007f76ce7340a3 in lcb_wait (instance=0x1dd9580) at src/wait.c:112
112 instance->io->v.v0.run_event_loop(instance->io);
(gdb) print *instance
$46 = {type = LCB_TYPE_BUCKET, host = "10.3.121.212\000\070\060\071\061", '\000' <repeats 1008 times>,
  port = "8091", '\000' <repeats 28 times>,
  http_uri = 0x1dd14a0 "GET /pools/default/bucketsStreaming/default HTTP/1.1\r\nAuthorization: Basic QWRtaW5pc3RyYXRvcjoxMjM0NTY=\r\n\r\n", n_http_uri_sent = 107, event = 0x1dd8900, vbucket_config = 0x7f76b4003760,
  vbucket_stream = {
    header = 0x1dd7fb0 "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\nServer: Couchbase Server 2.0.0-1956-rel-enterprise\r\nPragma: no-cache\r\nDate: Sun, 02 Dec 2012 16:58:58 GMT\r\nContent-Type: application/json; charset=utf-8\r\nC"..., input = {
      data = 0x1ddca90 "{\"name\":\"default\",\"bucketType\":\"membase\",\"authType\":\"sasl\",\"saslPassword\":\"\",\"proxyPort\":0,\"replicaIndex\":false,\"uri\":\"/pools/default/buckets/default?bucket_uuid=0535828b5b9f231994f876e3b4299956\",\"str"..., size = 4096, avail = 0}, chunk_size = 18446744073709551615, chunk = {

[CCBC-132] Making http request to ECONNREFUSED node seems to loop forever...

Wed, 6 Feb 2013 13:48:23 -0600

To reproduce, simply try to issue a raw request to say, "localhost:2"

[CCBC-131] Views sometimes failing if all nodes lack couchApiBase

Wed, 6 Feb 2013 13:48:24 -0600

Views sometimes failing if all nodes lack couchApiBase

[CCBC-130] Memory leak on header responses..

Wed, 6 Feb 2013 13:48:21 -0600

    memset(&htcmd, 0, sizeof(htcmd));
    OV1(&htcmd).content_type = "application/json";
    OV1(&htcmd).path = vqstr;
    OV1(&htcmd).npath = strlen(vqstr);
    OV1(&htcmd).chunked = 1;
    OV1(&htcmd).method = LCB_HTTP_METHOD_GET;

    rctx->callback = my_vr_callback;
    lcb_set_http_data_callback(instance, http_data_callback);

    err = lcb_make_http_request(instance,
                                rctx,
                                LCB_HTTP_TYPE_VIEW,
                                &htcmd,
                                &htreq);
    assert(err == LCB_SUCCESS);
    printf("Waiting for request..\n");
    err = lcb_wait(instance);
    assert (err == LCB_SUCCESS);

    printf("Got query string: %s\n", vqstr);
    free(vqstr);
    lcb_vopt_cleanup_list(&vopt_list, num_options, 1);
    free(vopt_list);
    lcb_vrow_free(rctx);

/**********************************************************/
mnunberg@csure:~/src/sdkd-cpp/src/views$ valgrind --leak-check=full ./test
==31887== Memcheck, a memory error detector
==31887== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==31887== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==31887== Command: ./test
==31887==
Waiting for request..
Got callback..
Got status 404
Parsing bytes '{"error":"not_found","reason":"Design document _design/design_doc not found"}
'
Got query string: _design/design_doc/_view/view_doc?stale=false&limit=150&debug=true
==31887==
==31887== HEAP SUMMARY:
==31887== in use at exit: 1,915 bytes in 18 blocks
==31887== total heap usage: 3,349 allocs, 3,331 frees, 305,322 bytes allocated
==31887==
==31887== 254 (88 direct, 166 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 10
==31887== at 0x4C272B8: calloc (vg_replace_malloc.c:566)
==31887== by 0x4E375FE: http_parser_headers_complete_cb (http.c:101)
==31887== by 0x4E463ED: http_parser_execute (http_parser.c:1602)
==31887== by 0x4E37D11: request_do_read (http.c:262)
==31887== by 0x4E37F07: request_event_handler (http.c:320)
==31887== by 0x6090CCB: ???
==31887== by 0x5E7F536: ???
==31887== by 0x4E4301E: lcb_wait (wait.c:112)
==31887== by 0x4015A0: schedule_http (main.c:139)
==31887== by 0x401B20: main (main.c:304)
==31887==
==31887== LEAK SUMMARY:
==31887== definitely lost: 88 bytes in 1 blocks
==31887== indirectly lost: 166 bytes in 10 blocks
==31887== possibly lost: 0 bytes in 0 blocks
==31887== still reachable: 1,661 bytes in 7 blocks
==31887== suppressed: 0 bytes in 0 blocks
==31887== Reachable blocks (those to which a pointer was found) are not shown.
==31887== To see them, rerun with: --leak-check=full --show-reachable=yes
==31887==
==31887== For counts of detected and suppressed errors, rerun with: -v
==31887== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 10 from 6)

==6220== HEAP SUMMARY:
==6220== in use at exit: 1,923 bytes in 18 blocks
==6220== total heap usage: 3,349 allocs, 3,331 frees, 427,889 bytes allocated
==6220==
==6220== 262 (88 direct, 174 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 10
==6220== at 0x4C272B8: calloc (vg_replace_malloc.c:566)
==6220== by 0x4E375FE: http_parser_headers_complete_cb (http.c:101)
==6220== by 0x4E4644D: http_parser_execute (http_parser.c:1602)
==6220== by 0x4E37D11: request_do_read (http.c:262)
==6220== by 0x4E37F07: request_event_handler (http.c:320)
==6220== by 0x6090CCB: ???
==6220== by 0x5E7F536: ???
==6220== by 0x4E4307E: lcb_wait (wait.c:112)
==6220== by 0x40158B: schedule_http (main.c:140)
==6220== by 0x401AF5: main (main.c:305)
==6220==
==6220== LEAK SUMMARY:
==6220== definitely lost: 88 bytes in 1 blocks
==6220== indirectly lost: 174 bytes in 10 blocks
==6220== possibly lost: 0 bytes in 0 blocks
==6220== still reachable: 1,661 bytes in 7 blocks
==6220== suppressed: 0 bytes in 0 blocks
==6220== Reachable blocks (those to which a pointer was found) are not shown.
==6220== To see them, rerun with: --leak-check=full --show-reachable=yes
==6220==
==6220== For counts of detected and suppressed errors, rerun with: -v
==6220== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 10 from 6)

[CCBC-129] Add libevent support for win32

Wed, 21 Nov 2012 12:53:20 -0600

I created a NMakefile and made changes to the code to support libevent on windows.

Files affected:
iofactory.c
plugin-libevent.c

New file:
NMakefile.libevent

I uploaded a zipfile of the changes if you are interested in merging them.

[CCBC-128] Allow semi-static bootstrapping

Tue, 5 Mar 2013 16:08:38 -0600

I'm marking this as critical because many folks seem to frown upon the high delays (read http connection latency) when establishing a new couchbase client.

In this case, this would be a constructor option available for lcb_create which will accept an opaque 'map file'. This 'map file' will really just be a dump of a vbucket configuration. What makes this different from compat mode is that the clients will bootstrap when a network error or timeout occurs.

However in the normal model of spawn -> mc operation, where the cluster config is fairly static, there is no need for the latency induced by an extra http connection.

We'd need to implement file-locking semantics to write to this file (we'd maybe just ask for a path to read/write the config to/from, perhaps checking the timestamp, too).. as we'd have multiple clients trying to read/write to this. This is simple enough in Unix

This is in conjunction with a different discussion regarding how to best handle re-bootstrapping, where it was suggested that we really don't need the bootstrap connection in the first place.

There are inherent risks involved with this approach, for example, if one specifies the wrong nodes, then we only know about this during the re-bootstrap.

[CCBC-127] libvbucket RPM is missing /usr/lib64/libvbucket.la file

Fri, 16 Nov 2012 17:51:51 -0600

The libvbucket1 rpm does not include the libvbucket.la file. The libcouchbase.la file references libvbucket.la as a dependency.

Linking via libtool against libcouchbase fails because of this. For example, compiling the php-ext-couchbase package fails like:

/bin/sh /root/code/php-ext-couchbase/libtool --mode=link cc -DPHP_ATOM_INC -I/root/code/php-ext-couchbase/include -I/root/code/php-ext-couchbase/main -I/root/code/php-ext-couchbase -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -I/usr/include/ -DHAVE_CONFIG_H -g -O2 -o couchbase.la -export-dynamic -avoid-version -prefer-pic -module -rpath /root/code/php-ext-couchbase/modules compress.lo couchbase.lo fastlz/fastlz.lo -lz -lcouchbase
grep: /usr/lib64/libvbucket.la: No such file or directory
/bin/sed: can't read /usr/lib64/libvbucket.la: No such file or directory
libtool: link: `/usr/lib64/libvbucket.la' is not a valid libtool archive
make: *** [couchbase.la] Error 1

I created a fake libvbucket.la by copying /usr/lib64/libcouchbase.la and editing it, and it allowed the compile to succeed.

[CCBC-126] Failed to create IO Instance error.

Wed, 21 Nov 2012 07:51:07 -0600

This is the php used when getting the error:

<?php
$cb = new Couchbase("localhost:8091", "Administrator", "password", "default");
$cb->set("a", 1);
var_dump($cb->get("a"));
?>

---------------------------
ERROR
----------------------------

PHP Warning: Couchbase::__construct(): failed to create IO instance in /var/www/test_couchbase.php on line 3

[CCBC-125] Man page for cbc does not describe the new -f argument for cbc hash

Mon, 3 Dec 2012 05:37:18 -0600

Man page for cbc does not describe the new -f argument for cbc hash

[CCBC-124] cbc hash shows incorrect replica bucket host

Wed, 21 Nov 2012 07:51:07 -0600

cbc hash --host <my host> <my key>
"<my key>" vBucket:931 Server:"172.16.10.119:11210" Replicas:"172.16.10.119:11210"

attaching the vbucket map from that host

[CCBC-123] Client does not skip over unreachable nodes

Duplicate — Tue, 13 Nov 2012 04:21:34 -0600

In some recent testing, I found that the client does not skip over the array to find a reachable node. For example when I'm running one on localhost, this fails:

<?php

$hosts = array(
'1.2.3.4',
'127.0.0.1'
);
$client = new Couchbase($hosts, "default", "", "default");

?>

Of course, when I leave out the first one it works. And it works too if I put the localhost before the invalid node.

Here is the libcouchbase error:

Warning: Failed to establish libcouchbase connection to server: Connection failure

Hopefully this should be easy to reproduce!

[CCBC-122] Handle timeout for connecting to a cluster

Duplicate — Thu, 10 Jan 2013 15:53:28 -0600

If we've got a list of servers to connect to we don't try the next server in the list if we're getting a timeout connecting to a host in that list.. We should probably try the next server in the list.. (alternatively we could try all of them in parallel ;))

[CCBC-120] Unhandled opcode=92

Dependency — Wed, 6 Feb 2013 13:48:23 -0600

#0 0x00007ffff6143acf in php_couchbase_error_callback (handle=0x11c0c70, error=LCB_EINTERNAL,
    errinfo=0x7fffffffa2a0 "Unknown implicit send message op=92") at /sources/lcb2.0-repo/php/couchbase.c:724
#1 0x00007ffff5f1aa85 in lcb_error_handler (instance=0x11c0c70, error=LCB_EINTERNAL,
    errinfo=0x7fffffffa2a0 "Unknown implicit send message op=92") at src/error.c:67
#2 0x00007ffff5f28278 in lcb_server_purge_implicit_responses (c=0x15e67d0, seqno=39360, end=1352507659266497000)
    at src/server.c:710
#3 0x00007ffff5f1af11 in parse_single (c=0x15e67d0, stop=1352507659266497000) at src/event.c:135
#4 0x00007ffff5f1b3ca in do_read_data (c=0x15e67d0) at src/event.c:232
#5 0x00007ffff5f1b5a4 in lcb_server_event_handler (sock=14, which=2, arg=0x15e67d0) at src/event.c:298
#6 0x00007ffff5a5dccc in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
#7 0x00007ffff5c95547 in lcb_io_run_event_loop (iops=0x11c0740) at plugins/io/libevent/plugin-libevent.c:320
#8 0x00007ffff5f2aa1c in lcb_wait (instance=0x11c0c70) at src/wait.c:116
#9 0x00007ffff61523f9 in observe_iterate (res=0x11a3c50, ocoll=0x7fffffffa720) at /sources/lcb2.0-repo/php/observe.c:529
#10 0x00007ffff61525b7 in observe_poll (res=0x11a3c50, ocoll=0x7fffffffa720, tprefs=0x7fffffffa700)
    at /sources/lcb2.0-repo/php/observe.c:597
#11 0x00007ffff6153096 in php_couchbase_observe_impl (ht=2, return_value=0x14e2af8, return_value_ptr=0x0, this_ptr=0x11a2f60,
    return_value_used=1, tsrm_ls=0xf53090, multi=1, oo=1, poll=1) at /sources/lcb2.0-repo/php/observe.c:862
#12 0x00007ffff614e089 in zim_couchbase_keyDurabilityMulti (ht=2, return_value=0x14e2af8, return_value_ptr=0x0,
    this_ptr=0x11a2f60, return_value_used=1, tsrm_ls=0xf53090) at /sources/lcb2.0-repo/php/couchbase.c:3468
#13 0x000000000088d5cd in zend_do_fcall_common_helper_SPEC (execute_data=0x7ffff5e96098, tsrm_ls=0xf53090)
    at /sources/php-5.3.17/Zend/zend_vm_execute.h:320
#14 0x000000000088dfe8 in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7ffff5e96098, tsrm_ls=0xf53090)
    at /sources/php-5.3.17/Zend/zend_vm_execute.h:425
#15 0x000000000088c588 in execute (op_array=0x11a3ce8, tsrm_ls=0xf53090) at /sources/php-5.3.17/Zend/zend_vm_execute.h:107
#16 0x0000000000851d7a in zend_execute_scripts (type=8, tsrm_ls=0xf53090, retval=0x0, file_count=3)
    at /sources/php-5.3.17/Zend/zend.c:1236
#17 0x00000000007b5106 in php_execute_script (primary_file=0x7fffffffde10, tsrm_ls=0xf53090)
    at /sources/php-5.3.17/main/main.c:2308
#18 0x0000000000951d89 in main (argc=3, argv=0x7fffffffe078) at /sources/php-5.3.17/sapi/cli/php_cli.c:1189

I don't really have a way to reproduce this right now, I just happened to have been in that situation.

[CCBC-119] Feed a different string (or hash?) for vBucket attribution

Thu, 10 Jan 2013 15:50:34 -0600

We currently are in a situation where we would like to use a different string than the key for hashing and selection of the destination vBucket. This can be useful to keep all of a single user data on many keys but on the same vBucket (and therefore servers), and allow for node downtime to only affect a percentage of the user base instead of the full user base. More generally, it could be used to gain better control over the distribution of data over the nodes.

Looking at the current code, I guess it would be safer not to touch the hashing but just allow for the hashing of a separate string?

[CCBC-118] the lcb_error_t member in the http callbacks shouldn't reflect the HTTP response code

Tue, 13 Nov 2012 04:21:35 -0600

The lcb_error_t member should reflect weather the operation succeeded on a libcouchbase level and if you may look into the http response object. In there you'll find the http return code indicating what the server mean about the request.

[CCBC-117] Cannot compile libev plugin when switching from v3 to latest 4.11

Wed, 6 Feb 2013 13:48:22 -0600

I'm trying to track down the cause of some timeout errors that I am now getting under heavy load since the
"network error" return issue has been resolved and I was able to abandon the use of lcb_wait(). I don't know
if this is due to using an old libev OR these are some commands being lost in ringbuffer logic when I pound it
with tons of lcb_get() calls within a loop (before control is returned to the event loop)

So, easiest step it to try a new libev first before I come here crying :>

When I try to compile the plugin with v4.11 (latest from Feb 2012) it will not compile
unless I add the following

#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#define INVALID_SOCKET -1

into either the header file or the top of the code. And, yes, I did add -DHAVE_LIBEV4 to my gcc line

warning: implicit declaration of function memset
warning: implicit declaration of function fcntl
error: INVALID_SOCKET undeclared (first use in this function)
warning: implicit declaration of function free
etc

Looks like the old libev headers sucked in more includes than they do now.

[CCBC-116] Heavy errors during short network loss

Wed, 14 Nov 2012 11:55:10 -0600

During network loss to the nodes, there are heavy network errors - a combination of both timed-out operations as well as operations which fail with a network error. Might we want to provide an option to retry these operations?

[CCBC-115] Async Mode results in "network error" when many lcb_get() calls are made without use of lcb_wait() within loop

Tue, 13 Nov 2012 04:21:33 -0600

Am receiving "network error" back from error callback when using lcb_get() within a loop, within a function, in libev async mode
(regardless if I make 1 call with a long command list or many calls of 1 command each) after returning control to the
event loop (ie; return back from this function).

If I add an occasional lcb_wait() during some interval of calls before returning back, then this does not happen but, of course, I am blocked
during this call so that is not really a solution.

Appears to be event overflow related but not sure. We are not talking about millions of calls, maybe 100k-ish over the network,
not loopback.

[CCBC-114] Starting with Beta2, only 1/2 my commands return and the others hang or timeout

Tue, 13 Nov 2012 04:21:35 -0600

Ok, I was on a previous beta taken from Sergey's git with libev embeded support. Mostly, things were working fine except having some issues with socket disconnect, etc.

I just examined beta2 from the 12th and I see that not only is all the libev code in it, there are additional changes...so, I grabbed it and am trying to get my test apps to work.

I am building the library

./configure --enable-static --enable-debug --disable-plugins --disable-couchbasemock --disable-tools

and then linking the libev plugin object into my application outside of libcouchbase.

Ok, so first thing I noticed was that:

        memset( &ciops, 0, sizeof( ciops ) );
        ciops.v.v0.type = LCB_IO_OPS_LIBEV;
        ciops.v.v0.cookie = loop;

used to work just fine. Now, it results in a "generic error" being returned from lcb_create_io_ops().

I switched it to

        ciops.version = 1;
        ciops.v.v1.sofile = NULL;
        ciops.v.v1.symbol = "lcb_create_libev_io_opts";
        ciops.v.v1.cookie = loop;

and now lcb_create_io_ops() works as expected. I don't think this is right? It looks like it's not handled properly
in the call where create_v0 -> create_v1 converts the stucture relating to the sofile struct member; however, I could be wrong,
or maybe this is functioning as designed? Don't know.

However, once I got past that - here is the real problem:

Only about 1/2 of my commands (whether it be from store or get) are comming back with LCB_SUCCESS.
If I connect and batch off say 100 calls to get, about 50 will come back LCB_SUCCESS and the other 50
will come back with a timeout error. The errors are NOT inter-mixed with the successful calls...meaning, whichever
50 come back good happen 1st and then the remaining events are all timeout (no mixing/alternating with the successful returns).

This was not the case with the git code I was using previously.

Now, if I do not use lcb_set_timeout(), I don't get the timeout error...it just hangs after the 50 LCB_SUCCESS
returns and no events appear to be firing

Help!!! :<

[CCBC-113] libcouchbase receives assert(relocate_packets: Assertion `!dst->connected' failed.) if server is stopped during active connection

Tue, 13 Nov 2012 04:21:34 -0600

Received:

relocate_packets: Assertion `!dst->connected' failed.
Aborted

When sending commands (store and get) in mass to a connection on which the corresponding beta 2.0 server is shutdown normally.
Basically, just keep sending commands to the connection and shut down the server (only one in the cluster) and you will see this.
Doesn't happen everytime, but most times.

Since my application cannot exit, this is a workstoppage for me.

I am having lots of issues with the library being able to properly handle connecting and disconnecting from a server that is
going up/down via normal command sequence as root.

[CCBC-112] Build source package with at least one event loop plugin unless explicitly building in 'pluingless' mode

Tue, 13 Nov 2012 04:21:35 -0600

When users build from the source, we should ensure that at least one event loop plugin is available for libcouchbase unless a user specifically says not to build any plugins. (i.e. --no-io-plugins)

Otherwise it becomes confusing for users -- a good example is when compiling as a dependency for the php couchbase extension

[CCBC-111] Make iops functions 'private', and discourage calls to run_event_loop/stop_event_loop

Mon, 10 Dec 2012 07:47:33 -0600

There's quite a bit of user code which calls io->run_event_loop and io->stop_event_loop.

These functions are largely private and are intended for use by libcouchbase itself. The fact that they are public API is only accidental, due to the fact that they are part of the iops structure.

These functions should never be called by user code:

1) They are a source of confusion (especially when mixed with lcb_wait)
2) They prevent any kind of wrapping/sanity logic done by lcb_wait (some already exists, and some can be implemented in future versions)

The following should be done

1) Remove the iops structure definition from being included by the main header file. Thus, #include <libcouchbase/couchbase.h> should not allow a dereference of the iops structure. Rather the iops structure should be defined in a new header file (iops.h) - so that users who want to explicitly manipulate and write event loop plugins can include it, but should otherwise not be present in 'user' code. types.h should include a predeclaration of the structure so that it may still be passed around as a simple pointer value.

2) Document the purpose of run_event_loop/stop_event_loop. Make it very clear that lcb_wait should be used in all synchronous use cases. If we have any examples using run_event_loop/stop_event_loop, remove them.

3) [ Optional? ] - In order to further discourage users from using these functions, throw a compiler warning/error for those functions unless LIBCOUCHBASE_INTERNAL is defined - tbd

[CCBC-110] Implement lcb_reconnect() which will reconnect all dead sockets

Tue, 20 Nov 2012 14:40:01 -0600

This functionality very important for long-living connection handles

[CCBC-109] test libev plugin under CI

Tue, 13 Nov 2012 04:21:35 -0600

Since libev support is graduating to production support, we need to be testing it under CI regularly.

[CCBC-108] Allow per-command retries for not-my-vbucket and other transient errors

Mon, 10 Dec 2012 07:47:32 -0600

We should avoid the possibility of 'infinite' errors and allow the user the ability of seeing the 'real' error which caused the command to fail..

http://review.couchbase.org/13177

[CCBC-107] Handle stale/already sent pending commands on network error

Wed, 21 Nov 2012 07:51:07 -0600

http://review.couchbase.org/13667

[CCBC-105] SASL Authentication errors (and possibly others) don't stop the event loop

Dependency — Tue, 13 Nov 2012 04:21:34 -0600

When encountering an authentication error, the client library should break out the event loop after invoking the error callback.

This means that users who are using libcouchbase and are somehow getting SASL auth errors will have their applications hang instantly, unless the user breaks out of the event loop manually from the error callback.

[CCBC-104] Client crashes when entrypoint node connection is dropped

Tue, 20 Nov 2012 20:46:27 -0600

'DelayMin': 1, 'DelayMax': 10}}
[driver driver.py:36] > MC_DS_MUTATE_SET:40 @19 {'DSType': 'DSTYPE_SEEDED', 'DS': {'Count': 1000, 'Repeat': '_REP_', 'VSeed': 'SimpleValue_', 'KSize': 100, 'Continuous': True, 'VSize': 12, 'KSeed': 'SimpleKey_'}, 'Options': {'TimeRes': 1, 'IterWait': True, 'DelayMin': 1, 'DelayMax': 10}}
[driver failover.py:134] Ramp for 3 seconds..
[driver failover.py:141] No service specified..
[driver rest_client.py:710] fail_over successful
sdkd_lcb: src/instance.c:495: relocate_packets: Assertion `ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes)' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff21a8700 (LWP 5726)]
0x00007ffff6e7d475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff6e7d475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff6e806f0 in *__GI_abort () at abort.c:92
#2 0x00007ffff6e76621 in *__GI___assert_fail (
assertion=0x7ffff6a20478 "ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes)",
file=<optimized out>, line=495, function=0x7ffff6a20790 "relocate_packets") at assert.c:81
#3 0x00007ffff6a10724 in vbucket_stream_handler () from /home/mnunberg/src/sdkd-cpp/inst/2.0.0beta//lib/libcouchbase.so.2
#4 0x00007ffff5bc90a4 in event_process_active_single_queue (activeq=0x635250, base=0x63c510) at event.c:1346
#5 event_process_active (base=<optimized out>) at event.c:1416
#6 event_base_loop (base=0x63c510, flags=0) at event.c:1617
#7 0x00007ffff7bbc922 in CBSdkd::Handle::postsubmit (this=0x63c810, rs=..., nsubmit=1) at Handle.cpp:206
#8 0x00007ffff7bbceeb in CBSdkd::Handle::dsMutate (this=0x63c810, cmd=..., ds=..., out=..., options=...) at Handle.cpp:300
#9 0x00007ffff7bc8ab1 in CBSdkd::WorkerDispatch::_process_request (this=0x63c230, req=..., rs=0x63c420) at IODispatch.cpp:494
#10 0x00007ffff7bc93d5 in CBSdkd::WorkerDispatch::run (this=0x63c230) at IODispatch.cpp:572
#11 0x00007ffff7bc7276 in CBSdkd::new_worker_thread (worker=0x63c230) at IODispatch.cpp:227
#12 0x00007ffff71d8b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#13 0x00007ffff6f2370d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#14 0x0000000000000000 in ?? ()
(gdb) [driver rest_client.py:710] fail_over successful
[driver failover.py:149] Sleeping for 3 seconds after failover
[driver driver.py:36] > CANCEL:41 @2 {}

[CCBC-103] Window plugins hangs when connecting to a couchbase cluster with 2 or more nodes

Tue, 13 Nov 2012 04:21:35 -0600

When running the event loop from a window build it was observed that lcb_io_run_event_loop would never complete when performing a get or set operation. lcb_wait was called after calling lcb_store or lcb_get and would never complete.

This issue was seen after adding a second server to the couchbase cluster.

I tracked down the issue to link_event and unlink_event in plugin-win32.c

Link event was not inserting new events into the linked link properly after the first 2 it would trash the second entry with any addition adds.

In unlink_event the loop would did not properly remove events from the linked list and would connect the head to the next pointer of the event being deleted skipping over any other events in the list.

Here is the fixed code: (link_timer and unlink_timer had the same logic so I updated those also)

static void link_event(struct winsock_io_cookie *instance,
                       struct winsock_event *event)
{
    event->next = instance->events;
instance->events = event;
}

static void unlink_event(struct winsock_io_cookie *instance,
                         struct winsock_event *event)
{
if (instance->events == NULL)
return;

    if (instance->events == event) {
        instance->events = event->next;
    } else {
        struct winsock_event *prev = instance->events;
        struct winsock_event *next = prev->next;
        while (next != NULL) {
            if (event == next) {
                prev->next = next->next;
                return;
            }
prev = next;
next = prev->next;
        }
    }
}

static void link_timer(struct winsock_io_cookie *instance,
                       struct winsock_timer *timer)
{
    timer->next = instance->timers;
instance->timers = timer;
}

static void unlink_timer(struct winsock_io_cookie *instance,
                         struct winsock_timer *timer)
{
if (instance->timers == NULL)
return;

    if (instance->timers == timer) {
        instance->timers = timer->next;
    } else {
        struct winsock_timer *prev = instance->timers;
        struct winsock_timer *next = prev->next;
        while (next != NULL) {
            if (timer == next) {
                prev->next = next->next;
                return;
            }
prev = next;
next = prev->next;
        }
    }
}

[CCBC-102] event sample does not compile across all platforms

Tue, 13 Nov 2012 04:21:34 -0600

A recent change broke compilation owing to some libevent looking change.

Where it was found to be broken:
http://sdkbuilds.couchbase.com/job/libcouchbase/237/console
This change:
https://github.com/couchbase/libcouchbase/commit/f82c93e018d723ab2adb36bd3dc94e4b9f3a11f8

[CCBC-101] allow google test suite to be preserved between invocations of make distclean and friends

Tue, 18 Sep 2012 11:35:28 -0500

additionally we should make it into a DSO and link the tests against that.. would make it somewhat quicker..

This is obviously not a major issue.. but would be nice :)

[CCBC-100] Allow Incremental Row Callbacks

Sat, 8 Dec 2012 00:06:11 -0600

View query results may return large amounts of data. We should offer a row callback which returns a JSON string whose contents is a single row.

There are awesome JSON libraries to do this task, and it's something that I've found myself doing in the Perl client.. so why not move it into libcouchbase and let everyone be happy? :).

Basic idea:
Two callbacks are put into place. The first is a 'row callback' which receives rows, and the second is a complete callback, at which time any other non-row data is returned as a JSON object.

So for example

{ "total_rows" : 30,
  "rows" : [
   { "id" : "foo" },
   { "id" : bar" },
     ....

The row callback will first receive the string '{ "id" : "foo" }', then the second time around, '{"id":"bar"}'
The complete callback will return:
' { "total_rows" : 30 } '

[CCBC-99] fix libcouchbase-10 build scripts to remove packages first

Tue, 13 Nov 2012 04:21:34 -0600

I ran into a build failure earlier today, and the root cause was a 1.1/2.0 libcouchbase package being installed. At some point, we need to clean up the php builds and libcouchbase buidls to be more stable.

[CCBC-98] Differentiate between TMPFAILs

Dependency — Thu, 27 Sep 2012 16:06:52 -0500

Differentiate between TMPFAIL error messages received becuase of lack of vbucket config, and TMPFAIL messages actually returned by the server.

This is confusing to the user, as he does not know whether there is a server issue, a command issue, or a usage issue.

[CCBC-97] libmemcached: doublefree

Tue, 13 Nov 2012 04:21:34 -0600

I know this is the wrong component, we don't have one for libmemcached. This is version 0.44

Customer experienced:
*** glibc detected *** /usr/bin/python: double free or corruption (!prev): 0x0000000002d2d300 ***
======= Backtrace: =========
/lib/libc.so.6(+0x775b6)[0x7f89d5d785b6]
/lib/libc.so.6(cfree+0x73)[0x7f89d5d7ee83]
/usr/lib/libmemcached.so.6(+0x57d3)[0x7f89d38417d3]
/usr/lib/libmemcached.so.6(+0xa6e0)[0x7f89d38466e0]
/usr/lib/libmemcached.so.6(+0x9a61)[0x7f89d3845a61]
/usr/lib/libmemcached.so.6(+0x9c28)[0x7f89d3845c28]
/usr/lib/libmemcached.so.6(+0xaa46)[0x7f89d3846a46]
/usr/lib/libmemcached.so.6(+0xb531)[0x7f89d3847531]
/usr/lib/libmemcached.so.6(memcached_fetch_result+0x63)[0x7f89d3842be3]
/usr/lib/libmemcached.so.6(memcached_fetch+0x4a)[0x7f89d3842d7a]
/usr/lib/libmemcached.so.6(memcached_get_by_key+0x80)[0x7f89d3843ef0]
/usr/lib/libmemcached.so.6(memcached_get+0x1f)[0x7f89d38440df]
/usr/local/lib/python2.6/dist-packages/pylibmc-1.1.1-py2.6-linux-x86_64.egg/_pylibmc.so(+0x56da)[0x7f89d3a566da]
/usr/bin/python(PyEval_EvalFrameEx+0x59be)[0x4a84ae]
/usr/bin/python(PyEval_EvalFrameEx+0x5a60)[0x4a8550]
/usr/bin/python(PyEval_EvalCodeEx+0x911)[0x4a9671]
/usr/bin/python(PyEval_EvalFrameEx+0x4d19)[0x4a7809]
/usr/bin/python(PyEval_EvalFrameEx+0x5a60)[0x4a8550]
/usr/bin/python(PyEval_EvalCodeEx+0x911)[0x4a9671]
/usr/bin/python[0x53771d]
/usr/bin/python(PyObject_Call+0x47)[0x41f0c7]
/usr/bin/python(PyEval_EvalFrameEx+0x3861)[0x4a6351]
/usr/bin/python(PyEval_EvalCodeEx+0x911)[0x4a9671]
/usr/bin/python(PyEval_EvalFrameEx+0x4d19)[0x4a7809]
/usr/bin/python(PyEval_EvalFrameEx+0x5a60)[0x4a8550]
/usr/bin/python(PyEval_EvalCodeEx+0x911)[0x4a9671]
/usr/bin/python[0x537620]
/usr/bin/python(PyObject_Call+0x47)[0x41f0c7]
/usr/bin/python[0x427dff]
/usr/bin/python(PyObject_Call+0x47)[0x41f0c7]
/usr/bin/python(PyEval_CallObjectWithKeywords+0x43)[0x4a1b03]
/usr/bin/python[0x4d7d3d]
/lib/libpthread.so.0(+0x69ca)[0x7f89d6f0d9ca]
/lib/libc.so.6(clone+0x6d)[0x7f89d5de770d]

[CCBC-96] POST/PUT requests not properly being appended to the buffer

Thu, 27 Sep 2012 16:06:52 -0500

From tcpdump:

PUT /default/_design/lcb_design_doc HTTP/1.1
User-Agent: libcouchbase/1.1.0dp9
Accept: application/json
Host: 10.3.121.209:8092
Co{ "id" : "_design/lcb_design_doc", "language" : "javascript", "views" : { "lcb-test-view" : "function(doc) { if (doc.testid == 'lcb') { emit(doc.id) } } " } }

- Content-Length gets truncated and replaced with the POST data.

[CCBC-95] Add TimS content/email on Views for Nielson to 2.0

Thu, 18 Oct 2012 15:04:22 -0500

Add TimS info into Dev Guide on Indexing and querying for Nielson.

[CCBC-94] NMakefile is missing in the libvbucket package from C Client Library

Tue, 13 Nov 2012 04:21:34 -0600

NMakefile is missing from the libvbucket package downloaded at the following link:
http://packages.couchbase.com/clients/c/libvbucket-1.8.0.4.tar.gz

As per the instructions provided in the link below:
http://www.couchbase.com/docs/couchbase-sdk-c-1.0/c-install-windows.html

The NMake file in the libvbucket needs to be edited for developing in Windows, Which is not available.

[CCBC-92] Memory leak when using more than 1 server

Tue, 13 Nov 2012 04:21:35 -0600

There is a memory leak in server.c file. In the function libcouchbase_purge_single_server ring buffer context "rest" initialized but never freed. Do call libcouchbase_ringbuffer_destruct(&rest) at the end of the function.

[CCBC-91] timeouts seen after failover, rebalance and add back

Tue, 13 Nov 2012 04:21:35 -0600

scenario:
1. Start PHP client in a loop setting and getting against a 2 node cluster
2. Click failover to kick a node out, click rebalance to make it unassociated
3. Walk through the setup wizard on that node, re-add it to the cluster
4. After adding, click rebalance

Expected behavior:
During rebalance in step 4, which is an add node scenario, no timeouts are expected.

Observed behavior:
During rebalance in step 4, I see timeouts from PHP, and they continue even after the rebalance has completed.

[CCBC-90] library crashes in instance.c libcouchbase_switch_to_backup_node.

Tue, 13 Nov 2012 04:21:34 -0600

library crashes when accessing into allocated memory at instance->backup_nodes[instance->backup_idx].

instance->backup_nodes[instance->backup_idx] is not necessarily NULL when instance->backup_idx is >= instance->nbackup_nodes.

This was fixed by changing line 729 from:

    if (instance->backup_nodes[instance->backup_idx] == NULL) {
        --instance->backup_idx;
        libcouchbase_error_handler(instance, error, reason);
        return -1;
    }

to:

    if (instance->backup_idx >= instance->nbackup_nodes || instance->backup_nodes[instance->backup_idx] == NULL) {
        --instance->backup_idx;
        libcouchbase_error_handler(instance, error, reason);
        return -1;
    }

This prevents indexing past the number of backup_nodes that have been allocated.

[CCBC-89] Memory leak in http interface

Tue, 13 Nov 2012 04:21:34 -0600

we don't release the memory for the parser

[CCBC-88] Observe: Segmentation fault

Tue, 13 Nov 2012 04:21:35 -0600

When trying to observe a document a Segmentation Fault breaks the library.

//---------------------------------------------------------------------------------------------------------------------------
bool MyClass::observe(char *str)
{
    char *keysList[1] = {0}; // Keys list.
    size_t keysSizes[1] = {0}; // Keys sizes list.
    libcouchbase_error_t oprc = LIBCOUCHBASE_SUCCESS; // Return control.

    // Observe.
    keysList[0] = str;
    keysSizes[0] = strlen(str);
    oprc = libcouchbase_observe(lcb, this, 1, (const void*const*)keysList,
                                keysSizes);
    if(oprc != LIBCOUCHBASE_SUCCESS) {
        return false;
    }

    // Force wait.
    lcbIO->run_event_loop(lcbIO);

   ..........
}
//---------------------------------------------------------------------------------------------------------------------------

//---------------------------------------------------------------------------------------------------------------------------
Command:
./cbc observe -h192.168.0.101 -bvp_transactions -uwarp -Ptesttest 0001

Result:
Segmentation fault (core dumped)

GDB LOG ATTACHED.
//---------------------------------------------------------------------------------------------------------------------------

//---------------------------------------------------------------------------------------------------------------------------
$ ./cbc cat -h192.168.0.101 -bvp_transactions -uwarp -Ptesttest 0001
"0001" Size:2 Flags:0 CAS:57eb22b63bfcae12
{}
//---------------------------------------------------------------------------------------------------------------------------

[CCBC-87] error codes need additional documentation in types.h and a reference from couchbase.h

Dependency — Tue, 13 Nov 2012 04:21:33 -0600

Some error codes, like LIBCOUCHBASE_ETMPFAIL, are defined to be returned for certain functions, but other functions don't tell users what may be returned from the header file or other documentation. It would be good to add this to the headers, but needs to be documented in some way.

[CCBC-86] libvbucket/libcouchbase must not assert on invalid maps

Tue, 13 Nov 2012 04:21:34 -0600

SUBJ

See http://www.couchbase.com/issues/browse/MB-5893

Even if input is invalid there's needs to be some sensible behavior.

[CCBC-85] Implement general purpose timers

Dependency — Tue, 13 Nov 2012 04:21:33 -0600

Just expose timers from underlying event IO engine

[CCBC-84] Updated screencast for /develop pages

Thu, 12 Jul 2012 00:56:01 -0500

Create an updated screencast to ship with the new 2.0 developer SDKs

[CCBC-83] Update the getting started to match the Beer sample DB

Thu, 18 Oct 2012 11:53:21 -0500

Once the sample DB has been integrated into the server, update the getting started (both on the web page and in the documentation) to use the sample database.

[CCBC-82] Replica read command

Duplicate — Tue, 13 Nov 2012 04:21:35 -0600

Implement a simple replica read command

[CCBC-81] Implement observe command

Duplicate — Tue, 13 Nov 2012 04:21:35 -0600

Add the basic low level observe command

[CCBC-80] Connection fails with IPV6 system

Thu, 27 Sep 2012 16:06:52 -0500

When a hostname request returns an IPV6 address the client fails to connect with LIBCOUCHBASE_CONNECT_ERROR

[CCBC-79] Need RPMs for CentOS/RedHat

Duplicate — Tue, 13 Nov 2012 04:21:34 -0600

Cleaning up the getting started and tutorial. Right now it is blank/ no command provided:

http://www.couchbase.com/docs/couchbase-sdk-c-1.0/installation_packages_linux_rh.html

Let me know and I will add.

[CCBC-78] Improved Sample Code for "Simple Example"

Thu, 5 Jul 2012 16:25:15 -0500

Hi,

I received a variety of content corrections from a customer for the C SDK. I made all possible, however some require changes in the sample code the SDK Team provides. The existing sample code is here:

http://www.couchbase.com/docs/couchbase-sdk-c-1.0/simple_example.html

The customer requests:

- The tutorial example bails from callbacks with exit(), rather than
passing control back to main (after setting a global flag, for example,
or one passed in via the cookie, etc.).
- The tutorial should also show use of libcouchbase_remove() and what to
expect from libcouchbase_mget() afterwards (the latter is in the Python
docs, but not the C docs).
- The tutorial should use libcouchbase_destroy().
- Some mention of whether a disconnect is needed or not would be
appropriate.

Let me know if you are able to demonstrate use of these other methods in the sample code and send the code. I will add the new sample and document.

Also let me know if a C client needs to be destroyed and if so how.

[CCBC-77] The client uses the same error code for server's and its own OOM errors

Tue, 13 Nov 2012 04:21:33 -0600

The function libcouchbase_error_t map_error(protocol_binary_response_status) which maps protocol error codes to client's:
https://github.com/couchbase/libcouchbase/blob/master/src/handler.c#L38

One of the cases when the client returns the same error code when it cannot allocate memory:
https://github.com/couchbase/libcouchbase/blob/master/src/handler.c#L128

[CCBC-76] libcouchbase calling connect(2) on each spooled command

Tue, 13 Nov 2012 04:21:33 -0600

If the server is not connected, a large spool operation will call connect() many times. For one, this makes it difficult to debug, and performs a lot of un-necessary system calls..

strace output and sample script attached

[CCBC-75] assertion failure in relocate_packets

Tue, 13 Nov 2012 04:21:33 -0600

#3 0x00002b0fe1766475 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00002b0fe17696f0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x00002b0fe175f621 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x00002b0fe8bdfaf7 in relocate_packets (dst_instance=<optimized out>, src=<optimized out>) at src/instance.c:387
#7 libcouchbase_update_serverlist (instance=<optimized out>) at src/instance.c:447
#8 vbucket_stream_handler (sock=<optimized out>, which=<optimized out>, arg=<optimized out>) at src/instance.c:820
#9 0x00002b0feac6e344 in event_base_loop () from /usr/lib/libevent-1.4.so.2

[CCBC-74] Getting SIGABRT from libcouchbase

Thu, 27 Sep 2012 16:06:52 -0500

From: http://www.couchbase.com/forums/thread/getting-sigabrt-libcouchbase

Please follow up with this thread upon resolution.

I was running through my program and suddenly caught SIGABRT while using libcouchbase.
This was what I was getting from GDB Debugging
I've wrapped your libcouchbase in order to have similar interface with JAVA.
347 RSLTCODE CMBClient::Set( const uint64_t cas, const char* key, const char* value, const time_t* expiry)
348 {
349 libcouchbase_error_t error = LIBCOUCHBASE_SUCCESS;
350 libcouchbase_error_t callback_error = LIBCOUCHBASE_SUCCESS;
351 error = libcouchbase_store ( m_instance , (void*)&callback_error, LIBCOUCHBASE_SET, key, strlen(key), value, strlen(value), 0, (libcouchbase_time_t)*expiry, cas);
352 if ( error != LIBCOUCHBASE_SUCCESS )
353 {
354 fprintf(stderr, "Set Failed\n");
355 return CheckError(error);
356 }
357
358 libcouchbase_wait (m_instance);
359
360 return CheckError(callback_error);
361 }
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff36a2700 (LWP 16100)]
0x00007ffff6e20445 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff6e20445 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff6e23bab in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff76c4c0d in libcouchbase_ringbuffer_consumed (buffer=, nb=56) at src/ringbuffer.c:243
#3 0x00007ffff76c017b in do_send_data (c=) at src/event.c:264
#4 libcouchbase_server_event_handler (sock=, which=, arg=0x6ab660) at src/event.c:314
#5 0x00007ffff60b5f53 in event_base_loop () from /usr/lib/libevent-1.4.so.2
#6 0x0000000000418588 in CMBClient::Set (this=0x7fffffffe1e0, cas=0, key=0x7ffff36a1da0 "172.16.0.116:9999.PCCNT", value=0x7ffff36a1de0 "1", expiry=0x7ffff36a1d90) at CMBClient.cpp:358
#7 0x0000000000408354 in PCNetMgr::checkAndUpdateSSCnt (this=0x7fffffffe1a8) at PCNetMgr.cpp:59
#8 0x000000000040a0ed in PCBatchJob::run (this=) at PCBatchJob.cpp:11
#9 0x000000000043620a in doWork (pThreadInfo=) at ThreadPool.cpp:211
#10 0x00007ffff7bc4e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#11 0x00007ffff6edc4bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x0000000000000000 in ?? ()

[CCBC-73] Command line parser from cbc tool leaks

Wed, 6 Feb 2013 13:48:20 -0600

See execution log here: https://gist.github.com/2909622

[CCBC-72] Compilation failure on macosx if you've got gtest installed

Tue, 13 Nov 2012 04:21:33 -0600

If we're trying to overload CPPFLAGS within an ifdef it doesn't inherit the default value for the else clause (if its not set)

[CCBC-71] Implement 'help' command for cbc

Tue, 13 Nov 2012 04:21:33 -0600

Currently to invoke list of commands user should call if without parameters or with bad parameter. It would be nice to have builtin 'help', which allows to print list of supported commands.

[CCBC-70] Missing dlcose() for dlopen() in create_func get_create_func results in unfreed memory at shutdown

Tue, 13 Nov 2012 04:21:33 -0600

In create_func get_create_func(), there is a dlopen() handle which only gets closed upon error - otherwise
it doesn't get closed by libcouchbase_destroy(handle) and results in valgrind complaining with

==30571== 32 bytes in 1 blocks are still reachable in loss record 1 of 1
==30571== at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==30571== by 0x3D76A0130F: _dlerror_run (in /lib64/libdl-2.12.so)
==30571== by 0x3D76A00EE0: dlopen@@GLIBC_2.2.5 (in /lib64/libdl-2.12.so)
==30571== by 0x4C1F88A: ??? (in /usr/lib64/libcouchbase.so.1.2.2)
==30571== by 0x4C1F90A: libcouchbase_create_io_ops (in /usr/lib64/libcouchbase.so.1.2.2)
==30571== by 0x4010A2: main (foo.c:75)

specifically:

void *dlhandle = dlopen(image, RTLD_NOW | RTLD_LOCAL);

[CCBC-69] Invoke auto-tools from autoreconf instead of manually trying to hunt them down

Tue, 13 Nov 2012 04:21:35 -0600

Invoke auto-tools from autoreconf instead of manually trying to hunt them down

[CCBC-68] add getl/unl commands

Tue, 13 Nov 2012 04:21:34 -0600

Currently, libcouchbase does not have support for either the "getl" command or the "unl" command. Both are defined and supported by the server and should be added to this client.

[CCBC-67] Use google test framework for unit testing

Tue, 13 Nov 2012 04:21:35 -0600

Use google test framework for unit testing

[CCBC-66] Add support for using gcov/tcov to get code coverage

Tue, 13 Nov 2012 04:21:34 -0600

Add support for using gcov/tcov to get code coverage

[CCBC-65] Memory leak in libcouchbase_create / libcouchbase_destroy (POC and patch included)

Tue, 13 Nov 2012 04:21:34 -0600

When the client calls libcouchbase_create and then later calls libcouchbase_destroy, some memory is leaked. For a client with many threads that open and close couchbase sessions, this will lead to memory exhaustion. The leak is caused by the use of strdup() in setup_boostrap_hosts(). These duplicated strings are not free'd in libcouchbase_destroy. The leak can be seen by using valgrind.

I have attached a proof of concept, and a patch with a proposed fix.

I have only tested on 1.0.2, but other versions may be affected.

[CCBC-64] PHP Client reports Time Out errors on simple SET and GET operations when Rebalance is in Progress

Tue, 13 Nov 2012 04:21:33 -0600

This bug is reported by a customer. Whenever they issue a rebalance(after removing a node from the cluster or add a new node), client operations(PHP - sample code attached) time out with the error below

PHP Warning: Couchbase::set(): Failed to store a value to server: Operation timed out in /root/test_worker.php on line <n>

This issue is consistent for both the customer as well as our in-house reproduction in Support. There are no errors on the server side. Rebalance does complete. And once rebalance is complete, the same script runs fine without any errors.

[CCBC-63] error: 'ssize_t' does not name a type (missing stddef.h)

Thu, 27 Sep 2012 16:06:52 -0500

In trying to compile a Qt c++ based application, the compiler cannot found a header file: stddef.h

Error log:
In file included from /opt/libcouchbase/include/libcouchbase/couchbase.h:29:0,
from ../../src/connector/vp_connector.cpp:8:
/opt/libcouchbase/include/libcouchbase/configuration.h:57:13: error: 'ssize_t' does not name a type
In file included from /opt/libcouchbase/include/libcouchbase/couchbase.h:31:0,
from ../../src/connector/vp_connector.cpp:8:
/opt/libcouchbase/include/libcouchbase/types.h:206:31: error: expected identifier before '*' token
/opt/libcouchbase/include/libcouchbase/types.h:210:47: error: ISO C++ forbids declaration of 'libcouchbase_ssize_t' with no type [-fpermissive]
/opt/libcouchbase/include/libcouchbase/types.h:210:47: error: 'libcouchbase_ssize_t' declared as function returning a function
......

It should be changed to unistd.h... at less in linux systems.

[CCBC-62] hang in libcouchbase_wait() after get timeout

Tue, 13 Nov 2012 04:21:33 -0600

When one node is down and dropping packets, a store op trying to write to that node times out (OK) but libcouchbase_wait doesn't return (not OK).

[root@localhost cb-crank]# gcc -g -Wall -W -I$I/include -L$I/lib t.c -lcouchbase -o t
[root@localhost cb-crank]# export LD_LIBRARY_PATH=$I/lib
[root@localhost cb-crank]# ./t
Stored: hello (6)
Got: hellow (7) = world! (7)
[root@localhost cb-crank]# ./t hellow foo
Stored: hellow (7)
Got: hellow (7) = foo (4)
[root@localhost cb-crank]# ./t xyz
Got: hellow (7) = foo (4)
Failed during store 'xyz': Operation timed out

Setup: Start two nodes of Couchbase Server. In my test, they're on 10.4.2.13 and 10.4.2.14. Client (see attached code) is being run on host 10.4.2.14.

On one node (10.4.2.13), simulate a network problem with iptables:

iptables -I INPUT 1 -p tcp --sport 1000:60000 -j DROP
iptables -I INPUT 2 -p tcp --dport 1000:60000 -j DROP

Run the sample app a few times with different keys until you hit one that needs to access the down node. As seen in the output above, the storage callback is called with a timeout error, but the libcouchbase_wait() call never returns. Attach debugger to the client and view thread stacks:

# gdb -p 21746
GNU gdb (GDB) CentOS (7.0.1-42.el5.centos)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Attaching to process 21746
Reading symbols from /root/code/cb-crank/t...done.
Reading symbols from /root/code/install/lib/libcouchbase.so.1...done.
Loaded symbols for /root/code/install/lib/libcouchbase.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /root/code/install/lib/libvbucket.so.1...done.
Loaded symbols for /root/code/install/lib/libvbucket.so.1
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /root/code/install/lib/libcouchbase_libevent.so.1...done.
Loaded symbols for /root/code/install/lib/libcouchbase_libevent.so.1
Reading symbols from /opt/couchbase/lib/libevent-2.0.so.5...(no debugging symbols found)...done.
Loaded symbols for /opt/couchbase/lib/libevent-2.0.so.5
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff822c0000
0x0000003407cd3603 in __epoll_wait_nocancel () from /lib64/libc.so.6
(gdb) where
#0 0x0000003407cd3603 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1 0x00002b0211713c28 in ?? () from /opt/couchbase/lib/libevent-2.0.so.5
#2 0x00002b0211702a4c in event_base_loop () from /opt/couchbase/lib/libevent-2.0.so.5
#3 0x0000000000400d60 in main (argc=1, argv=0x7fff822027f0) at t.c:113
(gdb) thread apply all bt

Thread 1 (Thread 0x2b02114eeaf0 (LWP 21746)):
#0 0x0000003407cd3603 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1 0x00002b0211713c28 in ?? () from /opt/couchbase/lib/libevent-2.0.so.5
#2 0x00002b0211702a4c in event_base_loop () from /opt/couchbase/lib/libevent-2.0.so.5
#3 0x0000000000400d60 in main (argc=1, argv=0x7fff822027f0) at t.c:113
(gdb) quit

Line t.c:113 is libcouchbase_wait().

Client compiled as:

gcc -g -Wall -W -I$I/include -L$I/lib t.c -lcouchbase -o t

[CCBC-61] pillowfight missing error variable assignment

Tue, 13 Nov 2012 04:21:34 -0600

https://github.com/couchbase/libcouchbase/blob/master/example/pillowfight.cc#L201

            if (libcouchbase_enable_timings(instance) != LIBCOUCHBASE_SUCCESS) {
                std::cerr << "Failed to enable timings!: "
                          << libcouchbase_strerror(instance, error) << std::endl;
                timings = false;
            }

Missing error = assignment here, so the error message printed here will be incorrect.

[CCBC-60] support errors on views

Thu, 27 Sep 2012 16:06:52 -0500

Views now have some options on capturing errors and stopping on errors as documented here:
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-views-writing-errors.html

The client needs to support these.

[CCBC-59] libcouchbase_wait() blocks forever if there is nothing queued

Tue, 13 Nov 2012 04:21:35 -0600

Very simply, if you call libcouchbase_wait() without queuing some operation first (e.g store a key/value, arithmetic, etc...)
the call to libcouchbase_wait() never returns.

This means that a single threaded application that calls libcouchbase_wait() at different times to make sure operations are sent to the server runs the risk of stalling indefinitely. This is a very likely scenario.

I've attached a simple proof of concept, which never completes. If you uncomment the store operation, the program completes successfully.

[CCBC-58] add support for unlock command

Thu, 27 Sep 2012 16:06:52 -0500

Couchbase server has a keyed unlock command, which goes along with the getl command. libcouchbase needs to support this since it's also needed for Ruby and PHP clients.

[CCBC-57] yum repository doesn't work because packages are not signed

Duplicate — Tue, 13 Nov 2012 04:21:33 -0600

Set this up on a development system earlier, and look at the last line of output:

$ sudo yum install libvbucket-debuginfo
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libvbucket-debuginfo.x86_64 0:1.8.0.3-1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
libvbucket-debuginfo x86_64 1.8.0.3-1 couchbase 59 k

Transaction Summary
================================================================================
Install 1 Package(s)
Upgrade 0 Package(s)

Total size: 59 k
Is this ok [y/N]: y
Downloading Packages:

Package libvbucket-debuginfo-1.8.0.3-1.x86_64.rpm is not signed

[CCBC-56] Update instructions for building libcouchbase to be more verbose

Mon, 6 May 2013 02:50:11 -0500

http://www.couchbase.com/docs/couchbase-sdk-c-1.0/linux_and_mac.html

This guy was a little bit confused about how to build because the first step wasn't to run config/autorun.sh. We should add that to the documentation.

Also, I think we should add the stuff that is in the comments to the docs. We should give the locations of where the downloads are. There is also no mention of needing memcached installed. This is a requirement since libcouchbase depends on memcached/protocol_binary.h.

[CCBC-55] support for replica read

Duplicate — Tue, 13 Nov 2012 04:21:34 -0600

To support replica read with 1.8.1 server, we need to add libcouchbase support for replica reads.

[CCBC-54] support new stale=false parameter

Tue, 13 Nov 2012 04:21:34 -0600

Starting with 2.0 DP4, the default for stale has changed to update_after and there is a new argument to the stale parameter, "false" which performs a view update before response. The client needs to support this new argument to the stale parameter.

[CCBC-53] make code examples copyable

Tue, 13 Nov 2012 04:21:33 -0600

It would be very nice if programlisting sections were copy/pasteable. I think to get there we'd have to do formatting with line numbers a bit differently, but it'd be worthwhile. An example is: http://www.couchbase.com/docs/couchbase-sdk-c-1.0/hello_couchbase.html

Tracking this as backlog.

[CCBC-52] code sample layout odd

Tue, 13 Nov 2012 04:21:33 -0600

On pages such as http://www.couchbase.com/docs/couchbase-sdk-c-1.0/hello_couchbase.html the code samples don't line up correctly. Note how line 1 is different.

Is it possible to make these align?

[CCBC-51] In store.c :67 libcouchbase_store_by_key does not check for negative value for idx, causing seg fault when nodes are down

Tue, 13 Nov 2012 04:21:35 -0600

I had 3 server nodes, 2 of which were down.

When I ran my test program (to store a value), instead of reporting an error or using the remaining node, the program seg faulted.
I traced this back to store.c :66, where vbucket_map(instance->vbucket_config, hashkey, nhashkey, &vb, &idx)
returned -1 for the value idx (the index into the vbucket); this probably indicates that there was no valid index.
The next line, 67, uses idx to reference a server struct. Using -1 causes an invalid memory access and the program crashes with a seg fault.

server = instance->servers + (size_t)idx;

There should be a statement to check the value idx is non negative before using it:

[CCBC-50] packaging is broken by removal of "g" from RPM

Tue, 13 Nov 2012 04:21:35 -0600

This was originally covered in ~~MB-4647~~ for libvbucket, but affects libcouchbase as well.

The dist-rpm task depends on dist, which creates a tarball with a "g" in it. The packaging fails with current.

[CCBC-49] API reference is complete

Mon, 10 Dec 2012 07:47:32 -0600

API reference is complete

[CCBC-48] tutorial chapter is complete

Duplicate — Tue, 25 Sep 2012 18:08:23 -0500

tutorial chapter is complete

[CCBC-47] getting started screencast is available

Tue, 13 Nov 2012 04:21:35 -0600

getting started screencast is available

[CCBC-46] getting started is also on the web page

Tue, 13 Nov 2012 04:21:35 -0600

getting started is also on the web page

[CCBC-45] getting started guide needed

Tue, 13 Nov 2012 04:21:34 -0600

getting started guide needed

[CCBC-44] deploying to download locations directly from CI

Tue, 13 Nov 2012 04:21:33 -0600

Idea here is that we have repos for both unreleased, but code reviewed builds and released builds.

[CCBC-43] building shippable artifacts under CI

Tue, 13 Nov 2012 04:21:35 -0600

building shippable artifacts under CI

[CCBC-42] running integration tests under CI

Tue, 13 Nov 2012 04:21:35 -0600

running integration tests under CI

[CCBC-41] running unit tests under CI

Tue, 13 Nov 2012 04:21:34 -0600

running unit tests under CI

[CCBC-40] building under CI

Thu, 12 Apr 2012 15:33:46 -0500

building under CI

[CCBC-39] redistributing operations doesn't take hashkey from store_by_key into account

Tue, 13 Nov 2012 04:21:34 -0600

See http://review.couchbase.org/12135 for a more detailed description of the issue.

[CCBC-38] need to connect to alternate nodes if the REST connection to an existing node is interrupted

Tue, 13 Nov 2012 04:21:34 -0600

Currently, the library won't connect to alternate nodes if the connection to the primary node fails. It needs to learn to maintain a couple of lists. The configured list, and the full list of nodes. In the event that the connection is dropped, work through the appropriate list to get a config from a node of the cluster.

[CCBC-37] libcouchbase examples should use envvars to simplify example execution

Tue, 13 Nov 2012 04:21:34 -0600

When running things like stats or pillowfight, it'd be handy if there were envvars to allow the user to not need to type their cluster URI, bucketname, password.

Suggested:
COUCHBASE_CLUSTER_URI="http://myserver:8091"
COUCHBASE_BUCKETNAME="mybucket"
COUCHBASE_PASSFILE=/path/to/password

The passfile must be permissions 600 and owned by the user attempting to run the utility.

It'd be valid to have just the CLUSTER_URI defined, and in that case the utility in question would default to the "default" bucket with no password.

[CCBC-36] libcouchbase error when trying to access SASL password protected bucket

Wed, 15 Feb 2012 00:55:34 -0600

Attached is a sample C program that reproduces the problem, and from the customer:
In the instance I have currently running, one of the threads is stalled for several minutes in a call to the couchbase library. There's the traceback.
Thread 7 (Thread 0xb5743b90 (LWP 9349)):
#0 0x00e45402 in __kernel_vsyscall ()
#1 0x0023c2c6 in epoll_wait () from /lib/libc.so.6
#2 0x007b6d70 in epoll_dispatch (base=0x94297b0, tv=0x0) at epoll.c:404
#3 0x007a40bb in event_base_loop (base=0x94297b0, flags=0) at event.c:1537
#4 0x00168ad8 in libcouchbase_io_run_event_loop (iops=0x94229e8) at src/plugin-libevent.c:269
#5 0x00ac1fc4 in libcouchbase_wait (instance=0x9438b68) at src/wait.c:48
#6 0x08049552 in CouchbaseGetExplicitCas ()
#7 0x08049701 in CouchbaseGetExplicit ()
#8 0x0804974c in CouchbaseGet ()
#9 0x08049e83 in sessionRun ()
#10 0x08049edd in MainLoop ()
#11 0x006e345b in start_thread () from /lib/libpthread.so.0
#12 0x0023bc4e in clone () from /lib/libc.so.6

[CCBC-35] create basic doc structure for libcouchbase

Wed, 15 Feb 2012 00:56:20 -0600

While we have the info, we don't currently emit anything for libcouchbase's documentation. We need to start emitting the new single manual format with our current level of docs to the doclist.

[CCBC-34] create package scripts for Debian platforms

Tue, 13 Nov 2012 04:21:35 -0600

create package scripts for Debian platforms

[CCBC-33] create package scripts for RPM

Tue, 13 Nov 2012 04:21:33 -0600

create package scripts for RPM

[CCBC-32] Repository for yum and RPM packages RedHat based systems

Fri, 13 Apr 2012 00:40:47 -0500

Repository for yum and RPM packages RedHat based systems

[CCBC-31] Repository for deb packages (to support Ubuntu and for Debian)

Tue, 13 Nov 2012 04:21:34 -0600

Repository for deb packages (to support Ubuntu and for Debian)

[CCBC-30] IPS pkg and SVr4 packages

Tue, 13 Nov 2012 04:21:35 -0600

IPS pkg and SVr4 packages

[CCBC-29] Automated generation of Windows artifacts

Thu, 10 Jan 2013 15:51:57 -0600

Automated generation of Windows artifacts

[CCBC-28] Automated generation of .rpm and .deb packages

Tue, 14 Feb 2012 18:20:20 -0600

Automated generation of .rpm and .deb packages

[CCBC-27] Automated generation of source tar.gz

Tue, 13 Nov 2012 04:21:34 -0600

Automated generation of source tar.gz

[CCBC-26] Automated testing under CI (Jenkins).

Tue, 13 Nov 2012 04:21:34 -0600

Automated testing under CI (Jenkins).

[CCBC-25] Configuration update tests

Tue, 13 Nov 2012 04:21:34 -0600

Tests for handling updated configuration from Couchbase Server (probably using CouchbaseMock).

[CCBC-24] Port tests from memcached

Tue, 13 Nov 2012 04:21:33 -0600

Port tests as much as possible (general tests of operations in a row) from Ruby client library to internal to libcouchbase.

[CCBC-23] FLUSH command

Tue, 13 Nov 2012 04:21:33 -0600

remember to add manpage

    libcouchbase_error_t libcouchbase_flush(libcouchbase_t instance,
                                            const void* cookie);

    typedef void (*libcouchbase_flush_callback)(libcouchbase_t instance,
                                                const void* command_cookie,
                                                libcouchbase_error_t error);

[CCBC-22] STAT command

Tue, 13 Nov 2012 04:21:33 -0600

Callback per stat returned from the server with a terminator (nkey == 0) (remember to add manpage):

    libcouchbase_error_t libcouchbase_server_stat(libcouchbase_t instance,
                                                  const void* cookie,
                                                  const void* key,
                                                  uint16_t nkey);

    typedef void (*libcouchbase_stat_callback)(libcouchbase_t instance,
                                               const void* command_cookie,
                                               libcouchbase_error_t error,
                                               const void* key,
                                               uint16_t nkey,
                                               const void* value,
                                               uint16_t nvalue);

[CCBC-21] View work using internal IO API instead of libevent.

Tue, 13 Nov 2012 04:21:33 -0600

View work using internal IO API instead of libevent.

[CCBC-20] Timeout options on operations.

Tue, 13 Nov 2012 04:21:35 -0600

Timeout options on operations.

[CCBC-19] install with prefix fails to find vbucket.h

Tue, 14 Feb 2012 18:19:35 -0600

Trying to install libcouchbase, could not successfully build owing to a bad search for libvbucket.

Steps:
1) Install libevent2 with prefix of $HOME/opt
2) Install libvbucket with prefix of $HOME/opt
3) Try to configure libvbucket with prefix of $HOME/opt

...
configure: error: Failed to locate memcached/vbucket.h
ingenthr@shorter:~/src/libcouchbase$ find ~/opt -name vbucket.h
/home/ingenthr/opt/include/libvbucket/vbucket.h

I'd even created a symlink in opt/memcached for vbucket.h, and it still failed.

With a separate prefix I think this should be working.

[CCBC-18] proper bootstrapping off of the REST interface should be part of the client

Tue, 13 Nov 2012 04:21:34 -0600

To maintain flexibility and scalability in interacting with the cluster, the client should follow the HATEOAS approach rather than simply assume URIs. For example, the client currently assumes /pools/default/buckets/{bucket name}, and that's not necessarily correct.

A description of how to interact with the server is listed here:
http://www.couchbase.org/wiki/display/membase/Membase+Management+REST+API#MembaseManagementRESTAPI-Bootstrapping

[CCBC-17] Implement Ketama distribution algorithm in libvbucket

Tue, 13 Nov 2012 04:21:33 -0600

Ketama needed to allow client to handle memcached buckets.

[CCBC-16] Don't connect to all of the nodes initially.. delay the connect until we need them

Thu, 27 Sep 2012 16:06:52 -0500

Right now we're always connecting to all of the nodes as part of the change of current vbucket configuration.. If you're running with 1k of nodes that's going to be a lot of connect requests and we might not need them ;)

[CCBC-15] implement OBSERVE command

Dependency — Tue, 13 Nov 2012 04:21:35 -0600

Once the server has the OBSERVE command implemented, it will need to be added to libcouchbase.

Note: the specific command name is still in the works.

[CCBC-14] creating and destroying design docs with views from the client library should be available

Tue, 13 Nov 2012 04:21:34 -0600

Separate from accessing views, there should be methods of creating and destroying views and the associated design documents.

[CCBC-13] a test that demonstrates topology changes are handled correctly is added

Thu, 27 Sep 2012 16:15:39 -0500

Now that CouchbaseMock is in libcouchbase, it should be able to simulate failures and verify successful recovery.

[CCBC-12] an interface to paginate across large view results is needed

Tue, 13 Nov 2012 04:21:35 -0600

There can be some situations where querying a view returns results that way too large for the client to handle. In these cases, the client library API can allow for one to iterate over Views which have very large results.

The Ruby client does this with streaming. The Java client does this with traditional pagination:
http://guide.couchdb.org/draft/recipes.html#pagination

[CCBC-11] An interface to access Couchbase Views is needed

Fri, 13 Apr 2012 00:36:34 -0500

Need an interface to be able to access Couchbase Views. Even without the ability to create and delete views, accessing view results is important, as views can be created through the Web UI interface.

The specific implementation should be specific to, and familiar to the surrounding language and library, but it should have some general capabilities:
- List views from a given cluster
- Get a handle on something that represents a view on a given cluster
- Assemble query parameters for a View
- Retrieve non-reduced results from a View
* Be able to get the underlying document from the non-reduced results (this request should flow over binprot)
- Retrieve reduced results from a View (the format is the same, but there is no associated docid)

[CCBC-10] set up continuious integration on linux/MacOS X for libcouchbase

Thu, 10 Jan 2013 15:57:01 -0600

libcouchbase needs to be set up under some kind of CI system to regularly be built whenever changes are made.

[CCBC-9] set up continuious integration on Windows for libcouchbase

Thu, 27 Sep 2012 16:06:52 -0500

libcouchbase needs to be set up under some kind of CI system to regularly be built whenever changes are made.

[CCBC-8] remove dependency on libhashkit by simply embedding the two hashing types we require

Tue, 13 Nov 2012 04:21:35 -0600

Currently, libcouchbase relies on libhashkit, though it doesn't really need the entire thing. Since libhashkit is only part of libmemcached, it'd be good to decouple this, and simply have the crc32 and ketama hashing available to us in libcouchbase.

I think the crc32 dependency is actually from libvbucket, but in any event, this issue is to track removing that dependency.

[CCBC-7] fails to link on Ubuntu natty

Tue, 14 Feb 2012 18:17:37 -0600

When trying to build on Ubuntu Natty, it seems to build okay but has issues linking against what seems to be libevent.

ubuntu@ip-10-64-59-212:~/src/libvbucket$ cd ../libcouchbase
ubuntu@ip-10-64-59-212:~/src/libcouchbase$ make install
  CC src/libcouchbase_la-arithmetic.lo
  CC src/libcouchbase_la-base64.lo
  CC src/libcouchbase_la-cookie.lo
  CC src/libcouchbase_la-error.lo
  CC src/libcouchbase_la-event.lo
  CC src/libcouchbase_la-execute.lo
  CC src/libcouchbase_la-get.lo
  CC src/libcouchbase_la-handler.lo
  CC src/libcouchbase_la-instance.lo
  CC src/libcouchbase_la-packet.lo
  CC src/libcouchbase_la-remove.lo
  CC src/libcouchbase_la-server.lo
  CC src/libcouchbase_la-store.lo
  CC src/libcouchbase_la-strerror.lo
  CC src/libcouchbase_la-tap.lo
  CC src/libcouchbase_la-timings.lo
  CC src/libcouchbase_la-touch.lo
  CC src/libcouchbase_la-utilities.lo
  CC src/libcouchbase_la-gethrtime.lo
  CCLD libcouchbase.la
  CC src/packet_debug.lo
  CC src/utilities.lo
  CCLD memcached_packet_debug.la
  CC example/memcat.o
  CCLD example/memcat
./.libs/libcouchbase.so: undefined reference to `evutil_closesocket'
collect2: ld returned 1 exit status
make: *** [example/memcat] Error 1
ubuntu@ip-10-64-59-212:~/src/libcouchbase$
ubuntu@ip-10-64-59-212:~/src/libcouchbase$
ubuntu@ip-10-64-59-212:~/src/libcouchbase$ dpkg --get-selections | grep libeve
libevent-1.4-2 install
libevent-core-1.4-2 install
libevent-dev install
libevent-extra-1.4-2 install

[CCBC-6] graceful disconnections from the server may end in abort()

Tue, 13 Nov 2012 04:21:33 -0600

See event.c around line 106.

If someone shuts down a server with traffic in flight, the clients shouldn't all crash.