[CCBC-212] IO Enhancements Created: 21/May/13  Updated: 22/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Epic Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on CCBC-216 Change 'lcb_wait' semantics Open
depends on CCBC-214 Re-design internal operations in a qu... Open
depends on CCBC-213 Unify internal connect() routines In Progress
Epic Name: IO Enhancements

 Description   
This super bug is an umbrella task for refactoring our IO system to be more streamlined, readable, and compatible with other I/O models.




[CCBC-216] Change 'lcb_wait' semantics Created: 22/May/13  Updated: 22/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks CCBC-212 IO Enhancements Open

 Description   
Currently we have 'wait' and 'breakout' which is optimized and termed for a synchronous model.

We should rename these functions (though we can still maintain aliases) to something like:

lcb_operations_flush:

Which indicates that the user has finished sending requests, and that packets should be sent

And:
lcb_operations_done_callback

Which will be called when the operations have been completed.

Effectively this is the same as lcb_wait and the stop_event_loop callback, but the usage and purpose of those two existing functions aren't clear.




[CCBC-215] Test whether server gets reconnected when user idle for longer than timeout interval Created: 22/May/13  Updated: 22/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
see inline discussion here: http://review.couchbase.org/#/c/26462/2/src/timeout.c

Basically, the semantics of the timeouts aren't clear - the timeout will unconditionally purge the server and disconnect if the timeout hits, even if there is nothing waiting.

In practice this might not be a bug, but it means needless connection code to the server (and SASL auth) is being executed each time.




[CCBC-213] Unify internal connect() routines Created: 21/May/13  Updated: 21/May/13

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks CCBC-212 IO Enhancements Open

 Description   
This will unify the connection routines in terms of error handling and callback invocations.




[CCBC-214] Re-design internal operations in a queue/linked list structure Created: 21/May/13  Updated: 21/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks CCBC-212 IO Enhancements Open




[CCBC-211] libcouchbase may touch a failed node after failover Created: 21/May/13  Updated: 21/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.6
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Ivan Chernetsky Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Observed on CentOS (details in the description)


 Description   
Library version: v2.0.6 (compiled from the source code, on x64)
Server version: v2.0.1
Number of nodes in my test setup: 3 (2 x CentOS 6.2 x64, and 1 x CentOS 5.8 x86, if it's of any relevance)

Steps to reproduce:

1. Create a new cluster
2. Print the returned value by lcb_get_server_list function (there shoud be 3 servers printed)
3. Bring down network on one of servers
4. Try to put several values, until one of them is routed to the failed node. The library should return an error, which is expected.
5. Repeat 2
6. Manually fail over the failed node.
7. Print the returned value by lcb_get_server_list function (there should be 2 servers printed)
8. Try to put several values, until one of them is routed to the failed node. This behaviour is not expected.
9. Start rebalance.
10. While rebalancing takes place, a request might be routed to the failed node. It is not expected as well.
11. Once rebalance is completed, no requests are routed to the failed node. It is as expected.

Thanks in advance for your support.




[CCBC-197] configuration socket reconnects infinitely when using cached config Created: 01/Apr/13  Updated: 16/May/13  Resolved: 11/May/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.5
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The connection to the REST configuration socket is infinitely retried when a cached configuration is used. This is because the refresh_config_cache function calls apply_vbucket_config; which changes the vbucket configuration and relocates the buffers, but does not tell the instance to stop trying to connect to the REST node.

The fix is to make it call the function which is called whenever a new configuration is received over the socket, and optionally close the socket and/or timers when the configuration is properly received.

 Comments   
Comment by Sergey Avseyev [ 15/May/13 ]
Could you refer to patxhset or version where it was fixed?
Comment by Mark Nunberg [ 16/May/13 ]
02c1e02800809b7d44a1e9124c548bdbddc8f626




[CCBC-210] segfault from lcb_wait and libev Created: 10/May/13  Updated: 16/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.6
Fix Version/s: 2.1.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When running a test with a broken cluster, I happened to have PHP segfault after a long, long wait.

#0 0xffffffffffffffff in ?? ()
#1 0x00002aab69d1fab4 in ?? () from /usr/lib64/libcouchbase.so.2
#2 0x00002aab69d16421 in ?? () from /usr/lib64/libcouchbase.so.2
#3 0x00002aab6b271e73 in ev_invoke_pending () from /usr/lib64/libev.so.4
#4 0x00002aab6b276688 in ev_run () from /usr/lib64/libev.so.4
#5 0x00002aab6b06dd86 in ?? () from /usr/lib64/libcouchbase_libev.so
#6 0x00002aab69d1e698 in lcb_wait () from /usr/lib64/libcouchbase.so.2
#7 0x00002aab69af22b3 in php_couchbase_view_impl (ht=2,
    return_value=0x178de850, return_value_ptr=0x0, this_ptr=0x178dcf28,
    return_value_used=1, oo=1, uri_only=0)
    at /tmp/tmpwlG54H/couchbase-1.1.5/views.c:347
#8 0x00002aab69adfe86 in zim_couchbase_view (ht=395387424, return_value=0x1,
    return_value_ptr=0x11, this_ptr=0x0, return_value_used=16)
    at /tmp/tmpwlG54H/couchbase-1.1.5/apidecl.c:768
#9 0x00000000005e78e9 in ?? ()
#10 0x00000000005bd9ab in execute ()
#11 0x0000000000599f45 in zend_execute_scripts ()
#12 0x000000000054ad48 in php_execute_script ()
#13 0x0000000000622cde in ?? ()
#14 0x0000003d16a1d994 in __libc_start_main () from /lib64/libc.so.6
#15 0x0000000000421789 in _start ()


 Comments   
Comment by Matt Ingenthron [ 10/May/13 ]
by the way, I always seem to get libev when installing libcouchbase2 on CentOS 5.8. It has libevent 1.4. Is this normal?
Comment by Sergey Avseyev [ 16/May/13 ]
libcouchbase2 depends on virtual package libcouchbase2-backend, which is provided by libcouchbase2-libev and libcouchbase2-libevent. I think it picks libev version because of alphabetical order. By the time I've written package description I cannot found how to write alternaltion rule, like for DEB packages (which is also ordered) https://github.com/couchbase/libcouchbase/blob/master/packaging/deb/control#L58.

$ ls -1
libcouchbase2-2.0.6-1.x86_64.rpm
libcouchbase2-bin-2.0.6-1.x86_64.rpm
libcouchbase2-core-2.0.6-1.x86_64.rpm
libcouchbase2-libev-2.0.6-1.x86_64.rpm
libcouchbase2-libevent-2.0.6-1.x86_64.rpm
libcouchbase-debuginfo-2.0.6-1.x86_64.rpm
libcouchbase-devel-2.0.6-1.x86_64.rpm
Comment by Sergey Avseyev [ 16/May/13 ]
and the stacktrace not complete, is it possible to install debug symbols and reproduce it?




[CCBC-4] libcouchbase lacks sample applications Created: 15/Aug/11  Updated: 11/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: New Feature Priority: Major
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Libcouchbase requires sample applications for completeness.

 Comments   
Comment by Trond Norbye [ 01/Sep/11 ]
This is not true. There are at least 4 sample applications in the source code: memcat, memcp, memrm and memdump
Comment by Matt Ingenthron [ 01/Sep/11 ]
This is on the "docs" component, and with all client libraries, we have a more comprehensive sample application, so you're correct, but there's more to it.
Comment by Sergey Avseyev [ 09/Apr/12 ]
There is vacuum application https://github.com/trondn/vacuum and the corresponding blog post on Trond's blog http://trondn.blogspot.com/2012/01/so-whats-story-about-libcouchbase-and.html

Why use them in docs?
Comment by Mark Nunberg [ 28/Aug/12 ]
From my experience in answering developers' questions regarding libcouchbase, I don't think sample applications would be as helpful as providing common code idioms, such as ways to handle multi-get, ways to handle errors; things like expiry etc.

It would take quite a bit to write a useful "application" for libcouchbase, so let's just stick to idioms for now.. and assume our readers know how to program in C :)




[CCBC-137] libcouchbase test improvements Created: 31/Aug/12  Updated: 11/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Sub-Tasks:
Key
Summary
Type
Status
Assignee
CCBC-138 allow libcouchbase tests to be execut... Technical task Closed Mark Nunberg  
CCBC-139 add timeout tests for libcouchbase Technical task Open Mark Nunberg  
CCBC-140 add http tests to libcouchbase Technical task Closed Mark Nunberg  
CCBC-141 test libcouchbase lock functionality Technical task Closed Mark Nunberg  
CCBC-142 test libcouchbase observe functionality Technical task Open Mark Nunberg  

 Description   
Generic improvements and fixes for libcouchbase test suite




[CCBC-198] Instance should close REST connection upon receiving valid config cache Created: 01/Apr/13  Updated: 11/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.5
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If a valid config cache configuration is received, there is no point in keeping the REST socket open; (one of the reasons someone would even want to use a cached config is in order to reduce load from the server - so we should try to minimize connections in such cases).

Close the socket immediately (or eventually) if a valid configuration is obtained. Re-open as necessary.




[CCBC-209] Make Windows builds work with jom Created: 09/May/13  Updated: 10/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Mark Nunberg Assignee: Trond Norbye
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Jom supports parallel-make for windows' nmake (the equivalent to normal make's '-j' option). Right now compiling on windows is painfully slow.

Another alternative would be to make a solution file.


http://qt-project.org/wiki/jom

 Comments   
Comment by Trond Norbye [ 10/May/13 ]
what is stopping the current makefile for being executed by a parallel make?




[CCBC-208] libvbucket CRC32 hashing function returns wrong hashes Created: 09/May/13  Updated: 09/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Orfeas Zafeiris Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Windows & Linux (debian)


 Description   
It seems that the CRC32 hashing function in libvbucket (hash_crc32()) returns an invalid hash.

For instance, hashing the key '1234567890' returns 9757 (0x261D), while it should return 639479525 (0x261DAEE5).

Every key I tried (on both win and linux) returns a 16bit hash.

The problem seems to be on line 85 in crc32.c.

Changing the return expression to 'return crc ^ ~0;' fixes the problem and computes the correct hash.

 Comments   
Comment by Sergey Avseyev [ 09/May/13 ]
The algorithm based on CRC32, but it isn't CRC32, for example java client does the same: https://github.com/dustin/java-memcached-client/blob/master/src/main/java/net/spy/memcached/DefaultHashAlgorithm.java#L103-108
Comment by Sergey Avseyev [ 09/May/13 ]
Do we have official documentation about "crc32-based hash" used in vbucket algorithm?
Comment by Orfeas Zafeiris [ 09/May/13 ]
I see, however the function name and the comments in vbucket.c make it really confusing for people developing their own clients for interacting with couchbase.




[CCBC-207] Failed connection attempts will hang until timeout interval is reached on win32 Created: 08/May/13  Updated: 08/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.6
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Failed connection errors are not detected in the select() call. This is because the select call does not add the socket to the exception fdset.

MSDN documentation states that failed connections are delivered through exceptfds, and that only successful connections are delivered as writability notifications. See http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141(v=vs.85).aspx




[CCBC-204] configuration cache does not work for memcached buckets Created: 29/Apr/13  Updated: 08/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.6
Fix Version/s: 2.1.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Trond Norbye
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: multiplatform


 Description   
Currently, the configuration cache relies on not_my_vbucket responses as an indicator that the configuration needs to be updated. In an environment with memcached buckets only though, there will not be such a response and thus we need another way to reliably update the configuration.

One thought would be to simply ensure the configuration cache is not more than NN seconds old. The process who has the configuration connection would simply need to touch it every NN-1 seconds if the configuration connection is still active.

If that process exits, then after NN seconds, processes will race to create a new configuration cache file.

The only downside I can see is that certain failure modes (stale TCP socket) would still need to be handled.




[CCBC-173] Small typo in C tutorial Created: 30/Jan/13  Updated: 06/May/13  Resolved: 06/May/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Tug Grall Assignee: Karen Zeller
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
From http://www.couchbase.com/forums/thread/string-typo-c-sdk-tutorial-sample-code

In the couchbase-sdk-c-2.0.pdf tutorial sample code, there's a typo in an error string:
{
lcb_store_cmd_t cmd;
const lcb_store_cmd_t *commands[1];
commands[0] = &cmd;
memset(&cmd, 0, sizeof(cmd));
cmd.v.v0.operation = LCB_SET;
cmd.v.v0.key = "foo";
cmd.v.v0.nkey = 3;
cmd.v.v0.bytes = "bar";
cmd.v.v0.nbytes = 3;
err = lcb_store(instance, NULL, 1, commands);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to get: %s\n", lcb_strerror(NULL, err));
return 1;
}
}
The err code there is for a set operation, so the error string in the fprintf call should say "Failed to set" not "Failed to get".

 Comments   
Comment by Sergey Avseyev [ 06/May/13 ]
https://github.com/couchbase/docs/pull/209
Comment by Karen Zeller [ 06/May/13 ]
merged




[CCBC-56] Update instructions for building libcouchbase to be more verbose Created: 02/Mar/12  Updated: 06/May/13  Resolved: 06/May/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mike Wiederhold Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://www.couchbase.com/docs/couchbase-sdk-c-1.0/linux_and_mac.html

This guy was a little bit confused about how to build because the first step wasn't to run config/autorun.sh. We should add that to the documentation.

Also, I think we should add the stuff that is in the comments to the docs. We should give the locations of where the downloads are. There is also no mention of needing memcached installed. This is a requirement since libcouchbase depends on memcached/protocol_binary.h.

 Comments   
Comment by Sergey Avseyev [ 06/May/13 ]
https://github.com/couchbase/libcouchbase#hacking




[CCBC-179] Provide and example and clear statement that for usual data connection user should specify bucket credentials instead of admin Created: 13/Feb/13  Updated: 06/May/13  Resolved: 06/May/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.3
Fix Version/s: 2.0.6
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Sergey Avseyev Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
https://github.com/couchbase/libcouchbase/blob/master/man/man3couchbase/lcb_create.3couchbase.txt

This doc lacks of example with bucket level credentials

 Comments   
Comment by Sergey Avseyev [ 06/May/13 ]
http://review.couchbase.org/26108




[CCBC-196] Handle -1 return value from found_incorrect_master Created: 01/Apr/13  Updated: 06/May/13  Resolved: 06/May/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.5
Fix Version/s: 2.0.5
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If vbucket_found_incorrect_master returns a negative value, the code in parse_single will fail:

assert((lcb_size_t)idx < instance->nservers);

This should be handled and the command should be failed out accordingly.

 Comments   
Comment by Sergey Avseyev [ 06/May/13 ]
https://github.com/couchbase/libcouchbase/commit/b2ad9312dfe25c792b03231166c80cf5761cafdd




[CCBC-201] Pass Host HTTP header when doing bootstrap Created: 16/Apr/13  Updated: 06/May/13  Resolved: 06/May/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.5
Fix Version/s: 2.0.6
Security Level: Public

Type: Task Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
According to HTTP/1.1 Host header is required.

Either pass Host header or use HTTP/1.0

 Comments   
Comment by Sergey Avseyev [ 06/May/13 ]
http://review.couchbase.org/25822




[CCBC-202] Initial connect timeout not being observed Created: 16/Apr/13  Updated: 06/May/13  Resolved: 06/May/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.5
Fix Version/s: 2.0.6
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Steps:

1) Set the timeout to something small (say, 1 second)
2) Suspend the mock server
3) Try to connect to the mock

This will hang the client until an actual connection error is received (or possibly is never received). The perl tests die with a SIGALRM in this case.

    note "Suspending mock server";
    $mock->suspend_process();
    $cbo->timeout(2);
    ok(!$cbo->connect(), "Connect failed");

Needless to say, $cbo->connect() stalls.

 Comments   
Comment by Mark Nunberg [ 23/Apr/13 ]
http://review.couchbase.org/25711




[CCBC-206] Improve libcouchbase rpm instructions Created: 19/Feb/13  Updated: 02/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Minor
Reporter: James Mauss Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: info-request
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There is currently no documentation about installing libcouchbase from the rpm or deb files.
The only documentation is for installation with yum or apt-get.

Since some packages are optional, more instructions would be nice.

 Comments   
Comment by Karen Zeller [ 28/Mar/13 ]
Hi,

Can you point me to any information about rpm install for the SDK on github, or anywhere else. Assign back to me to document.


Thanks,

Karen
Comment by Karen Zeller [ 02/May/13 ]
Can you close this and reopen this under the Couchbase C Client Library project?

We're doing massive housekeeping on docs requests and want to keep this separate from server.


Thanks!




[CCBC-3] libcouchbase lacks a tutorial Created: 15/Aug/11  Updated: 02/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: New Feature Priority: Major
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CCBC-48 tutorial chapter is complete Closed

 Description   
libcouchbase lacks a tutorial document

 Comments   
Comment by Karen Zeller [ 26/Mar/13 ]
Dependencies, Blockers:

-Need sample application for C SDK demonstrating CRUD and using Views from SDK.
-Need technical notes and information on sample application.
-Any related resources?

Comment by Karen Zeller [ 26/Mar/13 ]
See information required/dependencies.




[CCBC-161] libcouchbase_flush wrongly documented to "flush entire cluster" Created: 03/Jan/13  Updated: 02/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Tim Smith Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-c-2.0/api-reference-summary.html#table-couchbase-sdk-c-summary


 Description   
The docs say that libcouchbase_flush() will "Flush the entire cluster" and "libcouchbase_flush may be used to remove all key/value pairs from the entire cluster."

It actually flushes a bucket, not the entire cluster.




[CCBC-205] Provide stricter semantics/behavior for configuration cache. Created: 30/Apr/13  Updated: 01/May/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Trond Norbye
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We may want to create a 'cache-only' mode in which creation of the client fails if the cache cannot be created (e.g. a bad path was passed, or the file is not writeable, etc.)

Currently, the cache is mostly a transparent extra, but there is no way for a user to determine whether the mechanism is at all working.




[CCBC-188] library codedumped during rebalance process Created: 26/Feb/13  Updated: 23/Apr/13  Resolved: 23/Apr/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.3
Fix Version/s: 2.0.6
Security Level: Public

Type: Bug Priority: Critical
Reporter: Haster Assignee: Sergey Avseyev
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File libcouchbase-2.0.4.patch    
Flagged:
Release Note

 Description   
How to reproduce:
1) Attach second node to a Couchbase claster, start rebalance process and in that time call lcb_get function many times. As a result application go down (assert in 0).

assert(ringbuffer_read(&src->pending_cookies, &ct, sizeof(ct)) == sizeof(ct));

src->pending_cookies->nbytes == 0 in that case, so ringbuffer_read return 0 so assert is 0
and application is over.
We think, that nbytes == 0 is a common situation, than src->connected == 0 and this situation must not interpretated as error.

Below you can find additional information
Stack:
#0 0xc000000000211ab0:0 in _lwp_kill+0x30 () from /lib/hpux64/libpthread.so.1
#1 0xc000000000178810:0 in pthread_kill ()
at /ux/core/libs/threadslibs/src/common/pthreads/signal.c:207
#2 0xc0000000003f8140:0 in raise ()
at ../../../../../core/libs/libc/shared_em_64_perf/../core/gen/raise.c:33
#3 0xc000000000508c50:0 in abort ()
at ../../../../../core/libs/libc/shared_em_64_perf/../core/gen/abort_em.c:84
#4 0xc0000000003c3f90:0 in _assert ()
at ../../../../../core/libs/libc/shared_em_64_perf/../core/gen/assert.c:57
#5 0xc0000000076a9080:0 in relocate_packets (src=0x6000000001c63c00,
dst_instance=0x60000000064f6800) at src/instance.c:542
#6 0xc0000000076a9b40:0 in lcb_update_serverlist (instance=0x60000000064f6800)
at src/instance.c:606
#7 0xc0000000076ac7e0:0 in vbucket_stream_handler (sock=31, which=2,
arg=0x60000000064f6800) at src/instance.c:996
#8 0xc000000007d6db60:0 in event_base_loop (base=0x6000000006381780,
flags=<not available>) at event.c:1346
#9 0xc000000000a3aed0:0 in lcb_io_run_event_loop (iops=0x60000000061bff20)
at plugins/io/libevent/plugin-libevent.c:324
#10 0xc0000000076c2750:0 in lcb_wait (instance=0x60000000064f6800)
at src/wait.c:60
#11 0xc0000000109c5e00:0 in Cache_Storage::CCouchbaseStorage::do_usefull_work (
---Type <return> to continue, or q <return> to quit---
this=0x6000000006224280) at cache_source/ucouchbasestorage.cpp:524
#12 0xc0000000109c7850:0 in Cache_Storage::CCouchbaseStorage::execute (
this=0x6000000006224280) at cache_source/ucouchbasestorage.cpp:585
#13 0xc00000000782c0f0:0 in threads::thread_proc (thr_ptr=0x60000000062242a0)
at has_common_source/source/cpp/threads.cpp:225
#14 0xc00000000013fb20:0 in __pthread_bound_body ()
at /ux/core/libs/threadslibs/src/common/pthreads/pthread.c:4875

(gdb) list
537 }
538 dst = dst_instance->servers + (lcb_size_t)idx;
539 if (src->connected) {
540 assert(ringbuffer_read(&src->output_cookies, &ct, sizeof(ct)) == sizeof(ct));
541 } else {
542 assert(ringbuffer_read(&src->pending_cookies, &ct, sizeof(ct)) == sizeof(ct));
543 }
544
545 assert(ringbuffer_ensure_capacity(&dst->cmd_log, npacket));
546 assert(ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes));
(gdb) p src->pending_cookies
$5 = {root = 0x6000000001e3d200 "", read_head = 0x6000000001e3d200 "",
write_head = 0x6000000001e3d200 "", size = 512, nbytes = 0}
(gdb) p src->connected
$7 = 0

lcb_size_t ringbuffer_read(ringbuffer_t *buffer, void *dest, lcb_size_t nb)
{
char *d = dest;
lcb_size_t nr = 0;
lcb_size_t space;
lcb_size_t toRead;

if (buffer->nbytes == 0) {
return 0;
}


 Comments   
Comment by dongzaixx [ 27/Mar/13 ]
We have experienced the same problem, and we try to fix it as follow.

Firstly, When a (!connected) server is reconnected, the tasks in its "pending" buffer will be moved into "output" buffer. If it's connection is broken again immediately, relocate_packets() will go to wrong path.

Secondly, there seems to be another bug in handler:751 which could cause crash during failover. [Null pointer Check]

============================================================================================================

diff -rc libcouchbase-2.0.4/src/handler.c libcouchbase-2.0.4.patched/src/handler.c
*** libcouchbase-2.0.4/src/handler.c 2013-03-04 15:07:51.785943104 +0800
--- libcouchbase-2.0.4.patched/src/handler.c 2013-03-27 20:48:13.042296326 +0800
***************
*** 748,754 ****
  {
      lcb_uint16_t ret = ntohs(res->response.status);
      if (ret == PROTOCOL_BINARY_RESPONSE_SUCCESS) {
! sasl_dispose(&server->sasl_conn);
          server->sasl_conn = NULL;
          lcb_server_connected(server);
      } else if (ret == PROTOCOL_BINARY_RESPONSE_AUTH_CONTINUE) {
--- 748,756 ----
  {
      lcb_uint16_t ret = ntohs(res->response.status);
      if (ret == PROTOCOL_BINARY_RESPONSE_SUCCESS) {
! if (server->sasl_conn) {
! sasl_dispose(&server->sasl_conn);
! }
          server->sasl_conn = NULL;
          lcb_server_connected(server);
      } else if (ret == PROTOCOL_BINARY_RESPONSE_AUTH_CONTINUE) {
diff -rc libcouchbase-2.0.4/src/instance.c libcouchbase-2.0.4.patched/src/instance.c
*** libcouchbase-2.0.4/src/instance.c 2013-03-06 05:41:29.600470755 +0800
--- libcouchbase-2.0.4.patched/src/instance.c 2013-03-27 20:49:06.766296416 +0800
***************
*** 628,638 ****
              idx = vbucket_found_incorrect_master(dst_instance->vbucket_config, vb, idx);
          }
          dst = dst_instance->servers + (lcb_size_t)idx;
! if (src->connected) {
! assert(ringbuffer_read(&src->output_cookies, &ct, sizeof(ct)) == sizeof(ct));
! } else {
! assert(ringbuffer_read(&src->pending_cookies, &ct, sizeof(ct)) == sizeof(ct));
! }
  
          assert(ringbuffer_ensure_capacity(&dst->cmd_log, npacket));
          assert(ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes));
--- 628,635 ----
              idx = vbucket_found_incorrect_master(dst_instance->vbucket_config, vb, idx);
          }
          dst = dst_instance->servers + (lcb_size_t)idx;
! assert(ringbuffer_read(&src->output_cookies, &ct, sizeof(ct)) == sizeof(ct) ||
! ringbuffer_read(&src->pending_cookies, &ct, sizeof(ct)) == sizeof(ct));
  
          assert(ringbuffer_ensure_capacity(&dst->cmd_log, npacket));
          assert(ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes));

============================================================================================================
Comment by dongzaixx [ 28/Mar/13 ]
I have uploaded the patch file.
Comment by Sergey Avseyev [ 23/Apr/13 ]
Thank you for patches. I copied them to our review system: http://review.couchbase.org/25851 and http://review.couchbase.org/25852




[CCBC-200] lcb_make_http_request doesn't respect timeouts Created: 05/Apr/13  Updated: 05/Apr/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: matthew newhook Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Windows 7, VS2010

Attachments: Text File testcase.cpp    

 Description   
Run the attached testcase on Windows (after fixing the host and such). The lcb_wait after calling lcb_make_http_request will hang for a very long time (perhaps forever) if the server is no longer accessible.

I tested this by starting the client. When it prints "sleeping" I pull the ethernet cable on my machine. If the lcb_make_http_request is replaced by lcb_get then you'll get a timeout which is expected.




[CCBC-199] Problem with a newly created memcached bucket Created: 02/Apr/13  Updated: 02/Apr/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Trond Norbye Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
See: http://www.couchbase.com/forums/thread/libcouchbase-problem-newly-created-memcached-bucket





[CCBC-195] config cache will crash client due to relocating buffers Created: 01/Apr/13  Updated: 02/Apr/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.5
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
See CCBC-190

During a cluster configuration change, the buffers are relocated while they are being processed. This causes crashes.

A solution would be to "request" a configuration refresh and have that take place once it is safe to move buffers around (i.e. once we are out of the buffer processing call chain).

 Comments   
Comment by dongzaixx [ 02/Apr/13 ]
Same Issue: http://www.couchbase.com/issues/browse/CCBC-188




[CCBC-181] enhance observe man page with a more complete example Created: 15/Feb/13  Updated: 02/Apr/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.3
Fix Version/s: 2.1.0
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The current observe man page doesn't really go into how to interpret results from the request. I needs to have that.

Ideally it should also show how one would do a regular poll to show checking for replication or persistence.

 Comments   
Comment by Trond Norbye [ 02/Apr/13 ]
should probably be written by our native english speaking committer ;)




[CCBC-190] Conncache configuration not passing integration tests Created: 15/Mar/13  Updated: 01/Apr/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Trond Norbye
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
There are two primary issues which occur during server configuration changes (i.e. failover, rebalance):

1)
[SDKD(WARNING) 56.55 cbsdk.sdkd.remote remote.py:266] sdkd_lcb: src/event.c:211: parse_single: Assertion `(lcb_size_t)idx < c->instance->nservers' failed.

2)
The client may just hang.

 Comments   
Comment by Matt Ingenthron [ 19/Mar/13 ]
Trond, can you update this when you have a moment?
Comment by Matt Ingenthron [ 26/Mar/13 ]
Trond, I know you gave me status via IM, but please update this when you have a moment.
Comment by Trond Norbye [ 27/Mar/13 ]
I had problems reproducing any of these problems when I tested it at home (tested both adding nodes, removing nodes, failover nodes etc). I need more time to try to reproduce this, and more information on how to reproduce it.

I really don't like using jira as a "task list". JIRA should be used to file bugs, and one bug report PER issue, not multiple issues like this. It would be nice if we could use cbbugg or something with less overhead than the JIRA taskflow (and something that don't require me to log in on a daily basis, but allow me to let the tool be running over night.. after all this is bugs we're talking about, not my credit card information of a flight control system.)
Comment by Mark Nunberg [ 27/Mar/13 ]
sdkd_lcb: src/event.c:211: parse_single: Assertion `(lcb_size_t)idx < c->instance->nservers' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x54a20940 (LWP 13592)]
0x000000315d230285 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) bt
#0 0x000000315d230285 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x000000315d231d30 in abort () at abort.c:88
#2 0x000000315d229706 in __assert_fail (assertion=0x2aaaaaf6e438 "(lcb_size_t)idx < c->instance->nservers",
    file=0x2aaaaaf6e385 "src/event.c", line=211, function=0x2aaaaaf6e3bf "parse_single") at assert.c:78
#3 0x00002aaaaaf5448e in parse_single (c=0x2aaac0011000, stop=1364387212495254000) at src/event.c:211
#4 0x00002aaaaaf546e0 in do_read_data (c=0x2aaac0011000, allow_read=0) at src/event.c:275
#5 0x00002aaaaaf5490a in lcb_server_event_handler (sock=312, which=4, arg=0x2aaac0011000) at src/event.c:352
#6 0x00002aaaab39459a in event_base_loop () from /usr/lib64/libevent-1.4.so.2
#7 0x00002aaaab17c321 in lcb_io_run_event_loop (iops=0x6cc7a0) at plugins/io/libevent/plugin-libevent.c:320
#8 0x00002aaaaaf647d8 in lcb_wait (instance=0x6ce720) at src/wait.c:67
#9 0x00002aaaaaae2870 in CBSdkd::Handle::postsubmit (this=0x6cb5a0, rs=..., nsubmit=1) at Handle.cpp:209
#10 0x00002aaaaaae2e22 in CBSdkd::Handle::dsMutate (this=0x6cb5a0, cmd=..., ds=..., out=..., options=...)
    at Handle.cpp:301
#11 0x00002aaaaaaee6f8 in CBSdkd::WorkerDispatch::_process_request (this=0x2aaab00372c0, req=...,
    rs=0x2aaab002dec0) at IODispatch.cpp:606
#12 0x00002aaaaaaefc5d in CBSdkd::WorkerDispatch::run (this=0x2aaab00372c0) at IODispatch.cpp:711
#13 0x00002aaaaaaec2d4 in CBSdkd::new_worker_thread (worker=0x2aaab00372c0) at IODispatch.cpp:278
#14 0x000000315de0677d in start_thread (arg=<value optimized out>) at pthread_create.c:301
#15 0x000000315d2d3c1d in clone () from /lib64/libc.so.6
(gdb) frame 3
#3 0x00002aaaaaf5448e in parse_single (c=0x2aaac0011000, stop=1364387212495254000) at src/event.c:211
211 assert((lcb_size_t)idx < c->instance->nservers);
(gdb) print idx
$1 = -1
(gdb) print c->instance->nservers
$2 = 2


It seems that vbucket_found_incorrect_master is returning -1. I'm not sure why this would happen or what the appropriate solution is. In general this piece of "retry" code is subject to the thundering herd problem anyway.

A temporary fix would be to just retry the old index until we get a valid config. I'll modify the code and test with that hack, will report results again.
Comment by Mark Nunberg [ 27/Mar/13 ]
Additionally:

==16609== Thread 19:
==16609== Conditional jump or move depends on uninitialised value(s)
==16609== at 0x315D2791B4: strstr (strstr.c:110)
==16609== by 0x50B1C90: lcb_load_config_cache (config_cache.c:71)
==16609== by 0x50B1D40: lcb_refresh_config_cache (config_cache.c:99)
==16609== by 0x50B2499: parse_single (event.c:218)
==16609== by 0x50B280E: do_read_data (event.c:308)
==16609== by 0x50B2A38: lcb_server_event_handler (event.c:385)
==16609== by 0x62E2599: event_base_loop (in /usr/lib64/libevent-1.4.so.2.1.3)
==16609== by 0x60DB320: lcb_io_run_event_loop (plugin-libevent.c:320)
==16609== by 0x50C2907: lcb_wait (wait.c:67)
==16609== by 0x4C4086F: CBSdkd::Handle::postsubmit(CBSdkd::ResultSet&, unsigned int) (Handle.cpp:209)
==16609== by 0x4C40E21: CBSdkd::Handle::dsMutate(CBSdkd::Command, CBSdkd::Dataset const&, CBSdkd::ResultSet&, CBSdkd::ResultOptions const&) (Handle.cpp:301)
==16609== by 0x4C4C6F7: CBSdkd::WorkerDispatch::_process_request(CBSdkd::Request const&, CBSdkd::ResultSet*) (IODispatch.cpp:606)
Comment by Trond Norbye [ 27/Mar/13 ]
what is the content of buffer?
Comment by Mark Nunberg [ 27/Mar/13 ]
Fixed the issue, see http://review.couchbase.org/#/c/25372/
Comment by Mark Nunberg [ 27/Mar/13 ]
There are other issues. A rudimentary patch has fixed the assertion failure (by doing what I suggested). The subsequent effect however is that the newer configuration is never re-read (resulting in about half the ops getting a NETWORK_ERROR).
Comment by Mark Nunberg [ 27/Mar/13 ]
Can you explain the consensus process in how a single client decides it will be the "master" for storing the config information? I am tempted to try using flock(2) and see how that works. Keep in mind I am testing with many concurrent clients.
Comment by Trond Norbye [ 27/Mar/13 ]
There is no masters.. and in theory all of them may update the configuration at the same time.
Comment by Trond Norbye [ 27/Mar/13 ]
is there a reproducible test case one may run for this?? I had a php script looping in a while (true) mode creating a new cached config while I added, removed, failed over etc nodes and I was not able to reproduce the problem..
Comment by Mark Nunberg [ 28/Mar/13 ]
So I've found the underlying cause of the issue. The buffers are being relocated from underneath us while lcb_refresh_config_cache is called. This causes subsequent assertions to fail. Investigating this further now.
Comment by Mark Nunberg [ 01/Apr/13 ]
I'll split these into several issues (as they are really several issues)




[CCBC-194] Please add support for setsockopt() SO_KEEPALIVE & TCP_KEEPIDLE Created: 25/Mar/13  Updated: 25/Mar/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.4
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Michael Leib Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS 6.x, Linux


 Description   
Please add support for setsockopt() options SO_KEEPALIVE & TCP_KEEPIDLE through (suggestion)
lcb_connect() or some other such api so that I can control the connection's ability to not timeout
and have a firewall cause a disconnection from lack of inactivity...without such control, receive
NETWORK_ERROR from couchbase apis.

I cannot adjust the firewall, so I need to keep the connection alive.

 Comments   
Comment by Mark Nunberg [ 25/Mar/13 ]
As a workaround, you can wrap your own IOPS structure, and override the socket creation bits so as to set your own socket options.




[CCBC-193] libcouchbase build is not checking install properly Created: 22/Mar/13  Updated: 22/Mar/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I was trying to package the PHP client, which depends on libcouchase, and it is was refusing to build on some systems.

After looking into it a bit, there may be a couple of problems, but the big one is in the summary.

One problem is that it seems the dpkg -i doesn't actually get run. I added an echo of the label in there, and see that in the console output in Jenkins, but I don't see the lines under the label run. I'm stumped there.




[CCBC-192] Failure to handle host that has been removed from a cluster Created: 20/Mar/13  Updated: 20/Mar/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.4
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Toby Corkindale Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Linux, Ubuntu 12.04 Precise


 Description   
We're encountering an issue where both the Ruby and Perl clients get() method will consistently, repeatedly just return an error. On the Perl client it is "Temporary error. try again later", and on Ruby it is "Bucket not found".

This occurs when we have several couchbase servers, and have connected using the node list option to supply all of them to the library.

We remove the first server in the node list via the Couchbase GUI, trigger a rebalance, and wait for it to complete.
Once the rebalance has completed, we start a script that simply attempts to connect and retrieve some keys we previously inserted. Instead of success, we find that it repeatedly gives the errors mentioned above.

note that if we remain connected to the cluster, then everything seems fine. The errors only start occurring when a client connects.

note also that the issue only occurs if the first server (or servers) in the list are down; if the first server listed is currently actively in the cluster, then we're OK.

It seems like the problem is that the now-removed server is still accepting connections on port 8091, and thus the client library thinks that this means it is a valid server. However because it's removed from the pool, it causes confusion and errors to the client.

Obviously, the correct/desired behaviour would be for the client library to behave as if the removed-server was not accepting connections, and to move onto the next server in the node list.

 Comments   
Comment by Matt Ingenthron [ 20/Mar/13 ]
Others should comment as well, but our current expectation is that if you're removing a server from the cluster, you're also removing it from the bootstrap list. Thus, the best practice would be to remove it from the list first, then remove the server from the cluster.

We can probably make this better though.
Comment by Toby Corkindale [ 20/Mar/13 ]
That is our current work-around. However it is rather annoying, and makes rolling updates/reboots an extremely time consuming process.
ie. Update puppet with new node list, wait an hour for that to get deployed over every server, then rebalance and remove server1; apply system updates and reboot, then add back to cluster and rebalance. Then update puppet's node list to include that server, but not the next one, wait an hour for it to apply everywhere, etc etc etc.

There's also the danger that someone will forget to update the managed node list one day and remove a server from the cluster, expecting it to Just Work.

I also have to point out that the current behaviour is unexpected, and I don't recall seeing any warnings about it in the documentation or tutorials; thus it's likely other users may hit this in the future and have some accidental production downtime, as we did.
Comment by Matt Ingenthron [ 20/Mar/13 ]
Yes, all good points-- we'll definitely look to make it better.




[CCBC-191] Segfault when quering view with spaces Created: 12/Mar/13  Updated: 19/Mar/13  Resolved: 19/Mar/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.5
Security Level: Public

Type: Task Priority: Blocker
Reporter: Sergey Avseyev Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 12/Mar/13 ]
this is a libcouchbase issue. the HTTP message itself separates URLs and verbs by spaces. LCB should convert these properly before submission.
Comment by Mark Nunberg [ 12/Mar/13 ]
Program received signal SIGABRT, Aborted.
0x00007ffff4e2c475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff4e2c475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff4e2f6f0 in *__GI_abort () at abort.c:92
#2 0x00007ffff4e25621 in *__GI___assert_fail (
    assertion=0x7ffff3bb2e90 "!((*__ctype_b_loc ())[(int) ((ch))] & (unsigned short int) _ISspace)",
    file=<optimized out>, line=422, function=0x7ffff3bb3484 "parse_url_char") at assert.c:81
#3 0x00007ffff3ba713d in parse_url_char (s=s_req_path, ch=32 ' ') at contrib/http_parser/http_parser.c:422
#4 0x00007ffff3baafdf in http_parser_parse_url (
    buf=0x1009dd0 "http://127.0.0.1:8092/default/_design/foo bar/_view/bar baz", buflen=59, is_connect=0, u=0xfc41c0)
    at contrib/http_parser/http_parser.c:1955
#5 0x00007ffff3b9c024 in lcb_make_http_request (instance=0xfc25f0, command_cookie=0x7fffffffa490,
    type=LCB_HTTP_TYPE_VIEW, cmd=0x7fffffffa4c0, request=0x7fffffffa468) at src/http.c:730
#6 0x00007ffff3dd4833 in php_couchbase_view_impl (ht=<optimized out>, return_value=<optimized out>,
    return_value_ptr=<optimized out>, this_ptr=<optimized out>, return_value_used=0, oo=oo@entry=1,
    uri_only=uri_only@entry=0) at /sources/lcb2.0-repo/php/views.c:304
#7 0x00007ffff3dc3296 in zim_couchbase_view (ht=<optimized out>, return_value=<optimized out>,
    return_value_ptr=<optimized out>, this_ptr=<optimized out>, return_value_used=<optimized out>)
    at /sources/lcb2.0-repo/php/apidecl.c:758
#8 0x00007ffff3feeedc in xdebug_execute_internal (current_execute_data=0x7ffff7f86060, return_value_used=0)
    at /srv/debian_developer/xdebug/xdebug-2.2.1/build-php5/xdebug.c:1483
#9 0x00000000007468be in zend_do_fcall_common_helper_SPEC (execute_data=0x7ffff7f86060)
    at /tmp/buildd/php5-5.4.4/Zend/zend_vm_execute.h:644
#10 0x00000000007002c7 in execute (op_array=0x7ffff7fbded8) at /tmp/buildd/php5-5.4.4/Zend/zend_vm_execute.h:410
#11 0x00007ffff3feea81 in xdebug_execute (op_array=0x7ffff7fbded8)
    at /srv/debian_developer/xdebug/xdebug-2.2.1/build-php5/xdebug.c:1391
#12 0x00000000006a010e in zend_execute_scripts (type=8, retval=0x7ffff7fbde98, file_count=3)
    at /tmp/buildd/php5-5.4.4/Zend/zend.c:1279
#13 0x000000000063f6e3 in php_execute_script (primary_file=0x7ffff7dea9f2) at /tmp/buildd/php5-5.4.4/main/main.c:2473
#14 0x0000000000749033 in do_cli (argc=0, argv=0x7fffffffe3bd) at /tmp/buildd/php5-5.4.4/sapi/cli/php_cli.c:988
#15 0x00000000004310ca in main (argc=32767, argv=0xdb9210) at /tmp/buildd/php5-5.4.4/sapi/cli/php_cli.c:1361
Comment by Trond Norbye [ 19/Mar/13 ]
This is fixed in libcouchbase and will be part of libcouchbase 2.0.5




[CCBC-128] Allow semi-static bootstrapping Created: 19/Nov/12  Updated: 05/Mar/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0beta2
Fix Version/s: 2.1.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I'm marking this as critical because many folks seem to frown upon the high delays (read http connection latency) when establishing a new couchbase client.

In this case, this would be a constructor option available for lcb_create which will accept an opaque 'map file'. This 'map file' will really just be a dump of a vbucket configuration. What makes this different from compat mode is that the clients will bootstrap when a network error or timeout occurs.

However in the normal model of spawn -> mc operation, where the cluster config is fairly static, there is no need for the latency induced by an extra http connection.

We'd need to implement file-locking semantics to write to this file (we'd maybe just ask for a path to read/write the config to/from, perhaps checking the timestamp, too).. as we'd have multiple clients trying to read/write to this. This is simple enough in Unix

This is in conjunction with a different discussion regarding how to best handle re-bootstrapping, where it was suggested that we really don't need the bootstrap connection in the first place.

There are inherent risks involved with this approach, for example, if one specifies the wrong nodes, then we only know about this during the re-bootstrap.

 Comments   
Comment by Matt Ingenthron [ 19/Nov/12 ]
I'm certainly not opposed to this idea, but I'm not sure about it being at the critical level. I do think it'd have to be portable though.

Sergey, what do you think? Is this a hotly needed feature?
Comment by Sergey Avseyev [ 20/Nov/12 ]
Frankly speaking I don't understand the benefit from dumping some configuration to the disk. It might be better to implement lcb_reconnect() and in this case the user just won't destroy lcb_t internal structs and reuse previous in-memory config.
Comment by Mark Nunberg [ 20/Nov/12 ]
This is for performance enhancements, it's not directly related to failures, but can increase speed for some use cases dramatically, as it cuts out a lot of latency.

The idea is that the node map is static, so we don't need to have the initial HTTP connection.

If you really want to be inventive, this can also use POSIX IPC instead of actual 'disk' (i.e. define a common shm path). The idea here in any event is to not depend so heavily on the REST connection in normal non-failure situations.
Comment by Matt Ingenthron [ 20/Nov/12 ]
While I think this makes sense, I do believe it can be deferred.
Comment by Matt Ingenthron [ 13/Feb/13 ]
Let's see about this in the near future Sergey. We can discuss in our next sync up.




[CCBC-169] Handle 30x HTTP response code when chosen node doesn't have active vbuckets Created: 15/Jan/13  Updated: 05/Mar/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.2, 2.0.3
Fix Version/s: 2.1.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on MB-6922 ns_server should send 302 reply when ... Closed
Flagged:
Release Note

 Comments   
Comment by Matt Ingenthron [ 15/Jan/13 ]
Changed the description to be clear that we should be handling more 30x codes.




[CCBC-182] The library stops iterating backup nodes list if next one isn't accessible Created: 15/Feb/13  Updated: 05/Mar/13  Resolved: 15/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0, 2.0.1, 2.0.2, 2.0.3
Fix Version/s: 2.0.4
Security Level: Public

Type: Task Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
Because of typo in loop condition[1], it won't continue trying the backup nodes

https://github.com/couchbase/libcouchbase/blob/4053418dbdffa8e2315c35c433768d0fa19e9fd0/src/instance.c#L834

 Comments   
Comment by Sergey Avseyev [ 15/Feb/13 ]
The library stops iterating backup nodes list if next one isn't accessible




[CCBC-185] Incorrect parsing of URIs Created: 20/Feb/13  Updated: 05/Mar/13  Resolved: 20/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.3
Fix Version/s: 2.0.4
Security Level: Public

Type: Task Priority: Major
Reporter: Trond Norbye Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
The current parsing of the hostname believes that the entire thing specified is the hostname rather than stripping of potential http:// and paths




[CCBC-180] Segfault when in case server name resolved to several addresses, where first of them rejects couchbase connection Created: 28/Jan/13  Updated: 05/Mar/13  Resolved: 15/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.3
Fix Version/s: 2.0.4
Security Level: Public

Type: Bug Priority: Critical
Reporter: Tim Ray Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: linux - several

Flagged:
Release Note

 Description   
I get a lot of segfaults using the client when trying to connect to _external_ couchbase servers (i.e. when there is no couchbase server on localhost).

I've tested this on a couple different machines, with both ruby REE and 1.9.3 - same results every time.

To reproduce:
1) setup couchbase server / cluster on machine-1
2) setup ruby development environment on machine-2
-- I'm using rvm
-- I've install the couchbase library
3) run simple ruby program to test connections:
#simple_test.rb
require 'couchbase'
Couchbase.connection_options = {:bucket => 'rubytest', :node_list => ['123.45.67.89:8091'], :password => 'password'}
Couchbase.connect
4) watch it segfault when you run it

Essentially, it seems that setting the 'connection_optoins' has no effect on creating new buckets, they keep trying to connect to the built-in default of 'localhost'

This is especially problematic because of what it means trying to use couchbase in production: If I work around this by passing connection options to a new bucket, I will eventually run into this same segfault issue whenever the cluster I am connecting to is rebalancing - I've gotten quite a few segfaults when adding / removing nodes and then rebalancing.

 Comments   
Comment by Matt Ingenthron [ 28/Jan/13 ]
Sergey: do you think we can reproduce this and solve it for 1.2.2? Let's review on Tues.
Comment by Sergey Avseyev [ 08/Feb/13 ]
There are actually several resolution matching:

Cannot reproduce: I cannot cause segfault

Won't fix: connection_options are for cases when you need to use singleton method Couchbase.bucket, if I had name default_options if it was supposed to setup the defaults. If you decuded to use Couchbase.connect, then just pass options you need to this method directly. In your case try to access the connection using Couchbase.bucket it should be ok

Incomplete: could you attach the backtrace from the coredump?
Comment by Matt Ingenthron [ 12/Feb/13 ]
Sergey: let's carry this one a bit further. I know you and I'd talked about it the other day, but we should try to clarify further what should be done differently (I see your note there, but it's a bit confusing to me) and we should ask for a core dump or stack trace since you've not been able to reproduce it.
Comment by Matt Ingenthron [ 12/Feb/13 ]
After reading that section above a couple of times, and then reading what's in the rdoc at http://www.couchbase.com/autodocs/couchbase-ruby-client-1.2.1/index.html#USAGE, what I think you're saying is:

Either use...
1) Preferred form for production with multiple nodes:
c = Couchbase.connect(:bucket => "mybucket",
                      :node_list => ['example.com:8091', example.net'])

Or
2) If using the Couchbase.bucket singleton, set Couchbase.connection_options and then make calls through Couchbase.bucket like:
Couchbase.connection_options = {:bucket => 'blog'}
Couchbase.bucket.name #=> "blog"
Couchbase.bucket.set("foo", "bar") #=> 3289400178357895424

Do I read that correctly? We may want to tighten up the documentation there, as the next section goes back into Couchbase.connect discussion.
Comment by Sergey Avseyev [ 13/Feb/13 ]
you are correct OK n your reading
Comment by Tim Ray [ 13/Feb/13 ]
I spun up a couple machines to work on this issue. Please contact me directly if you would like access to these machines - I'm more than happy to open them up to you.

Please note that I know you're not supposed to call c = Couchbase.connect without specifying some options, but for it to segfault rather than throw an error is an issue.

A little more backstory: I originally encountered the segfault, realized I wasn't passing correct options, added the options and was on my merry way. Unfortunately, when connecting to the servers I had set up as a test cluster: when I performed rebalance operations after adding or dropping a node I would get occasional segfaults. I remembered the earlier segfaults I was having, forced that error again since it was easy to reproduce, and compared the results. It looks like the same underlying issue affects both.

My theory was that the c extension is assuming that there will always be a couchbase server on the client machine. I don't believe that this accurately reflects all use cases and certainly not mine. I'm unfortunately not familiar enough with c or the codebase to make a patch, though I spent a couple days digging around trying. I think the root cause is essentially that you get a segfault rather than throwing an error whenever the couchbase gem / c extension tries connect to the default, which appears to have a hardcoded expectation of 'localhost'.

irb session and segfault:
-----------------------------------
irb(main):001:0> require 'couchbase'
=> true
irb(main):002:0> c = Couchbase.connect
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase.rb:59: [BUG] Segmentation fault
ruby 1.9.3p385 (2013-02-06 revision 39114) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0026 p:---- s:0092 b:0092 l:000091 d:000091 CFUNC :initialize
c:0025 p:---- s:0090 b:0090 l:000089 d:000089 CFUNC :new
c:0024 p:0027 s:0087 b:0087 l:000086 d:000086 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase.rb:59
c:0023 p:0015 s:0083 b:0083 l:002198 d:0006c0 EVAL (irb):2
c:0022 p:---- s:0080 b:0080 l:000079 d:000079 FINISH
c:0021 p:---- s:0078 b:0078 l:000077 d:000077 CFUNC :eval
c:0020 p:0028 s:0071 b:0071 l:000070 d:000070 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/workspace.rb:80
c:0019 p:0033 s:0064 b:0063 l:000062 d:000062 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/context.rb:254
c:0018 p:0031 s:0058 b:0058 l:000828 d:000057 BLOCK /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:159
c:0017 p:0042 s:0050 b:0050 l:000049 d:000049 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:273
c:0016 p:0011 s:0045 b:0045 l:000828 d:000044 BLOCK /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:156
c:0015 p:0144 s:0041 b:0041 l:000024 d:000040 BLOCK /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:243
c:0014 p:---- s:0038 b:0038 l:000037 d:000037 FINISH
c:0013 p:---- s:0036 b:0036 l:000035 d:000035 CFUNC :loop
c:0012 p:0009 s:0033 b:0033 l:000024 d:000032 BLOCK /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:229
c:0011 p:---- s:0031 b:0031 l:000030 d:000030 FINISH
c:0010 p:---- s:0029 b:0029 l:000028 d:000028 CFUNC :catch
c:0009 p:0023 s:0025 b:0025 l:000024 d:000024 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:228
c:0008 p:0046 s:0022 b:0022 l:000828 d:000828 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:155
c:0007 p:0011 s:0019 b:0019 l:000d08 d:000018 BLOCK /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:70
c:0006 p:---- s:0017 b:0017 l:000016 d:000016 FINISH
c:0005 p:---- s:0015 b:0015 l:000014 d:000014 CFUNC :catch
c:0004 p:0183 s:0011 b:0011 l:000d08 d:000d08 METHOD /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:69
c:0003 p:0039 s:0006 b:0006 l:001358 d:002278 EVAL /home/tray/.rbenv/versions/1.9.3-p385/bin/irb:12
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:001358 d:001358 TOP

-- Ruby level backtrace information ----------------------------------------
/home/tray/.rbenv/versions/1.9.3-p385/bin/irb:12:in `<main>'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:69:in `start'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:69:in `catch'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:70:in `block in start'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:155:in `eval_input'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `each_top_level_statement'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `catch'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `block in each_top_level_statement'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `loop'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb:243:in `block (2 levels) in each_top_level_statement'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:156:in `block in eval_input'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:273:in `signal_status'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb:159:in `block (2 levels) in eval_input'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/context.rb:254:in `evaluate'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/workspace.rb:80:in `evaluate'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/workspace.rb:80:in `eval'
(irb):2:in `irb_binding'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase.rb:59:in `connect'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase.rb:59:in `new'
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase.rb:59:in `initialize'

-- C level backtrace information -------------------------------------------
irb() [0x5214e7]
irb() [0x56991b]
irb(rb_bug+0xb4) [0x56ac04]
irb() [0x4b43a2]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f7d7356ecb0] arguments.c:249
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so(+0xdd86) [0x7f7d6ee78d86] multithread_plugin.c:916
/usr/lib/libcouchbase.so.2(+0x9272) [0x7f7d6ec55272]
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so(+0xe904) [0x7f7d6ee79904] multithread_plugin.c:460
irb(rb_ensure+0xab) [0x418dbb]
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so(+0xdbd7) [0x7f7d6ee78bd7] multithread_plugin.c:893
/usr/lib/libcouchbase.so.2(lcb_wait+0x73) [0x7f7d6ec5b053]
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so(+0xa028) [0x7f7d6ee75028] bucket.c:389
/home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so(cb_bucket_init+0x13e) [0x7f7d6ee75e6e] bucket.c:539
irb() [0x5199f7]
irb(rb_class_new_instance+0x30) [0x44eaf0]
irb() [0x51dff9]
irb() [0x5126ab]
irb() [0x518002]
irb() [0x51845f]
irb() [0x5188ec]
irb() [0x51dff9]
irb() [0x5126ab]
irb() [0x518002]
irb() [0x518e2e]
irb() [0x51e618]
irb(rb_rescue2+0x163) [0x418993]
irb() [0x50d869]
irb() [0x51dff9]
irb() [0x5126ab]
irb() [0x518002]
irb() [0x518e2e]
irb() [0x51e67e]
irb(rb_catch_obj+0xbe) [0x50fa1e]
irb() [0x50faca]
irb() [0x51dff9]
irb() [0x5126ab]
irb() [0x518002]
irb() [0x518e2e]
irb() [0x51e67e]
irb(rb_catch_obj+0xbe) [0x50fa1e]
irb() [0x50faca]
irb() [0x51dff9]
irb() [0x5126ab]
irb() [0x518002]
irb(rb_iseq_eval_main+0xb1) [0x51f081]
irb() [0x41601a]
irb(ruby_run_node+0x36) [0x418506]
irb() [0x415959]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f7d7288076d]
irb() [0x415989]

-- Other runtime information -----------------------------------------------

* Loaded script: irb

* Loaded features:

    0 enumerator.so
    1 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/encdb.so
    2 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/trans/transdb.so
    3 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/defaults.rb
    4 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/rbconfig.rb
    5 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/deprecate.rb
    6 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/exceptions.rb
    7 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/custom_require.rb
    8 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems.rb
    9 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/pathname.so
   10 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/pathname.rb
   11 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/version.rb
   12 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/etc.so
   13 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/config_file.rb
   14 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/requirement.rb
   15 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/dependency.rb
   16 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/platform.rb
   17 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/specification.rb
   18 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/path_support.rb
   19 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/syntax_error.rb
   20 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/psych.so
   21 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/stringio.so
   22 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/node.rb
   23 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/stream.rb
   24 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/document.rb
   25 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/sequence.rb
   26 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/scalar.rb
   27 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/mapping.rb
   28 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes/alias.rb
   29 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/nodes.rb
   30 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/streaming.rb
   31 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors/visitor.rb
   32 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/strscan.so
   33 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/scalar_scanner.rb
   34 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors/to_ruby.rb
   35 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors/emitter.rb
   36 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors/yaml_tree.rb
   37 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/json/ruby_events.rb
   38 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors/json_tree.rb
   39 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors/depth_first.rb
   40 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/visitors.rb
   41 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/handler.rb
   42 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/tree_builder.rb
   43 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/parser.rb
   44 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/omap.rb
   45 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/set.rb
   46 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/coder.rb
   47 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/core_ext.rb
   48 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/date_core.so
   49 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/date/format.rb
   50 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/date.rb
   51 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/deprecated.rb
   52 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/stream.rb
   53 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/json/yaml_events.rb
   54 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/json/tree_builder.rb
   55 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/json/stream.rb
   56 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych/handlers/document_stream.rb
   57 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/psych.rb
   58 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/psych_additions.rb
   59 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/psych_tree.rb
   60 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/yaml.rb
   61 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/syck_hack.rb
   62 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/rubygems_integration.rb
   63 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/shared_helpers.rb
   64 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/fileutils.rb
   65 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/gem_path_manipulation.rb
   66 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/psyched_yaml.rb
   67 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/gem_helpers.rb
   68 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/match_platform.rb
   69 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/rubygems_ext.rb
   70 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/version.rb
   71 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler.rb
   72 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/settings.rb
   73 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest.so
   74 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/digest.rb
   75 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest/sha1.so
   76 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/set.rb
   77 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/definition.rb
   78 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/dependency.rb
   79 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/dsl.rb
   80 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/common.rb
   81 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/generic.rb
   82 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/ftp.rb
   83 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/http.rb
   84 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/https.rb
   85 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/ldap.rb
   86 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/ldaps.rb
   87 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri/mailto.rb
   88 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/uri.rb
   89 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/user_interaction.rb
   90 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/f_sync_dir.rb
   91 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/tar_header.rb
   92 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/iso_8859_1.so
   93 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/zlib.so
   94 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/tar_input.rb
   95 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/tar_output.rb
   96 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/tar_reader/entry.rb
   97 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/tar_reader.rb
   98 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package/tar_writer.rb
   99 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/package.rb
  100 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/format.rb
  101 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/ext/builder.rb
  102 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/ext/configure_builder.rb
  103 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/optparse.rb
  104 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/command.rb
  105 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/ext/ext_conf_builder.rb
  106 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/ext/rake_builder.rb
  107 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/ext.rb
  108 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/require_paths_builder.rb
  109 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/installer.rb
  110 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/remote_fetcher.rb
  111 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/errors.rb
  112 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/text.rb
  113 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/spec_fetcher.rb
  114 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/gem_installer.rb
  115 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/source.rb
  116 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/lockfile_parser.rb
  117 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/lazy_specification.rb
  118 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/tsort.rb
  119 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/forwardable.rb
  120 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/spec_set.rb
  121 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/environment.rb
  122 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/runtime.rb
  123 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/index.rb
  124 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/remote_specification.rb
  125 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/endpoint_specification.rb
  126 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/dep_proxy.rb
  127 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/rubygems/source_index.rb
  128 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/bundler-1.2.3/lib/bundler/setup.rb
  129 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/e2mmap.rb
  130 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/init.rb
  131 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/workspace.rb
  132 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/inspector.rb
  133 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/context.rb
  134 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/extend-command.rb
  135 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/output-method.rb
  136 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/notifier.rb
  137 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/slex.rb
  138 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-token.rb
  139 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/ruby-lex.rb
  140 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/src_encoding.rb
  141 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/magic-file.rb
  142 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/readline.so
  143 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/input-method.rb
  144 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb/locale.rb
  145 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/irb.rb
  146 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/version.rb
  147 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/multi_json-1.6.0/lib/multi_json.rb
  148 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/json/version.rb
  149 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/json/common.rb
  150 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16be.so
  151 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16le.so
  152 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32be.so
  153 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32le.so
  154 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/parser.so
  155 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/generator.so
  156 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/json/ext.rb
  157 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/json.rb
  158 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/multi_json-1.6.0/lib/multi_json/adapters/json_common.rb
  159 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/multi_json-1.6.0/lib/multi_json/adapters/json_gem.rb
  160 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/multi_json-1.6.0/lib/multi_json/version.rb
  161 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/ext/multi_json_fix.rb
  162 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/yaji/version.rb
  163 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/parser_ext.so
  164 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/yaji.rb
  165 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so
  166 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/constants.rb
  167 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/utils.rb
  168 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/bucket.rb
  169 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/view_row.rb
  170 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/base64.rb
  171 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/view.rb
  172 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/result.rb
  173 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase/cluster.rb
  174 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase.rb

* Process memory map:

00400000-0061c000 r-xp 00000000 ca:01 164081 /home/tray/.rbenv/versions/1.9.3-p385/bin/ruby
0081c000-0081d000 r--p 0021c000 ca:01 164081 /home/tray/.rbenv/versions/1.9.3-p385/bin/ruby
0081d000-0081f000 rw-p 0021d000 ca:01 164081 /home/tray/.rbenv/versions/1.9.3-p385/bin/ruby
0081f000-0083c000 rw-p 00000000 00:00 0
00e44000-01bc8000 rw-p 00000000 00:00 0 [heap]
7f7d6e829000-7f7d6e83e000 r-xp 00000000 ca:01 608378 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d6e83e000-7f7d6ea3d000 ---p 00015000 ca:01 608378 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d6ea3d000-7f7d6ea3e000 r--p 00014000 ca:01 608378 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d6ea3e000-7f7d6ea3f000 rw-p 00015000 ca:01 608378 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7d6ea3f000-7f7d6ea4b000 r-xp 00000000 ca:01 608534 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d6ea4b000-7f7d6ec4a000 ---p 0000c000 ca:01 608534 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d6ec4a000-7f7d6ec4b000 r--p 0000b000 ca:01 608534 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d6ec4b000-7f7d6ec4c000 rw-p 0000c000 ca:01 608534 /lib/x86_64-linux-gnu/libnss_files-2.15.so
7f7d6ec4c000-7f7d6ec69000 r-xp 00000000 ca:01 592591 /usr/lib/libcouchbase.so.2.0.3
7f7d6ec69000-7f7d6ee69000 ---p 0001d000 ca:01 592591 /usr/lib/libcouchbase.so.2.0.3
7f7d6ee69000-7f7d6ee6a000 r--p 0001d000 ca:01 592591 /usr/lib/libcouchbase.so.2.0.3
7f7d6ee6a000-7f7d6ee6b000 rw-p 0001e000 ca:01 592591 /usr/lib/libcouchbase.so.2.0.3
7f7d6ee6b000-7f7d6ee88000 r-xp 00000000 ca:01 230155 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so
7f7d6ee88000-7f7d6f087000 ---p 0001d000 ca:01 230155 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so
7f7d6f087000-7f7d6f088000 r--p 0001c000 ca:01 230155 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so
7f7d6f088000-7f7d6f089000 rw-p 0001d000 ca:01 230155 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/couchbase-1.2.2/lib/couchbase_ext.so
7f7d6f089000-7f7d6f08a000 rw-p 00000000 00:00 0
7f7d6f08a000-7f7d6f095000 r-xp 00000000 ca:01 230035 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/parser_ext.so
7f7d6f095000-7f7d6f294000 ---p 0000b000 ca:01 230035 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/parser_ext.so
7f7d6f294000-7f7d6f295000 r--p 0000a000 ca:01 230035 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/parser_ext.so
7f7d6f295000-7f7d6f296000 rw-p 0000b000 ca:01 230035 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/gems/1.9.1/gems/yaji-0.3.3/lib/parser_ext.so
7f7d6f296000-7f7d6f29f000 r-xp 00000000 ca:01 172042 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/generator.so
7f7d6f29f000-7f7d6f49e000 ---p 00009000 ca:01 172042 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/generator.so
7f7d6f49e000-7f7d6f49f000 r--p 00008000 ca:01 172042 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/generator.so
7f7d6f49f000-7f7d6f4a0000 rw-p 00009000 ca:01 172042 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/generator.so
7f7d6f4a0000-7f7d6f4a1000 r-xp 00000000 ca:01 164108 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32le.so
7f7d6f4a1000-7f7d6f6a0000 ---p 00001000 ca:01 164108 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32le.so
7f7d6f6a0000-7f7d6f6a1000 r--p 00000000 ca:01 164108 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32le.so
7f7d6f6a1000-7f7d6f6a2000 rw-p 00001000 ca:01 164108 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32le.so
7f7d6f6a2000-7f7d6f6a3000 r-xp 00000000 ca:01 164106 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32be.so
7f7d6f6a3000-7f7d6f8a2000 ---p 00001000 ca:01 164106 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32be.so
7f7d6f8a2000-7f7d6f8a3000 r--p 00000000 ca:01 164106 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32be.so
7f7d6f8a3000-7f7d6f8a4000 rw-p 00001000 ca:01 164106 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_32be.so
7f7d6f8a4000-7f7d6f8a5000 r-xp 00000000 ca:01 164110 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16le.so
7f7d6f8a5000-7f7d6faa4000 ---p 00001000 ca:01 164110 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16le.so
7f7d6faa4000-7f7d6faa5000 r--p 00000000 ca:01 164110 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16le.so
7f7d6faa5000-7f7d6faa6000 rw-p 00001000 ca:01 164110 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16le.so
7f7d6faa6000-7f7d6faa7000 r-xp 00000000 ca:01 164093 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16be.so
7f7d6faa7000-7f7d6fca6000 ---p 00001000 ca:01 164093 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16be.so
7f7d6fca6000-7f7d6fca7000 r--p 00000000 ca:01 164093 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16be.so
7f7d6fca7000-7f7d6fca8000 rw-p 00001000 ca:01 164093 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/utf_16be.so
7f7d6fca8000-7f7d6fcad000 r-xp 00000000 ca:01 172041 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/parser.so
7f7d6fcad000-7f7d6fead000 ---p 00005000 ca:01 172041 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/parser.so
7f7d6fead000-7f7d6feae000 r--p 00005000 ca:01 172041 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/parser.so
7f7d6feae000-7f7d6feaf000 rw-p 00006000 ca:01 172041 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/json/ext/parser.so
7f7d6feaf000-7f7d6fed1000 r-xp 00000000 ca:01 608418 /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f7d6fed1000-7f7d700d1000 ---p 00022000 ca:01 608418 /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f7d700d1000-7f7d700d5000 r--p 00022000 ca:01 608418 /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f7d700d5000-7f7d700d6000 rw-p 00026000 ca:01 608418 /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f7d700d6000-7f7d7010f000 r-xp 00000000 ca:01 608410 /lib/x86_64-linux-gnu/libreadline.so.6.2
7f7d7010f000-7f7d7030f000 ---p 00039000 ca:01 608410 /lib/x86_64-linux-gnu/libreadline.so.6.2
7f7d7030f000-7f7d70311000 r--p 00039000 ca:01 608410 /lib/x86_64-linux-gnu/libreadline.so.6.2
7f7d70311000-7f7d70317000 rw-p 0003b000 ca:01 608410 /lib/x86_64-linux-gnu/libreadline.so.6.2
7f7d70317000-7f7d70318000 rw-p 00000000 00:00 0
7f7d70318000-7f7d7031e000 r-xp 00000000 ca:01 164149 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/readline.so
7f7d7031e000-7f7d7051d000 ---p 00006000 ca:01 164149 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/readline.so
7f7d7051d000-7f7d7051e000 r--p 00005000 ca:01 164149 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/readline.so
7f7d7051e000-7f7d7051f000 rw-p 00006000 ca:01 164149 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/readline.so
7f7d7051f000-7f7d7052d000 r-xp 00000000 ca:01 164088 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/zlib.so
7f7d7052d000-7f7d7072c000 ---p 0000e000 ca:01 164088 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/zlib.so
7f7d7072c000-7f7d7072d000 r--p 0000d000 ca:01 164088 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/zlib.so
7f7d7072d000-7f7d7072e000 rw-p 0000e000 ca:01 164088 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/zlib.so
7f7d7072e000-7f7d70730000 r-xp 00000000 ca:01 164122 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/iso_8859_1.so
7f7d70730000-7f7d7092f000 ---p 00002000 ca:01 164122 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/iso_8859_1.so
7f7d7092f000-7f7d70930000 r--p 00001000 ca:01 164122 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/iso_8859_1.so
7f7d70930000-7f7d70931000 rw-p 00002000 ca:01 164122 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/iso_8859_1.so
7f7d70931000-7f7d70934000 r-xp 00000000 ca:01 164148 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest.so
7f7d70934000-7f7d70b33000 ---p 00003000 ca:01 164148 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest.so
7f7d70b33000-7f7d70b34000 r--p 00002000 ca:01 164148 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest.so
7f7d70b34000-7f7d70b35000 rw-p 00003000 ca:01 164148 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest.so
7f7d70b35000-7f7d70b4b000 r-xp 00000000 ca:01 608425 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f7d70b4b000-7f7d70d4a000 ---p 00016000 ca:01 608425 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f7d70d4a000-7f7d70d4b000 r--p 00015000 ca:01 608425 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f7d70d4b000-7f7d70d4c000 rw-p 00016000 ca:01 608425 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
7f7d70d4c000-7f7d70eeb000 r-xp 00000000 ca:01 608270 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f7d70eeb000-7f7d710ea000 ---p 0019f000 ca:01 608270 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f7d710ea000-7f7d71105000 r--p 0019e000 ca:01 608270 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f7d71105000-7f7d71110000 rw-p 001b9000 ca:01 608270 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f7d71110000-7f7d71114000 rw-p 00000000 00:00 0
7f7d71114000-7f7d71115000 r-xp 00000000 ca:01 164156 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest/sha1.so
7f7d71115000-7f7d71314000 ---p 00001000 ca:01 164156 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest/sha1.so
7f7d71314000-7f7d71315000 r--p 00000000 ca:01 164156 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest/sha1.so
7f7d71315000-7f7d71316000 rw-p 00001000 ca:01 164156 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/digest/sha1.so
7f7d71316000-7f7d71348000 r-xp 00000000 ca:01 164178 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/date_core.so
7f7d71348000-7f7d71548000 ---p 00032000 ca:01 164178 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/date_core.so
7f7d71548000-7f7d71549000 r--p 00032000 ca:01 164178 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/date_core.so
7f7d71549000-7f7d7154a000 rw-p 00033000 ca:01 164178 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/date_core.so
7f7d7154a000-7f7d7154c000 rw-p 00000000 00:00 0
7f7d7154c000-7f7d71551000 r-xp 00000000 ca:01 164168 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/strscan.so
7f7d71551000-7f7d71750000 ---p 00005000 ca:01 164168 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/strscan.so
7f7d71750000-7f7d71751000 r--p 00004000 ca:01 164168 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/strscan.so
7f7d71751000-7f7d71752000 rw-p 00005000 ca:01 164168 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/strscan.so
7f7d71752000-7f7d71759000 r-xp 00000000 ca:01 164162 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/stringio.so
7f7d71759000-7f7d71958000 ---p 00007000 ca:01 164162 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/stringio.so
7f7d71958000-7f7d71959000 r--p 00006000 ca:01 164162 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/stringio.so
7f7d71959000-7f7d7195a000 rw-p 00007000 ca:01 164162 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/stringio.so
7f7d7195a000-7f7d7197a000 r-xp 00000000 ca:01 164070 /home/tray/.rbenv/versions/1.9.3-p385/lib/libyaml-0.so.2.0.2
7f7d7197a000-7f7d71b79000 ---p 00020000 ca:01 164070 /home/tray/.rbenv/versions/1.9.3-p385/lib/libyaml-0.so.2.0.2
7f7d71b79000-7f7d71b7a000 r--p 0001f000 ca:01 164070 /home/tray/.rbenv/versions/1.9.3-p385/lib/libyaml-0.so.2.0.2
7f7d71b7a000-7f7d71b7b000 rw-p 00020000 ca:01 164070 /home/tray/.rbenv/versions/1.9.3-p385/lib/libyaml-0.so.2.0.2
7f7d71b7b000-7f7d71b81000 r-xp 00000000 ca:01 164167 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/psych.so
7f7d71b81000-7f7d71d80000 ---p 00006000 ca:01 164167 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/psych.so
7f7d71d80000-7f7d71d81000 r--p 00005000 ca:01 164167 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/psych.so
7f7d71d81000-7f7d71d82000 rw-p 00006000 ca:01 164167 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/psych.so
7f7d71d82000-7f7d71d85000 r-xp 00000000 ca:01 164153 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/etc.so
7f7d71d85000-7f7d71f84000 ---p 00003000 ca:01 164153 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/etc.so
7f7d71f84000-7f7d71f85000 r--p 00002000 ca:01 164153 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/etc.so
7f7d71f85000-7f7d71f86000 rw-p 00003000 ca:01 164153 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/etc.so
7f7d71f86000-7f7d71f8e000 r-xp 00000000 ca:01 164180 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/pathname.so
7f7d71f8e000-7f7d7218d000 ---p 00008000 ca:01 164180 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/pathname.so
7f7d7218d000-7f7d7218e000 r--p 00007000 ca:01 164180 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/pathname.so
7f7d7218e000-7f7d7218f000 rw-p 00008000 ca:01 164180 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/pathname.so
7f7d7218f000-7f7d72191000 r-xp 00000000 ca:01 164132 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/trans/transdb.so
7f7d72191000-7f7d72391000 ---p 00002000 ca:01 164132 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/trans/transdb.so
7f7d72391000-7f7d72392000 r--p 00002000 ca:01 164132 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/trans/transdb.so
7f7d72392000-7f7d72393000 rw-p 00003000 ca:01 164132 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/trans/transdb.so
7f7d72393000-7f7d72395000 r-xp 00000000 ca:01 164095 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/encdb.so
7f7d72395000-7f7d72594000 ---p 00002000 ca:01 164095 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/encdb.so
7f7d72594000-7f7d72595000 r--p 00001000 ca:01 164095 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/encdb.so
7f7d72595000-7f7d72596000 rw-p 00002000 ca:01 164095 /home/tray/.rbenv/versions/1.9.3-p385/lib/ruby/1.9.1/x86_64-linux/enc/encdb.so
7f7d72596000-7f7d7285f000 r--p 00000000 ca:01 590866 /usr/lib/locale/locale-archive
7f7d7285f000-7f7d72a14000 r-xp 00000000 ca:01 608524 /lib/x86_64-linux-gnu/libc-2.15.so
7f7d72a14000-7f7d72c13000 ---p 001b5000 ca:01 608524 /lib/x86_64-linux-gnu/libc-2.15.so
7f7d72c13000-7f7d72c17000 r--p 001b4000 ca:01 608524 /lib/x86_64-linux-gnu/libc-2.15.so
7f7d72c17000-7f7d72c19000 rw-p 001b8000 ca:01 608524 /lib/x86_64-linux-gnu/libc-2.15.so
7f7d72c19000-7f7d72c1e000 rw-p 00000000 00:00 0
7f7d72c1e000-7f7d72d19000 r-xp 00000000 ca:01 608532 /lib/x86_64-linux-gnu/libm-2.15.so
7f7d72d19000-7f7d72f18000 ---p 000fb000 ca:01 608532 /lib/x86_64-linux-gnu/libm-2.15.so
7f7d72f18000-7f7d72f19000 r--p 000fa000 ca:01 608532 /lib/x86_64-linux-gnu/libm-2.15.so
7f7d72f19000-7f7d72f1a000 rw-p 000fb000 ca:01 608532 /lib/x86_64-linux-gnu/libm-2.15.so
7f7d72f1a000-7f7d72f23000 r-xp 00000000 ca:01 608533 /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f7d72f23000-7f7d73123000 ---p 00009000 ca:01 608533 /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f7d73123000-7f7d73124000 r--p 00009000 ca:01 608533 /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f7d73124000-7f7d73125000 rw-p 0000a000 ca:01 608533 /lib/x86_64-linux-gnu/libcrypt-2.15.so
7f7d73125000-7f7d73153000 rw-p 00000000 00:00 0
7f7d73153000-7f7d73155000 r-xp 00000000 ca:01 608523 /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d73155000-7f7d73355000 ---p 00002000 ca:01 608523 /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d73355000-7f7d73356000 r--p 00002000 ca:01 608523 /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d73356000-7f7d73357000 rw-p 00003000 ca:01 608523 /lib/x86_64-linux-gnu/libdl-2.15.so
7f7d73357000-7f7d7335e000 r-xp 00000000 ca:01 608528 /lib/x86_64-linux-gnu/librt-2.15.so
7f7d7335e000-7f7d7355d000 ---p 00007000 ca:01 608528 /lib/x86_64-linux-gnu/librt-2.15.so
7f7d7355d000-7f7d7355e000 r--p 00006000 ca:01 608528 /lib/x86_64-linux-gnu/librt-2.15.so
7f7d7355e000-7f7d7355f000 rw-p 00007000 ca:01 608528 /lib/x86_64-linux-gnu/librt-2.15.so
7f7d7355f000-7f7d73577000 r-xp 00000000 ca:01 608526 /lib/x86_64-linux-gnu/libpthread-2.15.so
7f7d73577000-7f7d73776000 ---p 00018000 ca:01 608526 /lib/x86_64-linux-gnu/libpthread-2.15.so
7f7d73776000-7f7d73777000 r--p 00017000 ca:01 608526 /lib/x86_64-linux-gnu/libpthread-2.15.so
7f7d73777000-7f7d73778000 rw-p 00018000 ca:01 608526 /lib/x86_64-linux-gnu/libpthread-2.15.so
7f7d73778000-7f7d7377c000 rw-p 00000000 00:00 0
7f7d7377c000-7f7d7379e000 r-xp 00000000 ca:01 608538 /lib/x86_64-linux-gnu/ld-2.15.so
7f7d73889000-7f7d73890000 r--s 00000000 ca:01 11782 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7f7d73890000-7f7d73997000 rw-p 00000000 00:00 0
7f7d73997000-7f7d73998000 rw-p 00000000 00:00 0
7f7d73998000-7f7d73999000 ---p 00000000 00:00 0
7f7d73999000-7f7d7399e000 rw-p 00000000 00:00 0
7f7d7399e000-7f7d7399f000 r--p 00022000 ca:01 608538 /lib/x86_64-linux-gnu/ld-2.15.so
7f7d7399f000-7f7d739a1000 rw-p 00023000 ca:01 608538 /lib/x86_64-linux-gnu/ld-2.15.so
7fff3ab15000-7fff3ab36000 rw-p 00000000 00:00 0 [stack]
7fff3ab8a000-7fff3ab8b000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]


[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

Aborted (core dumped)
Comment by Sergey Avseyev [ 15/Feb/13 ]
http://review.couchbase.org/24620

Segmentation fault when the hostname resolved into several addresses
and first of them reject couchbase connections. In this case
libcouchbase releases socket and destroys the the event structure, so
that subsequent calls will try to dereference NULL-event.

The event struct should be deallocated only on handle destruction.

Tim, your ubuntu instance had two interfaces:

savseyev@couchbase-devel:~/code/couchtest$ host localhost
localhost has address 127.0.0.1
localhost has IPv6 address ::1

Unlike mine:

~ $ host localhost
localhost has address 127.0.0.1

This helped me to fix this particular issue.




[CCBC-147] function lcb_wait works infinitly after trying to connect (by calling lcb_connect) to Couchbase server that is on Pending state Created: 19/Dec/12  Updated: 05/Mar/13  Resolved: 05/Mar/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.4
Security Level: Public

Type: Bug Priority: Major
Reporter: Haster Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: I reproduced a bug on Linux Rad Hat 5.0 x64 and on Windows XP x32
Couchbase server version 1.8.1
libcouchbase version 2.0.1

Attachments: File cloader.rar     Text File NMakefile_new    
Issue Links:
Duplicate
is duplicated by CCBC-167 lcb_wait() waits infinitely Closed

 Description   
Start server with one node.
Create many buckets, that uses all allowed memory -> Server have to chnage status on Pending (In my situation server stay in Pending for all time and can't change it to Up)

After that try to connect to one of the bucket by calling lcb_connect.
Call lcb_wait to wait for connection is done.
As a result lcb_wait work infinitly and timeout doesn't happen

Below I share call stack in Red Hat Linux:
#0 0x00000034350d4473 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1 0x00002ad81b1dccc9 in ?? () from ./lib/libevent-2.0.so.5
#2 0x00002ad81b1c9cdc in event_base_loop () from ./lib/libevent-2.0.so.5
#3 0x00002ad8174c1dc6 in lcb_wait () from ./lib/libcouchbase.so.2
#4 0x000000000043c0e1 in Couchbase::connect (this=0x60df508) at couchbase_loader_source/couchbase.cpp:152
#5 0x000000000043c8b7 in connect_to_bucket (cbase=..., config=..., bucket_name=...) at couchbase_loader_source/couchbase.cpp:478
#6 0x0000000000440924 in couchbase_loader::writer_thread::writer_thread (this=0x60df480, config=..., bucket_name=..., dbh=...) at couchbase_loader_source/writer_thread.cpp:25
#7 0x000000000042168c in main (argc=1, argv=0x7fff22162028) at couchbase_loader_source/couchbase_loader.cpp:175


 Comments   
Comment by Haster [ 26/Dec/12 ]
I’ve investigated a little this problem and I have additional info.

My error happened scenario:

First of all I try to connect to some bucket (Primary for example) which is absent

lcb_connect
lcb_wait

here I get error. Then I create bucket, destroy instance by calling lcb_destroy and try connect to bucket again:

func_create_bucket()
sleep(some_time)
lcb_destroy()
lcb_create()
lcb_connect()
lcb_wait()

After that I receive 3 messages from my error_callback function, where errinfo is “Number of vBuckets must be a power of two > 0 and <= 65536”
And deadlock happened in lcb_wait.
Comment by Haster [ 27/Dec/12 ]
I've investigated this issue more deeply.
The problem take place then connection timeout happend.

connect
lcb_wait
  lcb_io_run_event_loop (for Windows Platform)
  ...
  select

  if (ret == 0) <-- 0 then timeout happend
  {
                                  <-- Here our problems begin
  }

After select returns zero code tries to reconnect in callback function(is it good idea? Maybe it is better to exit from loop and return error?), but some structures (as I think) are corrupted after that.
After that event_loop calls select function, but timeout variable contain zero value (infinitly execution)...

As I think, problem is near here
Comment by Sergey Avseyev [ 10/Jan/13 ]
could you post a piece of source code, demonstrating the issue?
Comment by Sergey Avseyev [ 10/Jan/13 ]
Possibly this patch has been fixed the issue https://github.com/couchbase/libcouchbase/commit/d4948192439e61a8cc23d5e8572e81db1aebef7f

Could you verify with libcouchbase from master?

To do so either pull and build the sources:

git clone git://github.com/couchbase/libcouchbase.git
cd libcouchbase
./config/autorun.sh && ./configure && make && sudo make install

or install from snapsot deb/rpm repositories, like for example for recent ubuntus

sudo wget -O/etc/apt/sources.list.d/couchbase-snapshot.list http://packages.couchbase.com/snapshot/ubuntu/couchbase-ubuntu1110.list
sudo aptitude update
sudo aptitude install libcouchbase-dev libcouchbase2-bin
Comment by Haster [ 10/Jan/13 ]
My code is very big :one application server creates many processes (couchbase_loader)(one process for each cache) and whose processes operate with couchbase.
But also they cooperate with oracle database...
If you want I can share all code, but you need Oracle to run it. Or I can try to write more simple testcase...

Also I will check patch tomorrow and share results
Comment by Haster [ 11/Jan/13 ]
It looks good now. Thanks a lot
Comment by Haster [ 11/Jan/13 ]
Sergey, I reproduced this issue on Windows platform.
Now it works more stable, but than I've tried to create 9 buckets in same time by executing 9 copies of my program 4 of them deadlocked in lcb_wait function.
Comment by Haster [ 13/Jan/13 ]
Добавлена тестовая программа для воспроизведения проблемы
Comment by Sergey Avseyev [ 06/Feb/13 ]
Fixed invalid memory access in win32 plugin http://review.couchbase.org/24451




[CCBC-178] Build error on solaris/sparc: -Werror=cast-align Created: 12/Feb/13  Updated: 05/Mar/13  Resolved: 14/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.3
Fix Version/s: 2.0.4
Security Level: Public

Type: Task Priority: Major
Reporter: Sergey Avseyev Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
  CC src/libcouchbase_la-server.lo
src/server.c: In function 'lcb_failout_observe_request':
src/server.c:47:8: error: cast increases required alignment of target type [-Werror=cast-align]





[CCBC-189] Create a compat mode that cache the configuration objects Created: 26/Feb/13  Updated: 26/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Trond Norbye Assignee: Trond Norbye
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-187] Set TCP_NODELAY flag Created: 25/Feb/13  Updated: 25/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Haster Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We think it is good idea to set TCP_NODELAY flag to switch off the Nagle mode
(don't wait for sending a short packets)
Couchbase server, as we think, use this flag




[CCBC-184] There are a lot of connections are established after restarting Couchbase server Created: 18/Feb/13  Updated: 25/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Haster Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Linux, HP-UX
libcouchbase-2.0.3
Couchbase Server 1.8.1


 Description   
1) Start Couchbase Server
2) Start to recieve some key from Couchbase server using a client based on libcouchbase library. We use about 250 threads for receiving key
3) Restart Couchbase server
4) After that a lot of connections are created (all of then in ESTABLISHED state). And soon quota for handlers is over.

 Comments   
Comment by Trond Norbye [ 21/Feb/13 ]
I'm not sure I understand the problem here. Are you seeing all of the connections on the server node or on the client node (or is this the same machine)?

If this is on the client nodes, are the "old" connections not properly shut down? or does it seem like it is creating multiple new connections per libcouchbase instance?
Comment by Haster [ 25/Feb/13 ]
The sockets are created on client side.
We investigate this problem and found that this truble happend than we try to get many values from Couchbase.
lcb_get();
lcb_get();
...
lcb_get();
lcb_wait();
And if we have temporary errors libcouchbase make reconnect many times.
I don't know is this bug or not. But we avoid this problem by receiving one get a time and waiting if temporary error is happend




[CCBC-186] Request to provide concrete examples in the documentation for OBSERVE functionality Created: 21/Feb/13  Updated: 21/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Saran Kumar Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://support.couchbase.com/tickets/2608

Customer are interested in using OBSERVE functionality to allow them to check whether items have been replicated successfully, as they are worried about unreplicated items.

https://github.com/couchbase/libcouchbase/blob/master/man/man3couchbase/lcb_observe.3couchbase.txt
has basic info on the OBSERE functionality in libcouchbase, but we need to provide concrete examples on how to check multiple keys having been replicated successfully (they do a bulk write, so should be more efficient to wait for all of them, rather than wait for them individually)

Examples provided to the customer :

You will find the below example very useful for simple operations :

https://github.com/couchbase/php-ext-couchbase/blob/master/simple_observe.c

The below example would provide more insights when you have any performance or reliability issues :

https://github.com/couchbase/php-ext-couchbase/blob/master/observe.c

The basic idea here is to constantly "poll" the keys (sleeping within a given interval) until all of them are declared as "OK". The structures involved are the "observe_expectation" (persistence/replication requirements), the "pollprefs" (i.e. timeout, interval, etc.) and the "keystate" - this keeps tabs on how the actual status of each key matches up with the specified criteria. The lcb_observe callback is called for each "status report" for each key from each server; so it is necessary to check and see if the requirements have been met during each individual update.




[CCBC-2] getting started guide needed Created: 15/Aug/11  Updated: 18/Feb/13  Resolved: 18/Feb/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Matt Ingenthron Assignee: MC Brown
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Existing libcouchbase has good built in documentation, but lacks a getting started guide.

 Comments   
Comment by MC Brown [ 18/Feb/13 ]
I've updated the libcouchbase getting started with install and basic app/compile instructions




[CCBC-183] Implement approach to specify behaviour for lcb_get_replica() Created: 15/Feb/13  Updated: 15/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The users might want to ask only first or single replica




[CCBC-177] Docs: Reference installation Created: 11/Feb/13  Updated: 11/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.3
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Perry Krug Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Customers have lots of choices at their fingertips, and many of those choices will result in a non-functioning installation, or at least a lot of time spent figuring out what the right pieces are.

Can we please provide as much details as possible about a reference installation that we can be sure will work for a customer who is just getting started? This is outside of our support for any one operatin system, but more about telling the customer "if you install with these versions and packages, this code will work"

-OS
-other packages/modules required and/or tested with




[CCBC-149] Docs: typo under Mac installation instructions Created: 20/Dec/12  Updated: 08/Feb/13  Resolved: 08/Feb/13

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://www.couchbase.com/develop/c/current

The text for the Mac installation instructions needs to be cleaned up a bit.




[CCBC-170] After upgrading to 2.0.2 with CBC163, I now get timesouts again!!! Created: 22/Jan/13  Updated: 06/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Michael Leib Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentsOS 5.X, 64 Bit, libev version


 Description   
I was using 2.0.0 beta3 with the patch for TIMEOUTS manually applied....this was working fine but I had high CPU. This ran for weeks (if not a month) without any
timeouts at all...I was the happiest with lcb as I had been in a long time. However, after getting some complaints from my server admins and NOC, I upgraded to
2.0.2 with the fix for high CPU...

I rebuilt all modules and rolled to production. It's now been about 4 hours and I have received 2 timeouts as to where I had NONE for weeks without this change.

The load is now normal though but somehow something has broken because of this fix.

I looked at the change with CBC-163, but I don't see how this could have broken the code...but, it has - I promise. Can someone please take a look?

HELP!

 Comments   
Comment by Michael Leib [ 06/Feb/13 ]
I have changed my code around so that I have an ev_timer throttle the millions of lcb_get() requests that I need to make over multiple request chunks from a queue rather than doing it all in a single loop and then letting libev handle it all and that appears to be working. There is still be a problem with the latest code if you do millions of requests within a function call before control is returned to the libev event loop. This doesn't happen everytime, but it does happen frequently enough.

Keeping the number of lcb_get() calls sitting on the event stack low makes a BIG difference.

Once again, with the version that didn't have the high-load fix implemented worked great (except for the load)...when the fix was applied, large amounts of pending events cause TIMEOUT returns from lcb_get(),
Comment by Michael Leib [ 06/Feb/13 ]
And, of course, event with an ev_timer throttle of .01 secs, performance is about 1/4 of what it was when I just threw everything on the queue -
basically about 5k/sec throughput vs 20k/sec just focusing on lcb_get() returns. My CB Cluster (2 servers) is reporting 50K ops/sec consistently
without issue.




[CCBC-157] libvbucket should validate vbucket indices before performing lookup Created: 27/Dec/12  Updated: 06/Feb/13  Resolved: 31/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Otherwise, this will cause a segfault and buffer overrun;
e.g. vb=44444 should raise an error rather than crash

 Comments   
Comment by Matt Ingenthron [ 29/Jan/13 ]
Also, anything <0.
Comment by Sergey Avseyev [ 31/Jan/13 ]
It cannot return negative value or something too big

https://github.com/couchbase/libcouchbase/blob/master/contrib/libvbucket/vbucket.c#L705-L711




[CCBC-131] Views sometimes failing if all nodes lack couchApiBase Created: 29/Nov/12  Updated: 06/Feb/13  Resolved: 04/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
LCB_NOT_SUPPORTED is returned.

 Comments   
Comment by Mark Nunberg [ 30/Nov/12 ]
http://review.couchbase.org/22918




[CCBC-153] core dump whenever CouchBase connection attempt fails Created: 21/Dec/12  Updated: 06/Feb/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Blocker
Reporter: James Mauss Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
We are seeing a core dump whenever CouchBase connection attempt fails for the first time and we try to reconnect.
This is a case when the client comes up before the Couchbase server.



 Comments   
Comment by Sergey Avseyev [ 02/Jan/13 ]
http://review.couchbase.org/23663

I've fixed one usage of addrinfo struct, is it possible to get minimal script to reproduce the issue, or maybe you can grab the patch, apply and verify it?
Comment by Sergey Avseyev [ 09/Jan/13 ]
Ok, I've found the issue, and they are right, rebuild instance will of course fix the issue. So there is workaround. I'm working on proper solution.
Comment by Sergey Avseyev [ 09/Jan/13 ]
There is some state which isn't reset on the second connection attempt: like instance->backup_idx and instance->configuration_callback_last. I'm working on the fix now
Comment by Sergey Avseyev [ 09/Jan/13 ]
The fix http://review.couchbase.org/23809
Comment by Sergey Avseyev [ 10/Jan/13 ]
Marking it as resolved and reassigning to James for feedback




[CCBC-160] Request timeout while connecting to protected bucket Created: 03/Jan/13  Updated: 06/Feb/13  Resolved: 05/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Sergey Avseyev Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Current master version 2.0.1-23-ga0b9d07


 Description   
The bug was introduced in this fix http://www.couchbase.com/issues/browse/CCBC-159

To reproduce just try to connect to SASL protected bucket

 Comments   
Comment by Sergey Avseyev [ 03/Jan/13 ]
Fixed in this patch http://review.couchbase.org/23684




[CCBC-155] Observe will malfunction in the case of multiple keys and server failure Created: 22/Dec/12  Updated: 06/Feb/13  Resolved: 05/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Observe requests and responses may contain multiple keys, but this is not handled during the server failure phase in the code; therefore a user will not get callbacks for all the keys.

We need to make a common handler for observe requests and responses

 Comments   
Comment by Mark Nunberg [ 31/Dec/12 ]
http://review.couchbase.org/#/c/23527/




libcouchbase test improvements (CCBC-137)

[CCBC-140] add http tests to libcouchbase Created: 05/Sep/12  Updated: 06/Feb/13  Resolved: 06/Sep/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-154] Missing documentation for some error codes Created: 22/Dec/12  Updated: 06/Feb/13  Resolved: 29/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Trond Norbye Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
LCB_SERVER_BUG and LCB_PLUGIN_VERSION_MISMATCH is not documented with lcb_strerror




[CCBC-156] don't retry OBSERVE on NOT_MY_VBUCKET Created: 23/Dec/12  Updated: 06/Feb/13  Resolved: 05/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Retrying observe on NOT_MY_VBUCKET might lead to an infinite network loop as NOT_MY_VBUCKET is received if any of the vbuckets are not belonging to the server.

This could happen if an OBSERVE command happens during a configuration change.

 Comments   
Comment by Matt Ingenthron [ 23/Dec/12 ]
But retry is expected, just on the next loop through (or one of them, eventually) the retry should hit the new location for the given vbucket.
Comment by Mark Nunberg [ 24/Dec/12 ]
When OBSERVE is used with multiple keys, the keys are encoded into the response like so:

[entry]
vb=13
key="foo"

[entry]
vb=53
key = "bar"

etc.

From what I've seen (see my comments on OBSERVE wiki), the server will respond with NOT_MY_VBUCKET if *any* of the keys mentioned therein are not controlled by the server (as either master or replica). When a configuration change occurs (which it very well may) there is no guarantee that all the vbuckets encoded into the request will still be hosted by that same server.

This is specifically for 'multi-observe'
Comment by Matt Ingenthron [ 24/Dec/12 ]
That all makes sense to me. So, when retrying, if we re-distribute to the nodes based on the new vbucket locations, all will be well, right? Are you saying multi-observe doesn't do this? Just trying to understand.
Comment by Mark Nunberg [ 24/Dec/12 ]
I'm saying we don't actually re-construct the packet (and well, we can't), we just re-send the packet out.

For single-key packets this workds because the vbid is at the beginning fo the header, so we just calculate the vbucket and ship it to a different node. For observe however the vbucket information is mixed, and is not encoded into the header of the packet..




[CCBC-132] Making http request to ECONNREFUSED node seems to loop forever... Created: 30/Nov/12  Updated: 06/Feb/13  Resolved: 03/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
To reproduce, simply try to issue a raw request to say, "localhost:2"

 Comments   
Comment by Mark Nunberg [ 30/Nov/12 ]
static void
test_refused(lcb_t handle)
{
    lcb_http_cmd_t cmd;
    lcb_http_request_t req;

    memset(&cmd, 0, sizeof(cmd));
    cmd.version = 1;
    OV1(&cmd).host = "127.0.0.1:2";
    OV1(&cmd).path = "/";
    OV1(&cmd).npath = 1;
    OV1(&cmd).content_type = "application/json";
    OV1(&cmd).method = LCB_HTTP_METHOD_GET;

    lcb_error_t err;
    err = lcb_make_http_request(handle, NULL, LCB_HTTP_TYPE_RAW, &cmd, &req);
    assert(err == LCB_SUCCESS);
    lcb_wait(handle);
    printf("Wait done..\n");
}



mnunberg@csure:~/src/sdkd-cpp/src/views$ time ./test
^C

real 1m37.361s
user 0m9.861s
sys 1m27.213s
mnunberg@csure:~/src/sdkd-cpp/src/views$
Comment by Mark Nunberg [ 30/Nov/12 ]
not really "blocking" me.. but this is a fatal issue..
Comment by Mark Nunberg [ 03/Dec/12 ]
https://github.com/couchbase/libcouchbase/commit/e37a9fab6d8ae7f556082a339cd8561ec1e93f32




[CCBC-133] HTTP Request hangs during failover Created: 02/Dec/12  Updated: 06/Feb/13  Resolved: 03/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
lcb_wait hangs..

Code:

    lcb_http_request_t htreq;
    lcb_error_t lcb_err;
    lcb_err = lcb_make_http_request(handle->getLcb(),
                                    this,
                                    LCB_HTTP_TYPE_VIEW,
                                    htcmd,
                                    &htreq);

    if (lcb_err != LCB_SUCCESS) {
        goto GT_ERR;
    }

    lcb_err = lcb_wait(handle->getLcb());
    printf("Wait returned..\n");

GDB:


$45 = {sock = -1275067896, io = 0x7f76b4000208, server = 0x0, instance = 0x4,
  url = 0x7f76ffffffff <Address 0x7f76ffffffff out of bounds>, nurl = 0, url_info = {field_set = 65535,
    port = 65535, field_data = {{off = 0, len = 0}, {off = 19968, len = 476}, {off = 0, len = 0}, {off = 65535,
        len = 65535}, {off = 65535, len = 65535}, {off = 337, len = 0}}}, host = 0x7f76b4000000 " ", port = 0x0,
  path = 0x8000000000 <Address 0x8000000000 out of bounds>, npath = 36, password = 0x7f76b4000b90 "\020",
  method = LCB_HTTP_METHOD_GET, parser = 0x0, parser_settings = {on_message_begin = 0x25,
    on_url = 0x3132312e332e3031, on_header_field = 0x3231313a3830322e, on_header_value = 0x3031,
    on_headers_complete = 0x25, on_body = 0x3132312e332e3031, on_message_complete = 0x323131003830322e},
  root_ai = 0x3031, curr_ai = 0x35, event = 0x312f2f3a70747468, chunked = 775106096,
  on_complete = 0x323930383a383032, on_data = 0x746c75616665642f, output = {root = 0x0,
    read_head = 0x35 <Address 0x35 out of bounds>, write_head = 0x3 <Address 0x3 out of bounds>, size = 7,
    nbytes = 8}, input = {root = 0x7f76b400e0c0 "", read_head = 0x0,
    write_head = 0x55 <Address 0x55 out of bounds>, size = 0, nbytes = 0}, result = {root = 0x0,
    read_head = 0x0, write_head = 0x0, size = 0, nbytes = 0}, command_cookie = 0x0, cancelled = 0,
  completed = 0, headers_list = 0x25, headers = 0x3132312e332e3031, nheaders = 4120855481762722350}
(gdb) frame 4
#4 0x00007f76ce7340a3 in lcb_wait (instance=0x1dd9580) at src/wait.c:112
112 instance->io->v.v0.run_event_loop(instance->io);
(gdb) print *instance
$46 = {type = LCB_TYPE_BUCKET, host = "10.3.121.212\000\070\060\071\061", '\000' <repeats 1008 times>,
  port = "8091", '\000' <repeats 28 times>,
  http_uri = 0x1dd14a0 "GET /pools/default/bucketsStreaming/default HTTP/1.1\r\nAuthorization: Basic QWRtaW5pc3RyYXRvcjoxMjM0NTY=\r\n\r\n", n_http_uri_sent = 107, event = 0x1dd8900, vbucket_config = 0x7f76b4003760,
  vbucket_stream = {
    header = 0x1dd7fb0 "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\nServer: Couchbase Server 2.0.0-1956-rel-enterprise\r\nPragma: no-cache\r\nDate: Sun, 02 Dec 2012 16:58:58 GMT\r\nContent-Type: application/json; charset=utf-8\r\nC"..., input = {
      data = 0x1ddca90 "{\"name\":\"default\",\"bucketType\":\"membase\",\"authType\":\"sasl\",\"saslPassword\":\"\",\"proxyPort\":0,\"replicaIndex\":false,\"uri\":\"/pools/default/buckets/default?bucket_uuid=0535828b5b9f231994f876e3b4299956\",\"str"..., size = 4096, avail = 0}, chunk_size = 18446744073709551615, chunk = {


 Comments   
Comment by Mark Nunberg [ 03/Dec/12 ]
#0 0x00007ffff69ea475 in *__GI_raise (sig=<optimized out>)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff69ed6f0 in *__GI_abort () at abort.c:92
#2 0x00007ffff74e66f6 in lcb_http_request_destroy (req=0x64e4e0) at src/http.c:39
#3 0x00007ffff74eef00 in lcb_server_destroy (server=0x620b10) at src/server.c:403
#4 0x00007ffff74ead8b in lcb_update_serverlist (instance=0x61b2b0)
    at src/instance.c:596
#5 0x00007ffff74ebaea in vbucket_stream_handler (sock=28, which=2, arg=0x61b2b0)
    at src/instance.c:970
#6 0x00007ffff5d7cccc in event_base_loop ()
   from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
#7 0x00007ffff4f60537 in lcb_io_run_event_loop (iops=0x61a570)
    at plugins/io/libevent/plugin-libevent.c:320
#8 0x00007ffff74f269f in lcb_wait (instance=0x61b2b0) at src/wait.c:112
#9 0x00007ffff7bbe5c9 in CBSdkd::ViewExecutor::runSingleView (
    this=0x7ffff475c710, htcmd=0x7ffff475c5a0) at ViewExecutor.cpp:255
#10 0x00007ffff7bbe9c0 in CBSdkd::ViewExecutor::executeView (this=0x7ffff475c710,
    cmd=..., out=..., options=..., req=...) at ViewExecutor.cpp:310
#11 0x00007ffff7bb2c49 in CBSdkd::WorkerDispatch::_process_request (
    this=0x61a0b0, req=..., rs=0x61a480) at IODispatch.cpp:548
#12 0x00007ffff7bb3574 in CBSdkd::WorkerDispatch::run (this=0x61a0b0)
    at IODispatch.cpp:634
#13 0x00007ffff7bb0c4a in CBSdkd::new_worker_thread (worker=0x61a0b0)
    at IODispatch.cpp:216
#14 0x00007ffff7952b50 in start_thread (arg=<optimized out>)
    at pthread_create.c:304
#15 0x00007ffff6a9070d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#16 0x0000000000000000 in ?? ()
(gdb)
Comment by Mark Nunberg [ 03/Dec/12 ]
^^^ Request is destroyed and the event loop is not properly stopped..
Comment by Mark Nunberg [ 03/Dec/12 ]
https://github.com/couchbase/libcouchbase/commit/e37a9fab6d8ae7f556082a339cd8561ec1e93f32




libcouchbase test improvements (CCBC-137)

[CCBC-138] allow libcouchbase tests to be executed against a real cluster Created: 31/Aug/12  Updated: 06/Feb/13  Resolved: 06/Sep/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 31/Aug/12 ]
Tests are now implemented and functional on a real cluster.. some nice dressing needs to happen in order to make it acceptable upstream..
Comment by Mark Nunberg [ 05/Sep/12 ]
This task is mainly done, pending some refactoring and polishing of the proposed gerrit changes
http://review.couchbase.org/#/c/20447/




[CCBC-120] Unhandled opcode=92 Created: 09/Nov/12  Updated: 06/Feb/13  Resolved: 08/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: During an observe + rebalance-in.

Cluster build 1941, lcb 2.0.0-beta2-something (git sha d34403763eec7f91e98fa13e260d47ef64a05945).. php is my own build atm (until i push)

Issue Links:
Dependency
blocks PCBC-143 segfault during lcb handle-level erro... Resolved

 Description   
#0 0x00007ffff6143acf in php_couchbase_error_callback (handle=0x11c0c70, error=LCB_EINTERNAL,
    errinfo=0x7fffffffa2a0 "Unknown implicit send message op=92") at /sources/lcb2.0-repo/php/couchbase.c:724
#1 0x00007ffff5f1aa85 in lcb_error_handler (instance=0x11c0c70, error=LCB_EINTERNAL,
    errinfo=0x7fffffffa2a0 "Unknown implicit send message op=92") at src/error.c:67
#2 0x00007ffff5f28278 in lcb_server_purge_implicit_responses (c=0x15e67d0, seqno=39360, end=1352507659266497000)
    at src/server.c:710
#3 0x00007ffff5f1af11 in parse_single (c=0x15e67d0, stop=1352507659266497000) at src/event.c:135
#4 0x00007ffff5f1b3ca in do_read_data (c=0x15e67d0) at src/event.c:232
#5 0x00007ffff5f1b5a4 in lcb_server_event_handler (sock=14, which=2, arg=0x15e67d0) at src/event.c:298
#6 0x00007ffff5a5dccc in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
#7 0x00007ffff5c95547 in lcb_io_run_event_loop (iops=0x11c0740) at plugins/io/libevent/plugin-libevent.c:320
#8 0x00007ffff5f2aa1c in lcb_wait (instance=0x11c0c70) at src/wait.c:116
#9 0x00007ffff61523f9 in observe_iterate (res=0x11a3c50, ocoll=0x7fffffffa720) at /sources/lcb2.0-repo/php/observe.c:529
#10 0x00007ffff61525b7 in observe_poll (res=0x11a3c50, ocoll=0x7fffffffa720, tprefs=0x7fffffffa700)
    at /sources/lcb2.0-repo/php/observe.c:597
#11 0x00007ffff6153096 in php_couchbase_observe_impl (ht=2, return_value=0x14e2af8, return_value_ptr=0x0, this_ptr=0x11a2f60,
    return_value_used=1, tsrm_ls=0xf53090, multi=1, oo=1, poll=1) at /sources/lcb2.0-repo/php/observe.c:862
#12 0x00007ffff614e089 in zim_couchbase_keyDurabilityMulti (ht=2, return_value=0x14e2af8, return_value_ptr=0x0,
    this_ptr=0x11a2f60, return_value_used=1, tsrm_ls=0xf53090) at /sources/lcb2.0-repo/php/couchbase.c:3468
#13 0x000000000088d5cd in zend_do_fcall_common_helper_SPEC (execute_data=0x7ffff5e96098, tsrm_ls=0xf53090)
    at /sources/php-5.3.17/Zend/zend_vm_execute.h:320
#14 0x000000000088dfe8 in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0x7ffff5e96098, tsrm_ls=0xf53090)
    at /sources/php-5.3.17/Zend/zend_vm_execute.h:425
#15 0x000000000088c588 in execute (op_array=0x11a3ce8, tsrm_ls=0xf53090) at /sources/php-5.3.17/Zend/zend_vm_execute.h:107
#16 0x0000000000851d7a in zend_execute_scripts (type=8, tsrm_ls=0xf53090, retval=0x0, file_count=3)
    at /sources/php-5.3.17/Zend/zend.c:1236
#17 0x00000000007b5106 in php_execute_script (primary_file=0x7fffffffde10, tsrm_ls=0xf53090)
    at /sources/php-5.3.17/main/main.c:2308
#18 0x0000000000951d89 in main (argc=3, argv=0x7fffffffe078) at /sources/php-5.3.17/sapi/cli/php_cli.c:1189


I don't really have a way to reproduce this right now, I just happened to have been in that situation.

 Comments   
Comment by Sergey Avseyev [ 12/Nov/12 ]
92 is observe command, could you show your PHP script?
Comment by Sergey Avseyev [ 13/Nov/12 ]
http://review.couchbase.org/22478
Comment by Sergey Avseyev [ 13/Nov/12 ]
Now it will respond with LCB_SERVER_BUG if found stale OBSERVE packet. It will use command callback instead of generic error callback, therefore you will have context and key name
Comment by Mark Nunberg [ 21/Nov/12 ]
I'll see if I recognize this one again.. this was rather difficult to reproduce and only happened accidentally -- I'm guessing that when and if the sdkd has observe implemented we'd be able to more reliably reproduce it
Comment by Sergey Avseyev [ 21/Nov/12 ]
And at least now, you won't get abort() in real life application, but still can abort in tests




[CCBC-136] abort from libcouchbase_purge_single_server at timeout.c:93 Created: 05/Dec/12  Updated: 06/Feb/13  Resolved: 10/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 1.0.4, 2.0.0
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
#0 0x40022424 in __kernel_vsyscall ()
#1 0x402bc1ef in raise () from /lib/i386-linux-gnu/libc.so.6
#2 0x402bf835 in abort () from /lib/i386-linux-gnu/libc.so.6
#3 0x40be9b7b in libcouchbase_purge_single_server (server=0x42043d58, tmo=2500000000, now=1354688386324058000, error=LIBCOUCHBASE_ETIMEDOUT) at src/server.c:234
#4 0x40bec9bc in libcouchbase_purge_timedout (instance=0x42003af8) at src/timeout.c:93
#5 0x40beca2c in libcouchbase_timeout_handler (sock=-1, which=0, arg=0x42003af8) at src/timeout.c:41
(trimmed...)

 Comments   
Comment by Sergey Avseyev [ 05/Dec/12 ]
fixed for 2.0.0 http://review.couchbase.org/23100




[CCBC-159] Scheduling large list of commands leads to timeouts Created: 27/Dec/12  Updated: 06/Feb/13  Resolved: 02/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 1.0.7, 2.0.0, 2.0.1
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Comments   
Comment by Sergey Avseyev [ 02/Jan/13 ]
The issue was fixed here http://review.couchbase.org/23662
Comment by Sergey Avseyev [ 02/Jan/13 ]
patch breaks the build http://sdkbuilds.couchbase.com/job/libcouchbase/406
Comment by Sergey Avseyev [ 02/Jan/13 ]
Test fixed in http://review.couchbase.org/23668




[CCBC-163] High CPU spikes for operations Created: 03/Jan/13  Updated: 06/Feb/13  Resolved: 03/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
High CPU spikes are observed in the latest HEAD on the 2.0.2 pre-release version. This is due to the LCB_WRITE_EVENT not being disabled

 Comments   
Comment by Mark Nunberg [ 03/Jan/13 ]
http://review.couchbase.org/23692




[CCBC-172] When connection aborted in GetError abort() function is called Created: 25/Jan/13  Updated: 06/Feb/13  Resolved: 06/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Haster Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: All Windows platforms
libcouchbase 2.0.1
Couchbase server 1.8.1

Flagged:
Release Note

 Description   
How to reproduce:
1) Start writing keys to a bucket.
2) Delete bucket from admin panel

As I understand problem is in getError function (plugin-win32.c)
In that case function WSAGetLastError() returns WSAECONNABORTED (10053)
But this value doesn't present in switch-case.
So executes code in default label and abort() function is called...

 Comments   
Comment by Sergey Avseyev [ 06/Feb/13 ]
http://review.couchbase.org/24418




[CCBC-117] Cannot compile libev plugin when switching from v3 to latest 4.11 Created: 07/Nov/12  Updated: 06/Feb/13  Resolved: 04/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta2
Fix Version/s: 2.0.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Michael Leib Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS 6.x, 64-bit


 Description   
I'm trying to track down the cause of some timeout errors that I am now getting under heavy load since the
"network error" return issue has been resolved and I was able to abandon the use of lcb_wait(). I don't know
if this is due to using an old libev OR these are some commands being lost in ringbuffer logic when I pound it
with tons of lcb_get() calls within a loop (before control is returned to the event loop)

So, easiest step it to try a new libev first before I come here crying :>

When I try to compile the plugin with v4.11 (latest from Feb 2012) it will not compile
unless I add the following

#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#define INVALID_SOCKET -1

into either the header file or the top of the code. And, yes, I did add -DHAVE_LIBEV4 to my gcc line

warning: implicit declaration of function memset
warning: implicit declaration of function fcntl
error: INVALID_SOCKET undeclared (first use in this function)
warning: implicit declaration of function free
etc

Looks like the old libev headers sucked in more includes than they do now.

 Comments   
Comment by Michael Leib [ 08/Nov/12 ]
Looking into this further, libev has a config.h which is used by ev.c.

You guys have a config.h as well, which (when loaded) pulls-up config_static.h which has all the goodies
needed to compile the plugin.

#include "config.h" at the top of "plugin-libev.c" maybe should be changed to include the full relative path?



Comment by Michael Leib [ 08/Nov/12 ]
btw, my problem still exists even with the latest libev....depending on load, I am getting "lost" events (ie lcb_get() never returns
with a result until I get timeout error) even if I set the timeout to multiple minutes.

I'm starting to lose faith.
Comment by Michael Leib [ 04/Dec/12 ]
Just an fyi, I just installed the latest available build of the 2.0 server and rebuilt my apps with lcb beta3 and still get random timeout errors with use of lcb_get() even with the timeout set to minutes.
Comment by Michael Leib [ 11/Dec/12 ]
I received the latest patch to the libev plugin code with the changes for timers. It appears my problem has been corrected as I am no longer receiving any timeouts.

Thanks to whomever found and corrected this issue.

I Think this CBC is still an issue, however, as it has to do with builds with libevent and not the timeout problem.
Comment by Sergey Avseyev [ 10/Jan/13 ]
Will check on recent master and will either fix and mark as resolved or just mark
Comment by Sergey Avseyev [ 04/Feb/13 ]
We've released 2.0.0 with libev plugins, therefore it works on our builders, including CentOS 6.2 64bit




[CCBC-151] Cancellation of the HTTP request might lead to segfault or memory leaks Created: 21/Dec/12  Updated: 06/Feb/13  Resolved: 21/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
* request_valid() and lcb_cancel_http_request() must not dereference request pointer until it makes sure that the pointer is valid

* lcb_cancel_http_request() must not remove the request from the hashset, because it will prevent it from destroying

* on_data callback should check if the request has been cancelled and finish it without on_complete notification.

 Comments   
Comment by Sergey Avseyev [ 21/Dec/12 ]
http://review.couchbase.org/23497




[CCBC-167] lcb_wait() waits infinitely Created: 10/Jan/13  Updated: 06/Feb/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates CCBC-147 function lcb_wait works infinitly aft... Closed

 Description   
From http://www.couchbase.com/forums/comment/reply/1002656

> Usually problem happens after creating bucket (I use curl for it).
> After that I try to connect to created bucket... and call lcb_wait function.
> And it executes infinitly...
> I investigate it a little and found, what the problem occures when in Couchbase server bucket loaded some times (in log there is note something like "bucket Name loader (3 times)").
> And why lcb_wait works infinitly???
> I add debug output and found that select function (on which event loop is based) returns 0 (that means that timeout happend) and after that timer is deleted... and than next time select calls, timeout is 0... that means that wait event infinitly, but events doesn't occurs.
> What do you think about that problem?
> P.S. Then I create many buckets in parallel, this bug occurs very often





[CCBC-171] Invalid read in plugin-libevent.c Created: 23/Jan/13  Updated: 06/Feb/13  Resolved: 31/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 1.0.7, 2.0.2
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Major
Reporter: James Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 2h
Time Spent: Not Specified
Original Estimate: 2h
Environment: Centos 5.5, Couchbase server 1.8, libcouchbase 1.02


 Description   
In plugin-libevent.c, event_new() mallocs a "struct event", which is not initialized, then passes it to event_assign(), which passes it into event_base_set(), causing an invalid read. We picked it up during a valgrind run of our program.

See lines 47 - 78 here: https://github.com/couchbase/libcouchbase/blob/master/plugins/io/libevent/plugin-libevent.c

Proposed patch:

Index: libcouchbase/src/plugin-libevent.c
===================================================================
--- libcouchbase/src/plugin-libevent.c (revision 16)
+++ libcouchbase/src/plugin-libevent.c (working copy)
@@ -47,7 +47,6 @@
              event_callback_fn callback,
              void *arg)
 {
- event_base_set(base, ev);
     ev->ev_callback = callback;
     ev->ev_arg = arg;
     ev->ev_fd = fd;
@@ -56,6 +55,7 @@
     ev->ev_flags = EVLIST_INIT;
     ev->ev_ncalls = 0;
     ev->ev_pncalls = NULL;
+ event_base_set(base, ev);

     return 0;
 }


 Comments   
Comment by Sergey Avseyev [ 31/Jan/13 ]
http://review.couchbase.org/24316




[CCBC-175] Network Error on HP-UX "Resource temporarily unavailable" Created: 06/Feb/13  Updated: 06/Feb/13  Resolved: 06/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.2, 2.0.3
Fix Version/s: 2.0.4
Security Level: Public

Type: Bug Priority: Minor
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CCBC-176 libcouchbase does not work on systems... Closed

 Comments   
Comment by Sergey Avseyev [ 06/Feb/13 ]
http://review.couchbase.org/24448




[CCBC-146] configuration.h not copied during installation Created: 13/Dec/12  Updated: 06/Feb/13  Resolved: 13/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Alexey Shevchenko Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: debian 2.6.32-5-amd64


 Comments   
Comment by Sergey Avseyev [ 13/Dec/12 ]
what package are you using to install libcouchbase headers?
Comment by Alexey Shevchenko [ 13/Dec/12 ]
I am use git source from github
Comment by Sergey Avseyev [ 13/Dec/12 ]
there was issue, but introduced after release, please use either deb packages or official tarballs. the bug was fixed in git master also
Comment by Sergey Avseyev [ 13/Dec/12 ]
then just update your copy




[CCBC-130] Memory leak on header responses.. Created: 29/Nov/12  Updated: 06/Feb/13  Resolved: 08/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0beta3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: lcb 2.0.0


 Description   
    memset(&htcmd, 0, sizeof(htcmd));
    OV1(&htcmd).content_type = "application/json";
    OV1(&htcmd).path = vqstr;
    OV1(&htcmd).npath = strlen(vqstr);
    OV1(&htcmd).chunked = 1;
    OV1(&htcmd).method = LCB_HTTP_METHOD_GET;

    rctx->callback = my_vr_callback;
    lcb_set_http_data_callback(instance, http_data_callback);

    err = lcb_make_http_request(instance,
                                rctx,
                                LCB_HTTP_TYPE_VIEW,
                                &htcmd,
                                &htreq);
    assert(err == LCB_SUCCESS);
    printf("Waiting for request..\n");
    err = lcb_wait(instance);
    assert (err == LCB_SUCCESS);

    printf("Got query string: %s\n", vqstr);
    free(vqstr);
    lcb_vopt_cleanup_list(&vopt_list, num_options, 1);
    free(vopt_list);
    lcb_vrow_free(rctx);

/**********************************************************/
mnunberg@csure:~/src/sdkd-cpp/src/views$ valgrind --leak-check=full ./test
==31887== Memcheck, a memory error detector
==31887== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==31887== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==31887== Command: ./test
==31887==
Waiting for request..
Got callback..
Got status 404
Parsing bytes '{"error":"not_found","reason":"Design document _design/design_doc not found"}
'
Got query string: _design/design_doc/_view/view_doc?stale=false&limit=150&debug=true
==31887==
==31887== HEAP SUMMARY:
==31887== in use at exit: 1,915 bytes in 18 blocks
==31887== total heap usage: 3,349 allocs, 3,331 frees, 305,322 bytes allocated
==31887==
==31887== 254 (88 direct, 166 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 10
==31887== at 0x4C272B8: calloc (vg_replace_malloc.c:566)
==31887== by 0x4E375FE: http_parser_headers_complete_cb (http.c:101)
==31887== by 0x4E463ED: http_parser_execute (http_parser.c:1602)
==31887== by 0x4E37D11: request_do_read (http.c:262)
==31887== by 0x4E37F07: request_event_handler (http.c:320)
==31887== by 0x6090CCB: ???
==31887== by 0x5E7F536: ???
==31887== by 0x4E4301E: lcb_wait (wait.c:112)
==31887== by 0x4015A0: schedule_http (main.c:139)
==31887== by 0x401B20: main (main.c:304)
==31887==
==31887== LEAK SUMMARY:
==31887== definitely lost: 88 bytes in 1 blocks
==31887== indirectly lost: 166 bytes in 10 blocks
==31887== possibly lost: 0 bytes in 0 blocks
==31887== still reachable: 1,661 bytes in 7 blocks
==31887== suppressed: 0 bytes in 0 blocks
==31887== Reachable blocks (those to which a pointer was found) are not shown.
==31887== To see them, rerun with: --leak-check=full --show-reachable=yes
==31887==
==31887== For counts of detected and suppressed errors, rerun with: -v
==31887== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 10 from 6)


==6220== HEAP SUMMARY:
==6220== in use at exit: 1,923 bytes in 18 blocks
==6220== total heap usage: 3,349 allocs, 3,331 frees, 427,889 bytes allocated
==6220==
==6220== 262 (88 direct, 174 indirect) bytes in 1 blocks are definitely lost in loss record 8 of 10
==6220== at 0x4C272B8: calloc (vg_replace_malloc.c:566)
==6220== by 0x4E375FE: http_parser_headers_complete_cb (http.c:101)
==6220== by 0x4E4644D: http_parser_execute (http_parser.c:1602)
==6220== by 0x4E37D11: request_do_read (http.c:262)
==6220== by 0x4E37F07: request_event_handler (http.c:320)
==6220== by 0x6090CCB: ???
==6220== by 0x5E7F536: ???
==6220== by 0x4E4307E: lcb_wait (wait.c:112)
==6220== by 0x40158B: schedule_http (main.c:140)
==6220== by 0x401AF5: main (main.c:305)
==6220==
==6220== LEAK SUMMARY:
==6220== definitely lost: 88 bytes in 1 blocks
==6220== indirectly lost: 174 bytes in 10 blocks
==6220== possibly lost: 0 bytes in 0 blocks
==6220== still reachable: 1,661 bytes in 7 blocks
==6220== suppressed: 0 bytes in 0 blocks
==6220== Reachable blocks (those to which a pointer was found) are not shown.
==6220== To see them, rerun with: --leak-check=full --show-reachable=yes
==6220==
==6220== For counts of detected and suppressed errors, rerun with: -v
==6220== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 10 from 6)



 Comments   
Comment by Mark Nunberg [ 30/Nov/12 ]
Yes, apparently this just leaks. It seems to happen on errors, chunked responses, and non-chunked responses..
Comment by Mark Nunberg [ 30/Nov/12 ]
http://review.couchbase.org/#/c/22922/




[CCBC-158] Unable to store 1M objects in a loop ( Internal error (0x6), Unknown implicit send message op=1 , eventually crash ) Created: 02/Jan/13  Updated: 06/Feb/13  Resolved: 07/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Jeroen van Bemmel Assignee: Trond Norbye
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 2 VMs running Couchbase 2.0 enterprise, with 2 buckets of 256MB each. libcouchbase from git ( latest pull 1/1/2013 ). Running own event loop calling event_base_loop in a separate thread


 Description   
When executing 1M lcb_store calls in a loop, storing items like "user0000001 = <16 hex bytes>" in the default bucket, I get a number of error callbacks saying "Internal error (0x6), Unknown implicit send message op=1" ( opcode 0x1 = PROTOCOL_BINARY_CMD_SET ). Finally, after a last "Internal error (0x6), Unknown implicit send message op=46" the process gets killed
.
Unfortunately I cannot tell where the crash happens, as running with gdb results in:
Couchbase event loop starting/build/buildd/gdb-7.4-2012.04/gdb/infrun.c:3189: internal-error: handle_inferior_event: Assertion `inf' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.

After a number of attempts in both debug and release mode, I ended up with almost 350K out of 1M items entered. However, I don't manage to run the loop to its completion; something always breaks before it completes



 Comments   
Comment by Jeroen van Bemmel [ 02/Jan/13 ]
Using gdb 7.5.1 I was able to generate a stacktrace:
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff7fe4700 (LWP 32200)]
0x00007ffff635a425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) backtrace
#0 0x00007ffff635a425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff635db8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff7bc2501 in ringbuffer_consumed (buffer=0x7d0048, nb=180) at src/ringbuffer.c:263
#3 0x00007ffff7bb73af in do_send_data (c=0x7d0000) at src/event.c:270
#4 0x00007ffff7bb74af in lcb_server_event_handler (sock=10, which=4, arg=0x7d0000) at src/event.c:308
#5 0x00007ffff777af64 in event_process_active_single_queue (activeq=0x79c010, base=0x7a0000) at event.c:1350
#6 event_process_active (base=<optimized out>) at event.c:1420
#7 event_base_loop (base=0x7a0000, flags=0) at event.c:1621
#8 0x000000000040cf12 in RegistrationDb::authDbEventThread(void*) ()
#9 0x00007ffff734de9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#10 0x00007ffff6417cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6

The abort at src/ringbuffer.c:263 is triggered:
void ringbuffer_consumed(ringbuffer_t *buffer, lcb_size_t nb)
{
    lcb_size_t n = ringbuffer_read(buffer, NULL, nb);
    if (n != nb) {
        abort();
    }
}

 
Comment by Jeroen van Bemmel [ 02/Jan/13 ]
During a different run, Valgrind reveals the following:
Couchbase config callback: 1==32436== Invalid write of size 1
==32436== at 0x4C2D1A5: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32436== by 0x4E41034: ringbuffer_write (ringbuffer.c:127)
==32436== by 0x4E405E5: lcb_server_buffer_start_packet (packet.c:46)
==32436== by 0x4E408F7: lcb_server_start_packet (packet.c:138)
==32436== by 0x4E4491E: lcb_store (store.c:127)
==32436== by 0x40CC92: RegistrationDb::populate(util::string const&, util::string const&, util::string const&) (RegistrationDb.cpp:209)
==32436== by 0x405BEE: Registrar::Registrar(char const*, unsigned int, char const*, char*) (HARegistrar.cpp:65)
==32436== by 0x40414F: main (StatefulHAProxy.cpp:238)
==32436== Address 0x1e is not stack'd, malloc'd or (recently) free'd
==32436==
Comment by Jeroen van Bemmel [ 02/Jan/13 ]
I should add that my program is linked with gperftools-2.0 ( malloc replacement )
Comment by Jeroen van Bemmel [ 02/Jan/13 ]
When I remove the separate thread for the event loop and call lcb_wait after each lcb_store, the system does not crash anymore, so I guess libcouchbase not being multi-thread safe also applies to the event handler loop. However, even when using only a single thread, I still don't reach 1M objects
Comment by Matt Ingenthron [ 02/Jan/13 ]
What happens in this single threaded case that prevents reaching 1M objects, if not a crash?
Comment by Trond Norbye [ 02/Jan/13 ]
Can you post the code somewhere (or email it to trond.norbye@couchbase.com) and I'll take a look at it. I've used libcouchbase in a multithreaded environment earlier so I is supported to be MT-safe)

CHeers,

Trond
Comment by Jeroen van Bemmel [ 02/Jan/13 ]
@Matt I got temporary failures in the store callback ( code 0xB IIRC ). I suppose I could change my code to retry, but that shouldn't be necessary with only a single client ( and isn't so straight forward from the callback handler, which was getting called by a separate thread )

@Trond the code is integrated with my own application, bit difficult to extract. You can easily reproduce it by taking your own "eventloop" example, changing the code to call the eventloop from a new pthread and adding a for loop with 1M times store( "user0xxxxx", "0123456789012345" ) after starting the thread ( from main() )
Comment by Trond Norbye [ 06/Jan/13 ]
Jeroen: I made an MT-version of the example program and uploaded it to http://review.couchbase.org/#/c/23743/ I ran that with 10 threads and 1 000 000 set operations per thread in two modes: one time where I batch up everything before I start sending it to the server, and a second time where the return of operation triggers the next operation. I was not able to make that crash...

Comment by Jeroen van Bemmel [ 06/Jan/13 ]
Hi Trond,

Thanks for following up. After briefly looking through your code, I think the difference is that in my case the lcb_store calls are made from a different thread than the one running the event loop. I can see how one can chain calls into libcouchbase by calling from a callback function, however the question is how one would do this in a regular program.

Suppose you have a thread listening on a socket, and doing a database lookup based on the contents of a packet it receives. How would use libcouchbase in such a case? Is there another way besides running the event loop after performing a lcb_get call, essentially making the code blocking?
Comment by Trond Norbye [ 07/Jan/13 ]
You need to synchronize the access to libcouchbase (and libevent). None of them use locks internally, but operates on their own "instance".

I'm not sure I understand your problem when you say in "regular program". If you hook libcouchbase into the same event loop you're currently doing your database requests in everything should work just as you want?? In the "callback" you have when you receive the result from your database you can simply schedule a libcouchbase operation. When you return to the event loop the event loop will continue to drive the libcouchbase operations as well? Or isn't your application using an event loop?

Part of the problem is that libcouchbase does more than simply ship a command to a server. It monitor the cluster to react to changes in the topology, send and receive data to the individual data nodes etc. If it wasn't for this you could in theory call lcb_flush_buffers() to have it send data and process all of the available input. It would however not detect changes in the cluster.

If you can't move your database code into an event framework (the same you use for libcouchbase), you could always create a dedicated libcouchbase thread running the event loop and just synchronize the communication channel between your "database" thread and the libcouchbase thread... (or even simpler you could have a threadpool of libcouchbase instances running in synchronized mode working off a single queue of operations). Anyway, it's hard to give any recommendations/suggestions when I don't understand your problem ;)
Comment by Jeroen van Bemmel [ 07/Jan/13 ]
I guess my problem is that my socket code does not use libevent or some other event framework. This makes it hard to use libcouchbase, because it implies a need to synchronize between threads as you point out. This in turn kind of defeats the purpose of using an event-based database client framework for performance reasons
Comment by Trond Norbye [ 07/Jan/13 ]
Yes, you would be better off to fix your socket code to be non-blocking and asynchronous so you could fit everything into an event loop...
Comment by Matt Ingenthron [ 07/Jan/13 ]
What is your code using? Something like /dev/poll on Solaris or epoll on linux or kqueue on BSD?

If so, you can probably continue to use that and then configure the event lib to use the same thing underneath.
Comment by Mark Nunberg [ 07/Jan/13 ]
I would second the proposition of having a 'libcouchbase thread' that handles command and communicates using some kind of queue. The problem you are having seems to be socket buffer synchronization.

Specifically whenever you schedule an lcb command (i.e. "lcb_store") it writes a packet to our own socket buffer. Considering libcouchbase is asynchronous, it will flush that buffer at some unknown time to the network (thereby modifying the socket buffer).

Therefore even if you ensure that you lock your handle with things like pthread_mutex_lock and such, you will still encounter issues unless you can also control when the socket buffer is actually flushed (i.e. when 'lcb_wait' is called).

As such, you'd probably need a libcouchbase "server" thread.

One possible implementation is having this thread run an event framework (e.g. libev) which will drive libcouchbase. In addition to the thread running libcouchbase asynchronously, it will also accept command proxies over a queue (i.e. proxied requests for "lcb_store", "lcb_get", etc.).

You may have this thread then schedule (within its own thread context) the command to libcouchbase. The thread will know about when there are commands available to shcedule by determining if there are items in the queue. This may be done in a non-blocking fashion by potentially constructing a 'socketpair', in which a byte of dummy data is written each time something si added to the queue.

Subsequently, the "libcouchbase thread" will receive callbacks. You can proxy these callbacks out to your own user-defined callbacks.

While this sounds complicated, I am guessing the whole thing (assuming you actually have a "Queue" implemented [ which being C++, you have several native structures to fit this model ]) shouldn't take up more than 300 lines of code.
Comment by Jeroen van Bemmel [ 07/Jan/13 ]
That would be possible, sure. For now I've decided to move away from using CouchBase/libcouchbase though, I'm only following up because I started this issue and to provide feedback to you guys.

If you want to make it easier for people to use libcouchbase, from my perspective you could
a) Document a bit better / more extensively what the implications are for the application. Basically the information from this thread would be good to have known upfront
b) Implement something like you describe above

I believe it could work well, the challenge is that the learning/adoption curve can be steep
Comment by Trond Norbye [ 07/Jan/13 ]
Earlier today I pushed through http://review.couchbase.org/#/c/23749/ giving you manual pages for "everything"... if you check out the lcb_attributes (looks better when you see it through man: http://review.couchbase.org/#/c/23749/3/man/svr4/man5/lcb_attributes.5 ) it'll describe how the MT thing fits together..

Comment by Mark Nunberg [ 07/Jan/13 ]
Looking at that manpage, the locking needs to be primarily focused on the state of the ringbuffers, meaning that two threads cannot be modifying the ringbuffer at any given time.

Locking simply before scheduling operations is not sufficient because at the same time, the event loop might have woken up and the lcb_t would have started writing/reading data to/from the socket. A model similar to the one I outlined would work well - but that is pretty much the only way to get true async operations from libcouchbase from multiple threads. A different option would be to lock it whenever calling lcb_wait() but this can result in a deadlock if someone tries to use lcb_store from within a callback itself.

Perhaps some stuff in this thread (i.e. jira thread) should be broken out into its own discussion about MT-safety and libcouchbase - the specific bits about maintaining ringbuffer/command log integrity are not immediately obvious; neither are its solutions




[CCBC-166] make distcheck may file depending on how the dist is built Created: 07/Jan/13  Updated: 06/Feb/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.3
Fix Version/s: 2.0.3
Security Level: Public

Type: Bug Priority: Major
Reporter: Trond Norbye Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The problem is how we're building the dist file. Currently we include all of the .txt files into the bundle and only a subset of the generated files (man pages). Instead we should list the files we want to include in the dist file, and that should be all of the _GENERATED_ files.



 Comments   
Comment by Trond Norbye [ 07/Jan/13 ]
http://review.couchbase.org/#/c/23749/




[CCBC-176] libcouchbase does not work on systems with EWOULDBLOCK != EAGAIN Created: 06/Feb/13  Updated: 06/Feb/13  Resolved: 06/Feb/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Trond Norbye Assignee: Sergey Avseyev
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: HP-UX

Issue Links:
Duplicate
duplicates CCBC-175 Network Error on HP-UX "Resource temp... Closed

 Description   
Some systems define EWOULDBLOCK, but have a different value for EAGAIN and returns EAGAIN instead of EWOULDBLOCK in some cases.

We found this "the hard way" on HP-UX where libcouchbase wouldn't work at all.




[CCBC-164] Concurrent access to continuum can cause invalid memory reference when servers are removed Created: 29/Dec/12  Updated: 06/Feb/13  Resolved: 29/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Jeroen van Bemmel Assignee: Trond Norbye
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In libvbucket/vbucket.c lines 121/122, function update_ketama_continuum first updates the array pointer, and then the counter.

In lines 615/616, function vbucket_map reads both the array pointer and the counter to determine the boundaries of the memory to scan.

In the (rare) event that a server is removed and thread A executes update_ketama_continuum line 121 with a smaller array and then blocks, a second thread B could execute vbucket_map, read the new array pointer and the (larger) size of the old array, and access memory that has been freed.

A similar issue exists in (some versions of) libmemcached code. To fix this, one could change the code to use a pointer to a struct which contains both the array and its size, or place both the array pointer and the size adjacent in memory and use an atomic operation to update both simultaneously ( less portable )

 Comments   
Comment by Trond Norbye [ 29/Jan/13 ]
This is not a bug in libcouchbase because each lcb_t should not be accessed from multiple threads without adding proper locking.




[CCBC-150] Observe (and possibly other) callbacks receiving NULL callback more than once when batched Created: 21/Dec/12  Updated: 06/Feb/13  Resolved: 29/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Mark Nunberg Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks CCBC-145 Provide API for durability operations In Progress

 Description   
See here http://paste.scsys.co.uk/219902 for an example.

This primarily affects observe operation in 'multi' mode; where multiple observe requests are queued up. This fails when the request distribution is asymmetric.

This also affects stats and version commands if coupled with other commands in an asynchronous mode.

This will definitely affect node.js.

In summary, the function lcb_lookup_server_with_packet tries to find 'associated' packets; but it does it wrongly. It assumes associated packets will always be at the tail (or pop-end) of the command log.

However this will only be the case in a purely synchronous mode where there are no subsequent commands. The following scenario demonstrates this rather well:

With three servers and two keys (and one replica), you have something like this for the vb/server map

"K1" = servers[1], servers[2]
"K2" = servers[0], servers[1]

When the commands are sent out, the command look for the server looks something like:


-- order in the buffer
BUFFERS:
[ OLD .. NEW]

0: "K2",
1: "K1", "K2"
2: "K1",

LOGICAL PRECEDENCE:
[ OLD .. NEW ]

"K1", "K2"

The idea to keep in mind is that even though we've scheduled the request for "K1" first, we may very well receive the response for "K2" first - this is especially true if server[0] is quicker than the other servers.

So what happens when "K2" is received first? - the code checks to see the older commands on all the other servers (only the oldest command), and if not, it assumes this is the last command to be received and notifies the user that the command has finished.

So now the buffers look like:

0: "K3"
1: "K1", "K2"
2: "K1"

Now, server[1] responds with "K1". Since the request for "K1" is still at the pop-end of server[2] nothing special happens here

The buffers look like:

0:
1: "K2"
2: "K1"

Then server[1] responds again, this time with the next response for "K2". It delivers the callback again. In a typical use case a user maintains a cookie tied to the command, and when the command is done, the cookie is freed. Plainly speaking, this double-singalling (i.e. telling the user a command has finished twice) segfaults the app.

 Comments   
Comment by Mark Nunberg [ 21/Dec/12 ]
I actually have a fix in mind, which is to make a refcounted cookie for each observe operation - this will bind all of them together and allow us to simply invoke the callback when the refcount is 0.

I've implemented this and it seems to be working..
Comment by Trond Norbye [ 27/Dec/12 ]
I think I would prefer a solution that works for all of these commands and not just one of them..
Comment by Matt Ingenthron [ 27/Dec/12 ]
Agreed with Trond, but not necessarily said here is that Observe is somewhat special.
Comment by Mark Nunberg [ 27/Dec/12 ]
After a re-analysis of the commands, I'm not so sure these things exist in multiple version or stats commands, because those commands are sent uniformly to all servers and is not asymmetric like observe. I stress 'possibly others' as I'm still not sure

In any event, it's perfectly possible to abstract/extrapolate this refcount approach to other commands in a later commit.
Comment by Trond Norbye [ 30/Dec/12 ]
I think we would be better off by just fixing the real problem here: the way lcb_lookup_server_with_command.. instead of expecting the response to match the first response we're awaiting for each socket, we should rather just check if it is in the "pipeline" for each server. Given that the opaque field is a command counter we know how deep we have to search for it..
Comment by Trond Norbye [ 30/Dec/12 ]
http://review.couchbase.org/#/c/23633/
Comment by Mark Nunberg [ 30/Dec/12 ]
The real problem is that implicitly relying on things existing within the ringbuffer is difficult to follow and error prone:

(1) In order to find 'associated' commands one needs to actually search through the command log of the server
(2) The command in the ringbuffer must not have already been removed somehow (the command is only removed during a server failure or when a response has been received, but this is a fairly contrived step).
(3) Simply searching through the server buffer is not enough. There is an explicit "are we done yet" step which needs to be performed every time we respond to a command.

This can also have negative performance hits in cases where lots of commands are scheduled (though in the single-command use case, this is normally not an issue)..

The fix for this needs to be coupled by a general 'invoke_callback' wrapper which will invoke the specific callback as well as check that 'neighbor' packets do not remain, and then invoke with the NULL parameter.
Comment by Trond Norbye [ 31/Dec/12 ]
I am not sure I follow the problem here... You would only have to search until you found the first entry for a command in a command buffer. You may have a large batch of commands in the pipeline, but I am failing to see the usecase where that would happen (given that that the normal code-path would be get/sets and those are sent to a single server). The commands that are distributed to multiple servers would be the observe, stats, version, verbosity commands, and I don't see where I would put them into big batches of other commands...

I would assume the logic for where I wanted to use observe would be together with the result of a store operation, and my application would normally "wait" for it to complete.. Yes, it could be used in a singlethreaded server where we have a ton of other logical threads using the same instance and hence end up with a huge batch of pending commands, but I would find that more of a corner case than the normal code path...
Comment by Trond Norbye [ 31/Dec/12 ]
ops.. pressed resolve before code was merged...
Comment by Mark Nunberg [ 31/Dec/12 ]
The issue of performance is indeed a corner case (as noted in my initial comment) - however the other two concerns - namely the tip-toeing needed and the whole lack of clarity when reading the code - and understanding how the commands are actually tied together, leads to a point of significant confusion (confusing enough that we're running into these issues now, which would have likely been avoided had the association been explicit). It's my opinion that there are too many implicit assumptions with little performance gain; I think we can agree that our performance (read: throughput) issues with observe will not be CPU-bound but rather I/O bound.

Anyway my point isn't so much about performance but rather about clarity in respect to what happens in such a central code path. In terms of performance for the normal case, looking in the command log might be "faster" because there's no need to allocate an extra chunk of memory to contain the refcount (assuming that there is only a handful of commands in the pipeline); however I think it's safe to say that observe/version/stats aren't going to be commands bound by client-side CPU or memory.

We'd also need to handle all these commands in the case of server failures (see patch/issue for CCBC-155) - which wouldn't be impossible; we would need to pass these functions the opaque value for the request/response.

Edit: I see that VERSION already adds the boilerplate there. In any event, OBSERVE is still special - and in the future should be split out into three commands (on the client) - durability (what I'm working on right now), broadcast (our current lcb_observe implementation) and check (which checks the master to see if the key exists without actually fetching its contents)
Comment by Trond Norbye [ 31/Dec/12 ]
these commands should go in libcouchbase_util or something... It woud be nice with a discussion about such commands prior to implementations....
Comment by Trond Norbye [ 29/Jan/13 ]
The fix is already merged...




[CCBC-73] Command line parser from cbc tool leaks Created: 11/Jun/12  Updated: 06/Feb/13  Resolved: 29/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Trond Norbye
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: libcouchbase 1.1.0dp5-10-g474ec9e


 Description   
See execution log here: https://gist.github.com/2909622

 Comments   
Comment by Trond Norbye [ 29/Jan/13 ]
cbc is a command line tool, and the "leaks" here will be cleaned up when the process exit anyway (its not worth trying to figure out why it behaves that way)




[CCBC-143] cbc version reports that uses 2.0.0, but really installed with 2.0.1 Created: 10/Dec/12  Updated: 06/Feb/13  Resolved: 10/Dec/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.1
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
$ cbc version
cbc built from: libcouchbase 2.0.1
    using libcouchbase: 2.0.0


 Comments   
Comment by Sergey Avseyev [ 10/Dec/12 ]
Not really a bug. Just mistype in configure.ac

This version isn't used by dynamic linker or tools like that. Fixed in http://review.couchbase.org/23183




[CCBC-148] intermittent test failures in MockUnitTest.testPurgedBody Created: 19/Dec/12  Updated: 06/Feb/13  Resolved: 05/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.1
Fix Version/s: 2.0.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, we see intermittent test failures all in the same test from sdk-repo. This has been at least since build #866, and probably earlier.

http://sdkbuilds.couchbase.com/job/sdk-repo/


[----------] Global test environment tear-down
[==========] 80 tests from 17 test cases ran. (49277 ms total)
[ PASSED ] 79 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] MockUnitTest.testPurgedBody

 Comments   
Comment by Mark Nunberg [ 19/Dec/12 ]
The test assumes that the mock server is timely enough, so that the command being sent to the mock is processed within 0.25 seconds.

The test might fail because of this.

There's no easy fix for this, other than to make the mock send ACKs to commands, so that we know when the mock has completed the operation.

https://github.com/couchbase/libcouchbase/blob/master/tests/mock-unit-test.cc#L568




[CCBC-168] Query on multiset and multiget libcouchbase Created: 11/Jan/13  Updated: 06/Feb/13  Resolved: 11/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Muthu Kumar Assignee: Sergey Avseyev
Resolution: Won't Fix Votes: 0
Labels: 2.0.1, customer, libcouchbase, multiget, multiset, ns_server, operations, writes
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
As per a query from the customer on multiset and multiget regarding libcouchbase 2.0.1 client

"When you calculate number of operation, do you consider the set of keys in Multiset ? Or is it just one operation?
I am trying to calculate the writes per second. The client would send 15k writes per second, each of them is a multiset with 20 keys. So is this 15k writes per second ?
Or 15*20= 300k writes per second ? "

Please help answer this.

 Comments   
Comment by Sergey Avseyev [ 11/Jan/13 ]
There no such operation multi-set or multi-get in memcached protocol. Libcouchbase doesn't count operations at all. If you are talking about stats in the Admin UI, then the answer probably will be 1 operation = 1 key, in this case 15k*20 = 300k. I recommend to move this ticket to another tracker.




[CCBC-174] Error handling documentation Created: 05/Feb/13  Updated: 05/Feb/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.2
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Perry Krug Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Please create some documentation specifying possible error/failures to operations, what they "look" like in the logs/exceptions/stack traces and what our recommendation is on how to handle them.

i.e. tmp_oom, timeouts (connection/operation/java-internal/etc), "get miss" (it's technically a failure, let's make it overly obvious what it means), CAS failure, add() failure, replace() failure,

Some of this should be covered in the API reference, but this bug is specifically for a single page where this information is aggregated that a customer/user could read about how to handle errors.




[CCBC-162] API reference uses old names, not current lcb_* names Created: 03/Jan/13  Updated: 31/Jan/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Tim Smith Assignee: MC Brown
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-c-2.0/api-reference-summary.html


 Description   
The API reference for all functions uses libcouchbase_* names (libcouchbase_create, etc.) instead of the correct lcb_* names (lcb_create, etc.).

 Comments   
Comment by Trond Norbye [ 29/Jan/13 ]
The manual pages for libcouchbase use the correct names :)
Comment by Karen Zeller [ 29/Jan/13 ]
There is something wrong. At this page, I get: Sorry. We can’t seem to find that page. Can you let me know where this actually appears (name guide, section, page, etc)
Comment by Tim Smith [ 30/Jan/13 ]
There's something even bigger wrong, because the page is missing. There's no reference material at all now in the C client library docs, the TOC has:

1. Getting Started
1.1. Downloading the Couchbase Client Library
1.2. Installing from Source: Linux and Mac OS X
1.3. Installing from Source: Microsoft Windows
1.4. Installing on CentOS/ReHat
1.5. Installing on Ubuntu
1.6. Hello C Couchbase
2. Tutorial
2.1. Simple Example
2.2. Additional Resources
A. Release Notes

The reference page that lists all of the functions, etc., has been removed entirely. Is it supposed to be auto-generated, and perhaps that process is failing? I don't know.
Comment by Karen Zeller [ 30/Jan/13 ]
I'm going to ask MC to look at this as it appears to be the production system. I noticed a couple pages that had correctly linked as cross references in the past are also 404 missing...
Comment by MC Brown [ 31/Jan/13 ]
Matt and I agreed to temporarily remove the API reference from the documentation because we know it needs significant updating. The source material for the API is based on the document that Trond produced last year, and which I have no ability to change or fix at the moment, but I'll work to get that updated.




[CCBC-134] Repo refuses to download packages: "Package does not match intended download" Created: 04/Dec/12  Updated: 10/Jan/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Jeff Minard Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS 5.8


 Description   
Doing this:

wget -O/etc/yum.repos.d/couchbase.repo http://packages.couchbase.com/rpm/couchbase-centos55-x86_64.repo
yum clean all
yum install -y libcouchbase2.x86_64 libcouchbase-devel

Results in a failed install/update with these errors:

[root@host ~]# yum install -y libcouchbase2.x86_64 libcouchbase-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.sonic.net
 * centosplus: mirror.hmc.edu
 * epel: mirrors.solfo.com
 * extras: mirror.stanford.edu
 * updates: mirrors.arpnetworks.com
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libcouchbase-devel.x86_64 0:2.0.0-1 set to be updated
--> Processing Dependency: libcouchbase2-core for package: libcouchbase-devel
--> Processing Dependency: libcouchbase.so.2()(64bit) for package: libcouchbase-devel
---> Package libcouchbase2.x86_64 0:2.0.0-1 set to be updated
--> Processing Dependency: libcouchbase2-backend for package: libcouchbase2
--> Running transaction check
---> Package libcouchbase2-core.x86_64 0:2.0.0-1 set to be updated
---> Package libcouchbase2-libev.x86_64 0:2.0.0-1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================
 Package Arch Version Repository Size
=======================================================================
Installing:
 libcouchbase-devel x86_64 2.0.0-1 couchbase 52 k
 libcouchbase2 x86_64 2.0.0-1 couchbase 26 k
Installing for dependencies:
 libcouchbase2-core x86_64 2.0.0-1 couchbase 82 k
 libcouchbase2-libev x86_64 2.0.0-1 couchbase 31 k

Transaction Summary
=======================================================================
Install 4 Package(s)
Upgrade 0 Package(s)

Total download size: 191 k
Downloading Packages:
(1/4): libcouchbase2-2.0.0-1.x86_64.rpm | 26 kB 00:00
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase2-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.
(2/4): libcouchbase2-libev-2.0.0-1.x86_64.rpm | 31 kB 00:00
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase2-libev-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.
(3/4): libcouchbase-devel-2.0.0-1.x86_64.rpm | 52 kB 00:01
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase-devel-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.
(4/4): libcouchbase2-core-2.0.0-1.x86_64.rpm | 82 kB 00:00
http://packages.couchbase.com/rpm/5.5/x86_64/libcouchbase2-core-2.0.0-1.x86_64.rpm: [Errno -1] Package does not match intended download
Trying other mirror.


Error Downloading Packages:
  libcouchbase2-2.0.0-1.x86_64: failure: libcouchbase2-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.
  libcouchbase2-core-2.0.0-1.x86_64: failure: libcouchbase2-core-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.
  libcouchbase-devel-2.0.0-1.x86_64: failure: libcouchbase-devel-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.
  libcouchbase2-libev-2.0.0-1.x86_64: failure: libcouchbase2-libev-2.0.0-1.x86_64.rpm from couchbase: [Errno 256] No more mirrors to try.

 Comments   
Comment by Sergey Avseyev [ 04/Dec/12 ]
Could you run yum check-update, just to ensure everything in sync?
Comment by Jeff Minard [ 04/Dec/12 ]
Yeah, doing

# yum clean all
# yum check-update

Shows a few out of data packages (httpd, iptables, kernel, etc) just general "haven't updated my dev machine in a while" stuff. Install command still fails the same.
Comment by Jeff Minard [ 04/Dec/12 ]
Possibly bad meta data? http://forums.famillecollet.com/viewtopic.php?id=402
Comment by Jeff Minard [ 04/Dec/12 ]
...aaaand now it's working. Maybe the repo files were slow to update? Or perhaps you fixed it? Either way, it installed now after another round of clean && check-update




[CCBC-10] set up continuious integration on linux/MacOS X for libcouchbase Created: 01/Sep/11  Updated: 10/Jan/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Task Priority: Major
Reporter: Matt Ingenthron Assignee: Ray Chin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
libcouchbase needs to be set up under some kind of CI system to regularly be built whenever changes are made.

 Comments   
Comment by Matt Ingenthron [ 01/Sep/11 ]
assigning to QA management to get the task scheduled
Comment by Trond Norbye [ 01/Sep/11 ]
(please note that libcouchbase is part of our 2.0 build, so it will be built as part of that if you set up a build slave for that.)
Comment by Sergey Avseyev [ 07/Jun/12 ]
It is done partially by http://sdkbuilds.couchbase.com/job/libcouchbase/

But we need builder slave for mac
Comment by Farshid Ghods [ 26/Jul/12 ]
would 10.6.x work ?
Comment by Matt Ingenthron [ 26/Jul/12 ]
Yes
Comment by Farshid Ghods [ 26/Jul/12 ]
Ray,

can we clone 10.6 VM and add it as slave to the sdk jenkijns
Comment by Ray Chin [ 26/Jul/12 ]
what is the ram/core requirements for this?
Comment by Matt Ingenthron [ 25/Sep/12 ]
No special requirements, just 256MByte memory, but 512MByte memory would be fine.




[CCBC-122] Handle timeout for connecting to a cluster Created: 12/Nov/12  Updated: 10/Jan/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0
Fix Version/s: 2.0.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Trond Norbye Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CCBC-123 Client does not skip over unreachable... Closed

 Description   
If we've got a list of servers to connect to we don't try the next server in the list if we're getting a timeout connecting to a host in that list.. We should probably try the next server in the list.. (alternatively we could try all of them in parallel ;))

 Comments   
Comment by Trond Norbye [ 12/Nov/12 ]
See also http://www.couchbase.com/issues/browse/PCBC-145
Comment by Sergey Avseyev [ 12/Nov/12 ]
Fix http://review.couchbase.org/22449




[CCBC-29] Automated generation of Windows artifacts Created: 27/Oct/11  Updated: 10/Jan/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Sergey Avseyev Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: package
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
generated under CI

 Comments   
Comment by Sergey Avseyev [ 10/Jan/13 ]
The artifacts are generating for now. ZIP archive which contain all libs, docs and binaries




[CCBC-119] Feed a different string (or hash?) for vBucket attribution Created: 08/Nov/12  Updated: 10/Jan/13  Resolved: 10/Jan/13

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta2
Fix Version/s: 2.0.0
Security Level: Public

Type: Improvement Priority: Major
Reporter: Marc Trudel Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We currently are in a situation where we would like to use a different string than the key for hashing and selection of the destination vBucket. This can be useful to keep all of a single user data on many keys but on the same vBucket (and therefore servers), and allow for node downtime to only affect a percentage of the user base instead of the full user base. More generally, it could be used to gain better control over the distribution of data over the nodes.

Looking at the current code, I guess it would be safer not to touch the hashing but just allow for the hashing of a separate string?

 Comments   
Comment by Trond Norbye [ 08/Nov/12 ]
To facilitate this we need to create an extended version of our arguments (the lcb_get_cmd_t etc) that includes a "blob" to use for hashing instead of the key.

We need to verify that the code in libcouchbase that handles "not-my-vbuckets" doesn't rehash the key, but just use the updated servermap to locate the correct server.
Comment by Sergey Avseyev [ 08/Nov/12 ]
I thought we removed *_by_key() functions for reason here http://review.couchbase.org/20073

See Matt's comment here http://review.couchbase.org/14956

> At this point in time, I do not think we want to include this. It is
> accurate that python exposes this to the user, but that's more out
> of python's legacy as testing code than a decision to add that to
> API.
>
> vbuckets are generally to be abstracted away from the user's app.
> let's not leak the abstraction out any more than we need to.
Comment by Matt Ingenthron [ 08/Nov/12 ]
Given that there is a user requirement for it, perhaps we should re-add it. Let me check into it.

The concern in the past has been that this will likely cause unbalanced clusters and can create confusion on how one can get to the data from one client that isn't available from another client.
Comment by Mark Nunberg [ 08/Nov/12 ]
Would it not be simpler from our perspective to simply provide a 'hash' field in the request structure?

Much less boilerplate in our code - change would simply be to use an existing hash if it's already there, and then we can do the normal vbucket mapping If it's 0, then calculate it.

This is actually quite familiar with APIs that do hashing.

Maybe expose an additional lcb_hash_key function.
Comment by Trond Norbye [ 09/Nov/12 ]
http://review.couchbase.org/#/c/22395/

Mark: unfortunately that would require a bigger change in the current code (we would have to supply a new API from libvbucket). Given that we haven't released any version of libcouchbase 2.0, I decided that we can just put this into the basic struct..




[CCBC-144] Standardize view option handling Created: 12/Dec/12  Updated: 10/Jan/13

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.1.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Centralize view option conversion, validation, and construction so dependent code does not have to repeat the same boilerplate all the time. This makes it easier for client libraries to adapt to new view options, as well as provide common expected behavior with current options.

 Comments   
Comment by Mark Nunberg [ 13/Dec/12 ]
http://review.couchbase.org/#/c/23249/




[CCBC-145] Provide API for durability operations Created: 13/Dec/12  Updated: 21/Dec/12

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.1.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on CCBC-150 Observe (and possibly other) callback... Closed

 Description   
This feature will provide a callback-oriented API for durability (aka observe-poll) operations. Considering that the logic of implementing these operations has proved somewhat more complex than intended, it would be better to centralize this in libcouchbase.

The API will provide a "durability set" which is a group of keys to be monitored, a "per-key" callback for when each key reaches its persistence/durability requirements, a "set callback" for when all keys have been endured (or have encountered a condition in which no further retries can be made)




[CCBC-152] Provide lcb_check to (weakly) check for the existence of a key Created: 21/Dec/12  Updated: 21/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.1
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This API will be a variant of OBSERVE and shall do a simple probe against the master node to see if the key exists in cache.

We need to have this in libcouchbase as normal observe is implemented by 'broadcasting' to all the nodes




[CCBC-135] lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed Created: 04/Dec/12  Updated: 14/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File core    

 Description   
Cannot reproduce, this is all the info I have

FAIL GetMulti - MgetOrdered [tests/phpt/GetMulti/MgetOrdered.phpt]

mnunberg@csure:/sources/lcb2.0-repo/php$ cat tests/phpt/GetMulti/*.out
PHP_COUCHBASE_OK
php: src/server.c:799: lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed.

*Might* be related to CCBC-120

To reiterate again, I cannot reproduce this - I just happened to notice it

 Comments   
Comment by Mark Nunberg [ 05/Dec/12 ]
#0 0x00007f4d29d00475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007f4d29d036f0 in *__GI_abort () at abort.c:92
#2 0x00007f4d29cf9621 in *__GI___assert_fail (assertion=0x7f4d28a88fd3 "nr == sizeof(req)",
    file=<optimized out>, line=799, function=0x7f4d28a893c0 "lcb_server_purge_implicit_responses")
    at assert.c:81
#3 0x00007f4d28a7ab53 in lcb_server_purge_implicit_responses (c=0x2c35ff0, seqno=34,
    end=1354691478793889000) at src/server.c:799
#4 0x00007f4d28a79a56 in lcb_server_destroy (server=0x2c35ff0) at src/server.c:394
#5 0x00007f4d28a74ba5 in lcb_destroy (instance=0x2c83ed0) at src/instance.c:347
#6 0x00007f4d28c9b836 in php_couchbase_res_dtor (rsrc=<optimized out>)
    at /sources/lcb2.0-repo/php/couchbase.c:43
#7 0x00000000006ae89e in list_entry_destructor (ptr=0x7f4d27d177d8)
    at /tmp/buildd/php5-5.4.4/Zend/zend_list.c:183
#8 0x00000000006ac6bd in zend_hash_del_key_or_index (ht=0xdb8750, arKey=0x0, nKeyLength=0, h=56,
    flag=753497888) at /tmp/buildd/php5-5.4.4/Zend/zend_hash.c:531
#9 0x00000000006ae9b1 in _zend_list_delete (id=14243) at /tmp/buildd/php5-5.4.4/Zend/zend_list.c:57
#10 0x000000000068fa5a in _zval_ptr_dtor (zval_ptr=0x7f4d27d175d8)
    at /tmp/buildd/php5-5.4.4/Zend/zend_variables.h:35
#11 0x00000000006bf947 in zend_object_std_dtor (object=0x7f4d27c9c978)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects.c:54
#12 0x00000000006bf979 in zend_objects_free_object_storage (object=0x7f4d27c9c978)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects.c:137
#13 0x00000000006c5693 in zend_objects_store_del_ref_by_handle_ex (handle=753383344, handlers=0x37a3)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects_API.c:220
#14 0x00000000006c56b3 in zend_objects_store_del_ref (zobject=0x7f4d27ca0b60)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects_API.c:172
#15 0x000000000068fa5a in _zval_ptr_dtor (zval_ptr=0x7f4d27cfc130)
    at /tmp/buildd/php5-5.4.4/Zend/zend_variables.h:35
#16 0x00000000006ac7b8 in zend_hash_destroy (ht=0x7f4d27cfd508)
    at /tmp/buildd/php5-5.4.4/Zend/zend_hash.c:560
#17 0x00000000006bf8ec in zend_object_std_dtor (object=0x7f4d27cb9900)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects.c:44
#18 0x00000000006bf979 in zend_objects_free_object_storage (object=0x7f4d27cb9900)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects.c:137
#19 0x00000000006c5693 in zend_objects_store_del_ref_by_handle_ex (handle=753383728, handlers=0x37a3)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects_API.c:220
#20 0x00000000006c56b3 in zend_objects_store_del_ref (zobject=0x7f4d27cb9100)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects_API.c:172
#21 0x000000000068fa5a in _zval_ptr_dtor (zval_ptr=0x7f4d27cb9a78)
    at /tmp/buildd/php5-5.4.4/Zend/zend_variables.h:35
#22 0x00000000006ac7b8 in zend_hash_destroy (ht=0x7f4d27cb9c58)
    at /tmp/buildd/php5-5.4.4/Zend/zend_hash.c:560
---Type <return> to continue, or q <return> to quit---
#23 0x000000000069d9f7 in _zval_dtor_func (zvalue=0x7f4d27cb9d38)
    at /tmp/buildd/php5-5.4.4/Zend/zend_variables.c:43
#24 0x000000000068fa5a in _zval_ptr_dtor (zval_ptr=0x7f4d27e325b0)
    at /tmp/buildd/php5-5.4.4/Zend/zend_variables.h:35
#25 0x00000000006bf947 in zend_object_std_dtor (object=0x7f4d27cf8fa8)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects.c:54
#26 0x00000000006bf979 in zend_objects_free_object_storage (object=0x7f4d27cf8fa8)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects.c:137
#27 0x00000000006c51c7 in zend_objects_store_free_object_storage (objects=0xdb8880)
    at /tmp/buildd/php5-5.4.4/Zend/zend_objects_API.c:92
#28 0x000000000068ffbc in shutdown_executor () at /tmp/buildd/php5-5.4.4/Zend/zend_execute_API.c:297
#29 0x000000000069e9e5 in zend_deactivate () at /tmp/buildd/php5-5.4.4/Zend/zend.c:941
#30 0x000000000063e36a in php_request_shutdown (dummy=0x37a3) at /tmp/buildd/php5-5.4.4/main/main.c:1781
#31 0x0000000000748374 in do_cli (argc=0, argv=0x7fff0f1f03e5)
    at /tmp/buildd/php5-5.4.4/sapi/cli/php_cli.c:1174
#32 0x00000000004310ca in main (argc=32767, argv=0x2a5e210)
    at /tmp/buildd/php5-5.4.4/sapi/cli/php_cli.c:1361
(gdb)
Comment by Mark Nunberg [ 05/Dec/12 ]
Core Dump -- self-compiled versions of lcb and php-ext
Comment by Matt Ingenthron [ 14/Dec/12 ]
Talking with Mark, he can't regularly repro this, but does still see it randomly




[CCBC-5] abort in do_read_data (c=0x7b09bf0) at src/event.c:105 Created: 17/Aug/11  Updated: 10/Dec/12  Resolved: 25/Aug/11

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: built with debug on
libcouchbase: b9796af Fixed compilation of the packet_debug extension
memcachetest: 0.8.3-1-geaf0f9f (my master branch)


 Description   
GDB seems to think libcouchbase is in the middle of a recv() call, but I don't think that's actually correct. I did a packet capture, and I see that libcouchbase hangs right after the noop request, but somehow slips in another request that's not in serial order right before it.

Some gdb output:
(gdb) run -h 10.1.5.7:8091 -L 5 -i 1000 -M 8165 -F
Starting program: /root/debugmemcachetest/bin/memcachetest -h 10.1.5.7:8091 -L 5 -i 1000 -M 8165 -F
[Thread debugging using libthread_db enabled]
[New Thread 47907415978192 (LWP 2195)]
[New Thread 1094388032 (LWP 2198)]
[Thread 1094388032 (LWP 2198) exited]
Failed to get server stats
[New Thread 1094388032 (LWP 2199)]

Program received signal SIGABRT, Aborted.
[Switching to Thread 1094388032 (LWP 2199)]
0x000000322ec30155 in raise () from /lib64/libc.so.6
(gdb) where
#0 0x000000322ec30155 in raise () from /lib64/libc.so.6
#1 0x000000322ec31bf0 in abort () from /lib64/libc.so.6
#2 0x00002b924ffe98ec in do_read_data (c=0x108b0bf0) at src/event.c:105
#3 0x00002b924ffe9aa7 in libcouchbase_server_event_handler (sock=10, which=2,
    arg=0x108b0bf0) at src/event.c:152
#4 0x00002b925020b59a in event_base_loop () from /usr/lib64/libevent-1.4.so.2
#5 0x00002b924ffe9d4d in libcouchbase_execute (instance=0x108608c0)
    at src/execute.c:39
#6 0x00000000004055a8 in memcached_get_wrapper (connection=0x1085bf90,
    key=0x413affb0 "335", nkey=3, size=0x413affa8, data=0x413affa0)
    at main.c:448
#7 0x00000000004052ec in test (ctx=0x1085df70) at main.c:727
#8 0x000000000040567e in test_thread_main (arg=0x1085df70) at main.c:759
#9 0x000000322f806307 in start_thread () from /lib64/libpthread.so.0
#10 0x000000322ecd1ded in clone () from /lib64/libc.so.6
(gdb) down
Bottom (innermost) frame selected; you cannot go down.
(gdb) up
#1 0x000000322ec31bf0 in abort () from /lib64/libc.so.6
(gdb) up
#2 0x00002b924ffe98ec in do_read_data (c=0x108b0bf0) at src/event.c:105
105 src/event.c: No such file or directory.
in src/event.c
(gdb) p/x req->request.magic
$1 = 0x81
(gdb) p req
$2 = (protocol_binary_request_header *) 0x1087e1f0
(gdb) p req[0]
$3 = {request = {magic = 129 '\201', opcode = 9 '\t', keylen = 0,
    extlen = 4 '\004', datatype = 0 '\0', vbucket = 0, bodylen = 3911122944,
    opaque = 1002, cas = 14723677855292588032},
  bytes = "\201\t\000\000\004\000\000\000\000\000\037??\003\000\000\000\000\000\000\000\003U?"}
(gdb) p operations
$4 = 2
(gdb) p nr
$5 = 0
(gdb) p c->cmd_log.avail
$6 = 51
(gdb) p c[0]
$7 = {hostname = 0x1085de90 "10.1.5.7", port = 0x1085de99 "11210", sock = 10,
  root_ai = 0x1085ea00, curr_ai = 0x1085ea00, output = {
    data = 0x108759c0 "\200\t", size = 16384, avail = 0}, output_cookies = {
    data = 0x108799d0 "\020?:A", size = 2048, avail = 16}, cmd_log = {
    data = 0x1087a1e0 "\200\t", size = 16384, avail = 51}, pending = {
    data = 0x108707d0 "\200\001", size = 16384, avail = 0}, pending_cookies = {
    data = 0x1086ffc0 "\220?:A", size = 2048, avail = 0},
  current_packet = 18446744073709551615, input = {data = 0x1087e1f0 "\201\t",
    size = 8192, avail = 8192}, sasl_conn = 0x108747e0, ev_event = {ev_next = {
      tqe_next = 0x0, tqe_prev = 0x108608f0}, ev_active_next = {
      tqe_next = 0x0, tqe_prev = 0x1085ca10}, ev_signal_next = {
      tqe_next = 0x0, tqe_prev = 0x0}, min_heap_idx = 4294967295,
    ev_base = 0x1085bff0, ev_fd = 10, ev_events = 18, ev_ncalls = 0,
    ev_pncalls = 0x413afe2e, ev_timeout = {tv_sec = 0, tv_usec = 0},
    ev_pri = 0,
    ev_callback = 0x2b924ffe9a78 <libcouchbase_server_event_handler>,
    ev_arg = 0x108b0bf0, ev_res = 2, ev_flags = 130}, ev_flags = 2,
  connected = true,
  ev_handler = 0x2b924ffe9a78 <libcouchbase_server_event_handler>,
  instance = 0x108608c0}


Though, interestingly, if I bump up the -i, it gets to 8335 operations before hitting an abort:
(gdb) run -h 10.1.5.7:8091 -L 5 -i 10000 -M 8165 -F
Starting program: /root/debugmemcachetest/bin/memcachetest -h 10.1.5.7:8091 -L 5 -i 10000 -M 8165 -F
[Thread debugging using libthread_db enabled]
[New Thread 47268074524880 (LWP 2825)]
[New Thread 1107622208 (LWP 2828)]
[Thread 1107622208 (LWP 2828) exited]
Failed to get server stats
[New Thread 1107622208 (LWP 2829)]

Program received signal SIGABRT, Aborted.
[Switching to Thread 1107622208 (LWP 2829)]
0x000000322ec30155 in raise () from /lib64/libc.so.6
(gdb) where
#0 0x000000322ec30155 in raise () from /lib64/libc.so.6
#1 0x000000322ec31bf0 in abort () from /lib64/libc.so.6
#2 0x00002afd744638ec in do_read_data (c=0x8163cb0) at src/event.c:105
#3 0x00002afd74463aa7 in libcouchbase_server_event_handler (sock=10, which=2,
    arg=0x8163cb0) at src/event.c:152
#4 0x00002afd7468559a in event_base_loop () from /usr/lib64/libevent-1.4.so.2
#5 0x00002afd74463d4d in libcouchbase_execute (instance=0x80aa200)
    at src/execute.c:39
#6 0x00000000004055a8 in memcached_get_wrapper (connection=0x80a58d0,
    key=0x4204efb0 "8335", nkey=4, size=0x4204efa8, data=0x4204efa0)
    at main.c:448
#7 0x00000000004052ec in test (ctx=0x80a78b0) at main.c:727
#8 0x000000000040567e in test_thread_main (arg=0x80a78b0) at main.c:759
#9 0x000000322f806307 in start_thread () from /lib64/libpthread.so.0
#10 0x000000322ecd1ded in clone () from /lib64/libc.so.6
(gdb) p c[0]
cannot subscript something of type `<data variable, no debug info>'
(gdb) up
#1 0x000000322ec31bf0 in abort () from /lib64/libc.so.6
(gdb) up
#2 0x00002afd744638ec in do_read_data (c=0x8163cb0) at src/event.c:105
105 src/event.c: No such file or directory.
in src/event.c
(gdb) p c[0]
$1 = {hostname = 0x80a77d0 "10.1.5.7", port = 0x80a77d9 "11210", sock = 10,
  root_ai = 0x80a8340, curr_ai = 0x80a8340, output = {
    data = 0x8128a80 "\200\t", size = 16384, avail = 0}, output_cookies = {
    data = 0x812ca90 "\020?\004B", size = 2048, avail = 16}, cmd_log = {
    data = 0x812d2a0 "\200\t", size = 16384, avail = 52}, pending = {
    data = 0x8123890 "\200\001", size = 16384, avail = 0}, pending_cookies = {
    data = 0x8123080 "\220?\004B", size = 2048, avail = 0},
  current_packet = 18446744073709551615, input = {data = 0x81312b0 "\201\t",
    size = 8192, avail = 8192}, sasl_conn = 0x81278a0, ev_event = {ev_next = {
      tqe_next = 0x0, tqe_prev = 0x80aa230}, ev_active_next = {tqe_next = 0x0,
      tqe_prev = 0x80a6350}, ev_signal_next = {tqe_next = 0x0,
      tqe_prev = 0x0}, min_heap_idx = 4294967295, ev_base = 0x80a5930,
    ev_fd = 10, ev_events = 18, ev_ncalls = 0, ev_pncalls = 0x4204ee2e,
    ev_timeout = {tv_sec = 0, tv_usec = 0}, ev_pri = 0,
    ev_callback = 0x2afd74463a78 <libcouchbase_server_event_handler>,
    ev_arg = 0x8163cb0, ev_res = 2, ev_flags = 130}, ev_flags = 2,
  connected = true,
  ev_handler = 0x2afd74463a78 <libcouchbase_server_event_handler>,
  instance = 0x80aa200}
(gdb) p req[0]
$2 = {request = {magic = 129 '\201', opcode = 9 '\t', keylen = 0,
    extlen = 4 '\004', datatype = 0 '\0', vbucket = 0, bodylen = 3911122944,
    opaque = 10002, cas = 477384859036155904},
  bytes = "\201\t\000\000\004\000\000\000\000\000\037?\022'\000\000\000\000\000\000\000\003?\006"}




Context from my libcouchbase:

    58 case PROTOCOL_BINARY_RES:
    59 libcouchbase_server_purge_implicit_responses(c, res->response.opaque);
    60 c->instance->response_handler[res->response.opcode](c,
    61 command_cookie,
    62 res);
    63 req = (protocol_binary_request_header*)c->cmd_log.data;
    64 processed = ntohl(req->request.bodylen) + sizeof(*req);
    65 assert(c->cmd_log.avail >= processed);
    66 memmove(c->cmd_log.data, c->cmd_log.data + processed,
    67 c->cmd_log.avail - processed);
    68 c->cmd_log.avail -= processed;
    69 req = (protocol_binary_request_header*)c->input.data;
    70
    71 memmove(c->output_cookies.data, cptr, (size_t)(end - cptr));
    72 c->output_cookies.avail -= sizeof(command_cookie);
    73 break;
    74 default:
    75 abort();
    76 }
    77 }
    78
    79 processed = ntohl(req->request.bodylen) + sizeof(*req);
    80 memmove(c->input.data, c->input.data + processed,
    81 c->input.avail - processed);
    82 c->input.avail -= processed;
    83 }
    84
    85 if (operations == operations_per_call) {
    86 // allow some other connections to process some data as well
    87 return ;
    88 }
    89
    90 nr = recv(c->sock,
    91 c->input.data + c->input.avail,
    92 c->input.size - c->input.avail,
    93 0);
    94
    95 if (nr == -1) {
    96 switch (errno) {
    97 case EINTR:
    98 break;
    99 case EWOULDBLOCK:
   100 return;
   101 default:
   102 abort();
   103 }
   104 } else if (nr == 0) {
   105 abort();
   106 } else {
   107 c->input.avail += (size_t)nr;
   108 }
   109 } while (true);


 Comments   
Comment by Sergey Avseyev [ 10/Dec/12 ]
Added SHA1 of the fix fb42840




[CCBC-111] Make iops functions 'private', and discourage calls to run_event_loop/stop_event_loop Created: 22/Oct/12  Updated: 10/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta2
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There's quite a bit of user code which calls io->run_event_loop and io->stop_event_loop.

These functions are largely private and are intended for use by libcouchbase itself. The fact that they are public API is only accidental, due to the fact that they are part of the iops structure.

These functions should never be called by user code:

1) They are a source of confusion (especially when mixed with lcb_wait)
2) They prevent any kind of wrapping/sanity logic done by lcb_wait (some already exists, and some can be implemented in future versions)

The following should be done

1) Remove the iops structure definition from being included by the main header file. Thus, #include <libcouchbase/couchbase.h> should not allow a dereference of the iops structure. Rather the iops structure should be defined in a new header file (iops.h) - so that users who want to explicitly manipulate and write event loop plugins can include it, but should otherwise not be present in 'user' code. types.h should include a predeclaration of the structure so that it may still be passed around as a simple pointer value.

2) Document the purpose of run_event_loop/stop_event_loop. Make it very clear that lcb_wait should be used in all synchronous use cases. If we have any examples using run_event_loop/stop_event_loop, remove them.

3) [ Optional? ] - In order to further discourage users from using these functions, throw a compiler warning/error for those functions unless LIBCOUCHBASE_INTERNAL is defined - tbd




[CCBC-49] API reference is complete Created: 12/Jan/12  Updated: 10/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 1.0
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Major
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-108] Allow per-command retries for not-my-vbucket and other transient errors Created: 04/Oct/12  Updated: 10/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0beta
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We should avoid the possibility of 'infinite' errors and allow the user the ability of seeing the 'real' error which caused the command to fail..

http://review.couchbase.org/13177




libcouchbase test improvements (CCBC-137)

[CCBC-139] add timeout tests for libcouchbase Created: 31/Aug/12  Updated: 08/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Farshid Ghods [ 31/Aug/12 ]
1- timeout to a server that is down
2- timeout a server that is slow
Comment by Mark Nunberg [ 05/Sep/12 ]
Apparently there are some new tests for this already.. I'll document them again during next pass




libcouchbase test improvements (CCBC-137)

[CCBC-141] test libcouchbase lock functionality Created: 10/Sep/12  Updated: 08/Dec/12  Resolved: 12/Sep/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





libcouchbase test improvements (CCBC-137)

[CCBC-142] test libcouchbase observe functionality Created: 10/Sep/12  Updated: 08/Dec/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-100] Allow Incremental Row Callbacks Created: 11/Sep/12  Updated: 08/Dec/12

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
View query results may return large amounts of data. We should offer a row callback which returns a JSON string whose contents is a single row.

There are awesome JSON libraries to do this task, and it's something that I've found myself doing in the Perl client.. so why not move it into libcouchbase and let everyone be happy? :).

Basic idea:
Two callbacks are put into place. The first is a 'row callback' which receives rows, and the second is a complete callback, at which time any other non-row data is returned as a JSON object.

So for example

{ "total_rows" : 30,
  "rows" : [
   { "id" : "foo" },
   { "id" : bar" },
     ....

The row callback will first receive the string '{ "id" : "foo" }', then the second time around, '{"id":"bar"}'
The complete callback will return:
' { "total_rows" : 30 } '





[CCBC-125] Man page for cbc does not describe the new -f argument for cbc hash Created: 14/Nov/12  Updated: 03/Dec/12  Resolved: 14/Nov/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Trond Norbye Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-129] Add libevent support for win32 Created: 20/Nov/12  Updated: 21/Nov/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0beta2
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Minor
Reporter: Brett Harrison Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive win32libevent.zip    

 Description   
I created a NMakefile and made changes to the code to support libevent on windows.

Files affected:
iofactory.c
plugin-libevent.c

New file:
NMakefile.libevent

I uploaded a zipfile of the changes if you are interested in merging them.

 Comments   
Comment by Sergey Avseyev [ 21/Nov/12 ]
it would be awesome if you can move your patch to our review system. http://review.couchbase.com
Comment by Brett Harrison [ 21/Nov/12 ]
Is there a guild on how to use the review system? If you can explain the steps I'll upload the patch.
Comment by Matt Ingenthron [ 21/Nov/12 ]
It's roughly http://www.couchbase.com/wiki/display/couchbase/Contributing+Changes

Repo tool isn't required for this project though
Comment by Matt Ingenthron [ 21/Nov/12 ]
Also, Brett: saw your CLA on review.couchbase.org and you're all set.




[CCBC-107] Handle stale/already sent pending commands on network error Created: 04/Oct/12  Updated: 21/Nov/12  Resolved: 19/Nov/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.0beta
Fix Version/s: 2.0.0beta3
Security Level: Public

Type: Improvement Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://review.couchbase.org/13667

 Comments   
Comment by Sergey Avseyev [ 13/Nov/12 ]
what is stale command? libcouchbase doesn't have timeout per each command, so if server required to respond, but doesn't do it -- it is server bug. from other hand, if socket looks like dead, we are using timeout to purge requests
Comment by Matt Ingenthron [ 14/Nov/12 ]
Mark: Do you believe this needs to be addressed before 2.0 GA? If so, please drop in a note with the details.

My current read on this is that it's not really needed.
Comment by Sergey Avseyev [ 15/Nov/12 ]
Putting explicit link to test case http://review.couchbase.org/#/c/13667/12/tests/smoke-test.c
Comment by Matt Ingenthron [ 15/Nov/12 ]
Sergey: please extract the test in the referenced patchset and get it correct. Then, please update this issue with any findings from running that test. We'll see from there whether or not to keep it as a blocker.

I do agree that if operations timeout, that's okay when the connection is interrupted.
Comment by Sergey Avseyev [ 19/Nov/12 ]
The test has been ported to recent libcouchbase http://review.couchbase.org/22585 and currently it is passing
Comment by Sergey Avseyev [ 19/Nov/12 ]
Resolving the ticket, because the unit test from that patch is passing on recent version




[CCBC-126] Failed to create IO Instance error. Created: 27/Oct/12  Updated: 21/Nov/12  Resolved: 20/Nov/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta2
Fix Version/s: 2.0.0beta3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Paul B Assignee: Sergey Avseyev
Resolution: Won't Fix Votes: 0
Labels: php
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: - CentOS release 5.4 (Final)
- PHP 5.3.18

and

Latest Basic Amazon AMI 64 bit (3.2.30-49.59.amzn1.x86_64) --> http://aws.amazon.com/amazon-linux-ami/


 Description   
This is the php used when getting the error:

<?php
$cb = new Couchbase("localhost:8091", "Administrator", "password", "default");
$cb->set("a", 1);
var_dump($cb->get("a"));
?>



---------------------------
ERROR
----------------------------

PHP Warning: Couchbase::__construct(): failed to create IO instance in /var/www/test_couchbase.php on line 3


 Comments   
Comment by Matt Ingenthron [ 28/Oct/12 ]
Sergey: I believe this is caused by a problem with the new packaging. Amazon Linux doesn't install an event backend by default for libcouchbase. Can you look into this?
Comment by Jeff Minard [ 29/Oct/12 ]
I had the same in centos5. Base on your suggestion Matt, I ran

    sudo yum install libcouchbase2-libevent.x86_64


and the error went away.
Comment by Paul B [ 30/Oct/12 ]
Jeff Minard:

I currently have
libevent-2.0.18-1.9.amzn1.x86_64
libevent-devel-2.0.18-1.9.amzn1.x86_64

 when I try install that I get this error:

Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libcouchbase2-libevent.x86_64 0:2.0.0beta2-1 will be installed
--> Processing Dependency: libevent-1.4.so.2()(64bit) for package: libcouchbase2-libevent-2.0.0beta2-1.x86_64
--> Finished Dependency Resolution
Error: Package: libcouchbase2-libevent-2.0.0beta2-1.x86_64 (couchbase)
           Requires: libevent-1.4.so.2()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Does anyone know where to get this file as I cannot seem to locate it.


Comment by Sergey Avseyev [ 30/Oct/12 ]
Could you please point where amazon said that amazon latest ami is centos 5.5? We doesn't build RPMs for amazon AMI. You should install libcouchbase from sources.
Comment by Sergey Avseyev [ 30/Oct/12 ]
You should use only official centos packages (5.5 for example has only libevent1)

[root@ip-10-28-193-82 ~]# yum install libcouchbase2-libevent
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.us.leaseweb.net
 * extras: mirror.cogentco.com
 * updates: mirror.cogentco.com
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libcouchbase2-libevent.i386 0:2.0.0beta2-1 set to be updated
--> Processing Dependency: libcouchbase2 for package: libcouchbase2-libevent
--> Running transaction check
---> Package libcouchbase2.i386 0:2.0.0beta2-1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================
 Package Arch Version Repository Size
=========================================================================================================================================================================
Installing:
 libcouchbase2-libevent i386 2.0.0beta2-1 couchbase 28 k
Installing for dependencies:
 libcouchbase2 i386 2.0.0beta2-1 couchbase 81 k

Transaction Summary
=========================================================================================================================================================================
Install 2 Package(s)
Upgrade 0 Package(s)

Total download size: 109 k
Is this ok [y/N]: y
Downloading Packages:
(1/2): libcouchbase2-libevent-2.0.0beta2-1.i386.rpm | 28 kB 00:00
(2/2): libcouchbase2-2.0.0beta2-1.i386.rpm | 81 kB 00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 374 kB/s | 109 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : libcouchbase2-libevent 1/2
  Installing : libcouchbase2 2/2

Installed:
  libcouchbase2-libevent.i386 0:2.0.0beta2-1

Dependency Installed:
  libcouchbase2.i386 0:2.0.0beta2-1

Complete!
Comment by Sergey Avseyev [ 30/Oct/12 ]
curl -s http://vault.centos.org/5.5/os/i386/CentOS/ | grep libevent
Comment by Matt Ingenthron [ 16/Nov/12 ]
Sergey: you marked this one as resolved, but did not indicate what the resolution was. Is the resolution "workaround, please use libevent1?".

Please clarify and change the state.
Comment by Sergey Avseyev [ 20/Nov/12 ]
Yes, all RPM systems where we are building _binary_ packages have only libevent1, therefore if someone decided to install libevent2, he should either install libcouchbase from tarball, or make his own RPM packages using 'make dist-rpm' target




[CCBC-124] cbc hash shows incorrect replica bucket host Created: 13/Nov/12  Updated: 21/Nov/12  Resolved: 15/Nov/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.0beta3
Security Level: Public

Type: Bug Priority: Minor
Reporter: Chad Kouse Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File vbucketmap    

 Description   
cbc hash --host <my host> <my key>
"<my key>" vBucket:931 Server:"172.16.10.119:11210" Replicas:"172.16.10.119:11210"

attaching the vbucket map from that host

 Comments   
Comment by Sergey Avseyev [ 13/Nov/12 ]
could you tell what key you are trying?
Comment by Chad Kouse [ 13/Nov/12 ]
cbc hash --host mem16.shared.day1.tunewiki.net tw:twobbleitem:1:1352759847634412209
"tw:twobbleitem:1:1352759847634412209" vBucket:931 Server:"172.16.10.119:11210" Replicas:"172.16.10.119:11210"
Comment by Chad Kouse [ 13/Nov/12 ]
vbuckettool output for the same key:

elliott@mem16:/opt/couchbase/bin$ curl http://localhost:8091/pools/default/buckets/default | ./vbuckettool - tw:twobbleitem:1:1352759847634412209
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 17071 100 17071 0 0 397k 0 --:--:-- --:--:-- --:--:-- 416k
key: tw:twobbleitem:1:1352759847634412209 master: 172.16.10.119:11210 vBucketId: 931 couchApiBase: (null) replicas: 172.16.10.123:11210
elliott@mem16:/opt/couchbase/bin$ curl http://mem12.shared.day1.tunewiki.net:8091/pools/default/buckets/default | ./vbuckettool - tw:twobbleitem:1:1352759847634412209
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 17095 100 17095 0 0 20907 0 --:--:-- --:--:-- --:--:-- 20949
key: tw:twobbleitem:1:1352759847634412209 master: 172.16.10.119:11210 vBucketId: 931 couchApiBase: (null) replicas: 172.16.10.123:11210
Comment by Sergey Avseyev [ 13/Nov/12 ]
fix http://review.couchbase.org/22488
Comment by Sergey Avseyev [ 13/Nov/12 ]
the fix was trivial, cbc tool just picked wrong server index




[CCBC-104] Client crashes when entrypoint node connection is dropped Created: 02/Oct/12  Updated: 20/Nov/12  Resolved: 20/Nov/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta
Fix Version/s: 2.0.0beta3
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Entry point node is the REST node passed to the constructor


 Description   
 'DelayMin': 1, 'DelayMax': 10}}
[driver driver.py:36] > MC_DS_MUTATE_SET:40 @19 {'DSType': 'DSTYPE_SEEDED', 'DS': {'Count': 1000, 'Repeat': '_REP_', 'VSeed': 'SimpleValue_', 'KSize': 100, 'Continuous': True, 'VSize': 12, 'KSeed': 'SimpleKey_'}, 'Options': {'TimeRes': 1, 'IterWait': True, 'DelayMin': 1, 'DelayMax': 10}}
[driver failover.py:134] Ramp for 3 seconds..
[driver failover.py:141] No service specified..
[driver rest_client.py:710] fail_over successful
sdkd_lcb: src/instance.c:495: relocate_packets: Assertion `ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes)' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff21a8700 (LWP 5726)]
0x00007ffff6e7d475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff6e7d475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff6e806f0 in *__GI_abort () at abort.c:92
#2 0x00007ffff6e76621 in *__GI___assert_fail (
    assertion=0x7ffff6a20478 "ringbuffer_write(&dst->cmd_log, cmd.bytes, sizeof(cmd.bytes)) == sizeof(cmd.bytes)",
    file=<optimized out>, line=495, function=0x7ffff6a20790 "relocate_packets") at assert.c:81
#3 0x00007ffff6a10724 in vbucket_stream_handler () from /home/mnunberg/src/sdkd-cpp/inst/2.0.0beta//lib/libcouchbase.so.2
#4 0x00007ffff5bc90a4 in event_process_active_single_queue (activeq=0x635250, base=0x63c510) at event.c:1346
#5 event_process_active (base=<optimized out>) at event.c:1416
#6 event_base_loop (base=0x63c510, flags=0) at event.c:1617
#7 0x00007ffff7bbc922 in CBSdkd::Handle::postsubmit (this=0x63c810, rs=..., nsubmit=1) at Handle.cpp:206
#8 0x00007ffff7bbceeb in CBSdkd::Handle::dsMutate (this=0x63c810, cmd=..., ds=..., out=..., options=...) at Handle.cpp:300
#9 0x00007ffff7bc8ab1 in CBSdkd::WorkerDispatch::_process_request (this=0x63c230, req=..., rs=0x63c420) at IODispatch.cpp:494
#10 0x00007ffff7bc93d5 in CBSdkd::WorkerDispatch::run (this=0x63c230) at IODispatch.cpp:572
#11 0x00007ffff7bc7276 in CBSdkd::new_worker_thread (worker=0x63c230) at IODispatch.cpp:227
#12 0x00007ffff71d8b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#13 0x00007ffff6f2370d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#14 0x0000000000000000 in ?? ()
(gdb) [driver rest_client.py:710] fail_over successful
[driver failover.py:149] Sleeping for 3 seconds after failover
[driver driver.py:36] > CANCEL:41 @2 {}


 Comments   
Comment by Mark Nunberg [ 03/Oct/12 ]
This happens during failover of a non-entrypoint node.

#0 __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:2571
#1 0x00007ffff6a11832 in ringbuffer_write ()
   from /home/mnunberg/src/sdkd-cpp/inst/2.0.0beta//lib/libcouchbase.so.2
#2 0x00007ffff6a10110 in vbucket_stream_handler ()
   from /home/mnunberg/src/sdkd-cpp/inst/2.0.0beta//lib/libcouchbase.so.2
#3 0x00007ffff5bc90a4 in event_process_active_single_queue (activeq=0x656360, base=0x65dc80) at event.c:1346
#4 event_process_active (base=<optimized out>) at event.c:1416
#5 event_base_loop (base=0x65dc80, flags=0) at event.c:1617
#6 0x00007ffff7bbc922 in CBSdkd::Handle::postsubmit (this=0x65df80, rs=..., nsubmit=1) at Handle.cpp:206
#7 0x00007ffff7bbcadd in CBSdkd::Handle::dsGet (this=0x65df80, cmd=..., ds=..., out=..., options=...)
    at Handle.cpp:244
#8 0x00007ffff7bc8b0a in CBSdkd::WorkerDispatch::_process_request (this=0x65d9a0, req=..., rs=0x65db90)
    at IODispatch.cpp:498
#9 0x00007ffff7bc93d5 in CBSdkd::WorkerDispatch::run (this=0x65d9a0) at IODispatch.cpp:572
#10 0x00007ffff7bc7276 in CBSdkd::new_worker_thread (worker=0x65d9a0) at IODispatch.cpp:227
#11 0x00007ffff71d8b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#12 0x00007ffff6f2370d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#13 0x0000000000000000 in ?? ()
(gdb) [driver failover.py:293] Sleeping 10 seconds after all failovers
Comment by Farshid Ghods [ 03/Oct/12 ]
do client crashes count as blocker ?
is this a scenario where users will run into during failover operation ?
Comment by Mark Nunberg [ 03/Oct/12 ]
Client crashes are fatal errors, but I wouldn't yet mark the bug as a 'blocker'.. (it is 'major', but it doesn't prevent normal client functionality). But feel free to re-classify.

I would guess this is a scenario in which users might run into.
Comment by Mark Nunberg [ 03/Oct/12 ]
stack trace with line/context infromation.

I might also note that this does *not* happen when the cluster is immediately rebalanced...

#0 __memcpy_ssse3_back ()
    at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:1995
#1 0x00007ffff6a10239 in ringbuffer_write (buffer=0x7fffd80256e0,
    src=0x7fffee99c4a0, nb=24) at src/ringbuffer.c:127
#2 0x00007ffff6a0d86d in relocate_packets (src=0x7fffd8001be0,
    dst_instance=0x66bdf0) at src/instance.c:495
#3 0x00007ffff6a0dc8f in lcb_update_serverlist (instance=0x66bdf0)
    at src/instance.c:556
#4 0x00007ffff6a0ea51 in vbucket_stream_handler (sock=125, which=2,
    arg=0x66bdf0) at src/instance.c:938
#5 0x00007ffff5bc50a4 in event_process_active_single_queue (activeq=0x66b820,
    base=0x66b200) at event.c:1346
#6 event_process_active (base=<optimized out>) at event.c:1416
#7 event_base_loop (base=0x66b200, flags=0) at event.c:1617
#8 0x00007ffff5dfc68d in lcb_io_run_event_loop (iops=0x66b610)
    at plugins/io/libevent/plugin-libevent.c:315
#9 0x00007ffff6a1526d in lcb_wait (instance=0x66bdf0) at src/wait.c:112
#10 0x00007ffff7bbc922 in CBSdkd::Handle::postsubmit (this=0x66b500, rs=...,
    nsubmit=1) at Handle.cpp:206
#11 0x00007ffff7bbcadd in CBSdkd::Handle::dsGet (this=0x66b500, cmd=...,
    ds=..., out=..., options=...) at Handle.cpp:244
#12 0x00007ffff7bc8b0a in CBSdkd::WorkerDispatch::_process_request (
    this=0x66af20, req=..., rs=0x66b110) at IODispatch.cpp:498
#13 0x00007ffff7bc93d5 in CBSdkd::WorkerDispatch::run (this=0x66af20)
---Type <return> to continue, or q <return> to quit---
    at IODispatch.cpp:572
#14 0x00007ffff7bc7276 in CBSdkd::new_worker_thread (worker=0x66af20)
    at IODispatch.cpp:227
#15 0x00007ffff71d8b50 in start_thread (arg=<optimized out>)
    at pthread_create.c:304
#16 0x00007ffff6f2370d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#17 0x0000000000000000 in ?? ()
Comment by Matt Ingenthron [ 14/Nov/12 ]
I believe this is the one that is causing many SDK test failures.

Mark: can you confirm?

Marking it as a blocker until fully assessed.
Comment by Sergey Avseyev [ 15/Nov/12 ]
Just verified with local cluster of 5 nodes (8091 using ./install/bin/couchbase-server and 9000-9003 using ./cluster_run --nodes=4) and this ruby script (all versions are most recent)

conn = Couchbase.bucket
i = 0
loop do
  i += 1
  STDERR.print("\r#{i} ")
  begin
    conn.set("foo-#{i % 1_000}", i)
  rescue Couchbase::Error::Base => ex
    STDERR.puts("#{ex.class}: #{ex}\n")
  end
end

Tested two scenarios:

First

1) the single thread is connecting to 8091 node and start load
2) kill the 8091 process
3) the client hops to another node for config connection and start reporting Connect errors from data sockets
4) restart 8091
5) the client start reporting TMP_FAIL and once cluster make 8091 node green, it shuts up and continue load

Second

1) the single thread is connecting to 8091 node and start load
2) kill the 8091 process
3) the client hops to another node for config connection and start reporting Connect errors from data sockets
4) failover and rebalance node
5) the client receive updated config and stops sending data to broken node
Comment by Matt Ingenthron [ 15/Nov/12 ]
Since more realistic testing scenarios show regular failures here, it'd be good to try to duplicate this there.

Can you work with Mark to try this with a real cluster? He's able to hit this regularly. Schedule time with him as needed.
Comment by Matt Ingenthron [ 19/Nov/12 ]
Scenario to be concerned with is:
client is working with the cluster, bootstrapping off of node A of a cluster of three (A, B, C)
client is listening for config changes from node A
trond walks by and hits the reset button on node A (because he's mean)
no TCP RST
operations for node A keep timing out
even if autofailover is enabled, and the config updates
and node B and C takeover
the client will never get an update and never reconfigure
Comment by Sergey Avseyev [ 20/Nov/12 ]
This patch implements issue counting for paired memcached socket, and once number of failures exceed threshold -- it reconnect config connection

http://review.couchbase.org/22686
Comment by Sergey Avseyev [ 20/Nov/12 ]
Need verification
Comment by Sergey Avseyev [ 20/Nov/12 ]
http://review.couchbase.org/22691

The patch has been merged already, but requires verification on the test installation
Comment by Mark Nunberg [ 20/Nov/12 ]
Fixed and verified in latest commit




[CCBC-110] Implement lcb_reconnect() which will reconnect all dead sockets Created: 17/Oct/12  Updated: 20/Nov/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This functionality very important for long-living connection handles

 Comments   
Comment by Matt Ingenthron [ 20/Nov/12 ]
This issue needs more of a description.

As discussed with Sergey, it's visible when using libcouchbase in a completely asynchronous app with an external event loop. According to Sergey, this does not affect any of our main use cases with Ruby/PHP/node and rebuilding the connection.




[CCBC-127] libvbucket RPM is missing /usr/lib64/libvbucket.la file Created: 16/Nov/12  Updated: 16/Nov/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 1.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Tim Smith Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: libvbucket1-1.8.0.4-1
centos 5.5 couchbase repo
x86_64


 Description   
The libvbucket1 rpm does not include the libvbucket.la file. The libcouchbase.la file references libvbucket.la as a dependency.

Linking via libtool against libcouchbase fails because of this. For example, compiling the php-ext-couchbase package fails like:

/bin/sh /root/code/php-ext-couchbase/libtool --mode=link cc -DPHP_ATOM_INC -I/root/code/php-ext-couchbase/include -I/root/code/php-ext-couchbase/main -I/root/code/php-ext-couchbase -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -I/usr/include/ -DHAVE_CONFIG_H -g -O2 -o couchbase.la -export-dynamic -avoid-version -prefer-pic -module -rpath /root/code/php-ext-couchbase/modules compress.lo couchbase.lo fastlz/fastlz.lo -lz -lcouchbase
grep: /usr/lib64/libvbucket.la: No such file or directory
/bin/sed: can't read /usr/lib64/libvbucket.la: No such file or directory
libtool: link: `/usr/lib64/libvbucket.la' is not a valid libtool archive
make: *** [couchbase.la] Error 1

I created a fake libvbucket.la by copying /usr/lib64/libcouchbase.la and editing it, and it allowed the compile to succeed.






[CCBC-116] Heavy errors during short network loss Created: 07/Nov/12  Updated: 14/Nov/12

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0beta2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
During network loss to the nodes, there are heavy network errors - a combination of both timed-out operations as well as operations which fail with a network error. Might we want to provide an option to retry these operations?



 Comments   
Comment by Matt Ingenthron [ 14/Nov/12 ]
While this would be nice, I think failure of operations during network loss is okay as long as we highlight the loss up the chain appropriately. Leaving this as major.




[CCBC-34] create package scripts for Debian platforms Created: 01/Dec/11  Updated: 13/Nov/12  Resolved: 12/Jan/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-91] timeouts seen after failover, rebalance and add back Created: 13/Aug/12  Updated: 13/Nov/12  Resolved: 18/Aug/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 1.0.4
Fix Version/s: 1.0.5
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: PHP 5.3.3 (cli) (built: Jun 27 2012 12:25:48)
libcouchbase1-1.0.4-1
CentOS release 5.8 (Final), x86_64
Couchbase Server 1.8.1 Enterprise

Attachments: GZip Archive php-ext-couchbase.tar.gz    

 Description   
scenario:
1. Start PHP client in a loop setting and getting against a 2 node cluster
2. Click failover to kick a node out, click rebalance to make it unassociated
3. Walk through the setup wizard on that node, re-add it to the cluster
4. After adding, click rebalance

Expected behavior:
During rebalance in step 4, which is an add node scenario, no timeouts are expected.

Observed behavior:
During rebalance in step 4, I see timeouts from PHP, and they continue even after the rebalance has completed.

 Comments   
Comment by Matt Ingenthron [ 13/Aug/12 ]
A packet capture of this same issue, with the client on MacOS X and CentOS 5.8 servers with Couchbase Server 1.8.1 enterprise edition may be found at http://dl.dropbox.com/u/1537838/failover-maybe-issue
Comment by Sergey Avseyev [ 13/Aug/12 ]
http://review.couchbase.org/19563
Comment by Matt Ingenthron [ 13/Aug/12 ]
Note from discussion, this is a possible fix, not sure.
Comment by Matt Ingenthron [ 13/Aug/12 ]
Sergey and I reproduced the issue, and it's related to the series of steps outlined above. The underlying libcouchbase is not receiving the updated configuration for some reason, and thus is sending items to the wrong node, and then they're timing out.

Sergey will do more work on finding the specific cause.
Comment by Sergey Avseyev [ 14/Aug/12 ]
The patch http://review.couchbase.org/19599 and aforementioned http://review.couchbase.org/19563 solves the issue.

To reproduce it for sure you should failover the node is currently used by client to listen config changes. (Usually it is the first successfull node from initial node list)




[CCBC-109] test libev plugin under CI Created: 11/Oct/12  Updated: 13/Nov/12  Resolved: 06/Nov/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Blocker
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Since libev support is graduating to production support, we need to be testing it under CI regularly.

 Comments   
Comment by Sergey Avseyev [ 06/Nov/12 ]
http://sdkbuilds.couchbase.com/job/libcouchbase/ fixed

[libevent libev]
  x
[centos55_32 centos55_64 centos62_32 centos62_64 ubuntu1004_32 ubuntu1004_64 ubuntu1110_32 ubuntu1110_64]




[CCBC-15] implement OBSERVE command Created: 09/Sep/11  Updated: 13/Nov/12  Resolved: 13/Jul/12

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 1.1.0dp8
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks RCBC-6 implement OBSERVE command Closed
Duplicate
is duplicated by CCBC-81 Implement observe command Closed