[CCBC-528] Large observe packets may cause partial buffers to be sent Created: 18/Oct/14  Updated: 18/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.4
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If a single output buffer for an observe command exceeds 512 bytes, the library may silently drop subsequent items to be appended to the command body.

This is normally not an issue in practice as the observe buffers are typically issued on a per-key basis; and since a key is never larger than 256b, the library accidentally never runs into this bug. This only seems to be an issue with so-called "multi observe" or "multi endure" commands.




[CCBC-527] Durability timeouts may be a bit jumpy Created: 18/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0, 2.4.3
Fix Version/s: 2.4.4
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Durability timeouts may be a bit jumpy due to the code still using the older ns-to-us conversion. Sometimes the values might be subject to integer overflow and cause incorrect timeout behavior.

 Comments   
Comment by ivulfson [ 20/Oct/14 ]
This issue was exhibited while testing the perl Couchbase client. The issue is described in detail here:

https://github.com/mnunberg/perl-Couchbase-Client/issues/25

I've tested Mark's fix in his libcouchbase master branch, and it works.




[CCBC-526] Configuration source may revert to HTTP (port 8091) during excessive errors Created: 16/Oct/14  Updated: 17/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0, 2.4.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Perry Krug Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In AWS, I am running this command:
cbc-pillowfight -U couchbase://ec2-54-176-251-0.us-west-1.compute.amazonaws.com/event_stream -t 128 -B 10000 -I 10000 -m 1024 -M 1024 -n -c -1

I am observing many:
Operation(s) failed: [0x17] Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout

I assume this is related to resource exhaustion and is not the main focus of this bug...but I think it is related so I included it.

The main concern is that within a few seconds of running this command, I observe a large increase in the number of 8091 connections to my Couchbase Server cluster. With the exact command above, the increase is in the range of 60-80 connections before I terminate the command. When adding a "-v" to capture logs, it's only around 6-10 connections increase, but is still more than I would expect and hopefully representative of the problem. With "-vv or -vvv" I am unable to reproduce the behavior which I assume is due to some race condition/context switching.

Logs are here: http://customers.couchbase.com.s3.amazonaws.com/perry/libcouchbase (sorry, it's rather big, the increase of connections should be relatively near the end)

 Comments   
Comment by Mark Nunberg [ 16/Oct/14 ]
I'm moving this to minor because in any event the client will use the "terse" URI (starting from lcb version 2.4.0), which is much more resource friendly to the cluster.

The reason, in this case, that it's falling back to port 8091 is because the connection via the 11210 port is timing out, and the client will switch to the next available bootstrap method.

Note that interaction with port 8091 can be disabled entirely by using 'bootstrap_on=cccp' in the connection string. Note that this behavior is not the default in order to support older servers (which do not have CCCP) and memcached buckets (which do not support CCCP).




[CCBC-525] Make failure tests more deterministic Created: 16/Oct/14  Updated: 17/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Subhashni Balakrishnan
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
From a community user:
I'm trying to build libcouchbase from sources on Ubuntu 12.04 and have several tests failing. Here's the output:

[ RUN ] MockUnitTest.testReconfigurationOnNodeFailover
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:316: Failure
Value of: lcb_get_num_nodes(instance)
Actual: 10
Expected: 9
[ FAILED ] MockUnitTest.testReconfigurationOnNodeFailover (9225 ms)
[ RUN ] MockUnitTest.testBufferRelocationOnNodeFailover
tests/iotests/t_netfail.cc:414: Failure
Value of: rv.error
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
[ FAILED ] MockUnitTest.testBufferRelocationOnNodeFailover (36752 ms)
[ RUN ] MockUnitTest.testSaslMechs
[ OK ] MockUnitTest.testSaslMechs (1853 ms)
[ RUN ] MockUnitTest.testMemcachedFailover
tests/iotests/t_netfail.cc:536: Failure
Value of: lcb_get_num_nodes(instance)
Actual: 10
Expected: 9
[ FAILED ] MockUnitTest.testMemcachedFailover (1743 ms)

Tried building master, release 2.4.2 and 2.3.2.

 Comments   
Comment by Mark Nunberg [ 16/Oct/14 ]
Unfortunately these tests are not fully deterministic; all I can say is that they pass "most of the time", and the reason why the failures appear are due to timing issues between the mock and the client library.

Development of a new, more deterministic mock server is underway, but it will be quite some time until it's properly integrated within the client.

Generally, ensuring there is sufficient CPU and memory resources on the machine running the tests will ensure they have a high pass rate, but this is not guaranteed.
Comment by Mark Nunberg [ 16/Oct/14 ]
For further reference, our tests are executed on our Jenkins builders for each commit. They typically have at least one test that fails on one or another configuration; this will typically be a result of insufficient CPU resources which cause the mock to become slow. See: http://sdkbuilds.couchbase.com/job/lcb-linux-cmake/ for the latest test results.

Comment by Mark Nunberg [ 17/Oct/14 ]
This isn't specific to 2.4.2, but all versions of the library. Assigning to subhashni :) - I know she is working on something




[CCBC-524] OS X/Homebrew install instructions missing from dev guide Created: 15/Oct/14  Updated: 17/Oct/14  Resolved: 15/Oct/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 15/Oct/14 ]
https://github.com/couchbase/docs/pull/12




[CCBC-523] Drop support for anything below TLSv1 Created: 15/Oct/14  Updated: 15/Oct/14  Resolved: 15/Oct/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0, 2.4.1, 2.4.2
Fix Version/s: 2.4.3
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 15/Oct/14 ]
http://review.couchbase.org/#/c/42172/




[CCBC-522] Doc Issue with Personalized Logger in LCB Created: 13/Oct/14  Updated: 14/Oct/14  Resolved: 14/Oct/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Jeff Dillon Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Customer has issue with the sample code at http://docs.couchbase.com/couchbase-sdk-c-2.4/#logging

From the customer:

"In the example at the address above, the procs structure is allocated on the stack, in the scope of the function apply_logging, so this procs object gets deleted when this function exits, whereas the object pointed by the "instance" pointer may still exist.

This seems to be in contradiction with the statement "The lcb_logprocs pointer must point to valid memory and must not be freed by the user after passing it to the library until the instance associated with it is destroyed."

AFAIK, the fact that apply_logging and logger functions are static does not change anything to that."

ASK: Is the customer concern valid?

 Comments   
Comment by Mark Nunberg [ 13/Oct/14 ]
The logging interface in the library is subject to change and thus marked as uncommitted: http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.2/group___l_c_b___c_n_t_l.html#gac5e559536b7d0666361b97e0cfae6495

The example in the doc link above _is_ wrong, though - but that documentation should no longer be visible.

The lcb_logprocs structure as the current interface has it, is stored in the library (as a pointer). The goal of not passing in a callback directly is for the logging callback to have access to whatever (if any) application-specific logging framework context being used (so the callback can just proxy over to it).
Comment by Mark Nunberg [ 13/Oct/14 ]
https://github.com/couchbaselabs/docs-ng/pull/163




[CCBC-520] Incorrect commands for installing libcouchbase on Debian/Ubuntu Created: 10/Oct/14  Updated: 20/Oct/14  Resolved: 13/Oct/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Terry Dhariwal Assignee: Terry Dhariwal
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: N/A

Attachments: PNG File Screenshot 2014-10-10 15.19.56.png    

 Description   
The command:
===========
wget -O- http://packages.couchbase.com/ubuntu/couchbase.key | sudo apt-key add
should be:
wget O- http://packages.couchbase.com/ubuntu/couchbase.key | sudo apt-key add

/etc/apt/sources.list
===============
Also there's a line for creating a new file in /etc/apt/sources.list.d directory. This seems incorrect... I changed a file called /etc/apt/sources.list and that seemed to work.





 Comments   
Comment by Subhashni Balakrishnan [ 10/Oct/14 ]
I believe the alternate way is to add it in a separate file like couchbase.list to /etc/apt/sources.list.d
Comment by Mark Nunberg [ 10/Oct/14 ]
mnunberg@mbp15 ~/Source/libcouchbase $ wget O- http://packages.couchbase.com/ubuntu/couchbase.key
--2014-10-10 20:52:58-- http://o-/
Resolving o-... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address 'o-'


..
Following the instructions from the manual (as I did today) work fine for me (just had to look them up earlier today for something else)
Comment by Matt Ingenthron [ 20/Oct/14 ]
Terry: With Mark's comments here, is it okay?




[CCBC-519] Memory (possibly connection) leak in cbc-pillowfight Created: 02/Oct/14  Updated: 12/Oct/14  Resolved: 08/Oct/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.2
Fix Version/s: 2.4.3
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I can't tell whether this is only pillowfight or the libcouchbase underneath, but running the following command for about an hour shows a continuously growing RAM usage by the cbc-pillowfight process:

cbc-pillowfight -U couchbase://172.23.100.38/default -m 1 -M 1 -t 64 -B 10000 -I 30000 -n -c -1 -r 5 2> /dev/null




[CCBC-518] Provide string ctl to modify retry backoff Created: 26/Sep/14  Updated: 17/Oct/14  Resolved: 12/Oct/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.3
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 08/Oct/14 ]
http://review.couchbase.org/#/c/41979/




[CCBC-517] More information should be printed when a new config is obtained. Created: 26/Sep/14  Updated: 12/Oct/14  Resolved: 12/Oct/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.2
Fix Version/s: 2.4.3
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We should print out the following info:

* RevID
* Why a config was rejected (if it in fact was rejected)

 Comments   
Comment by Mark Nunberg [ 08/Oct/14 ]
http://review.couchbase.org/#/c/41978/1




[CCBC-516] unable yum install libcouchbase-devel.x86_64 Created: 23/Sep/14  Updated: 29/Sep/14  Resolved: 29/Sep/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: kelvin Assignee: Mark Nunberg
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS release 6.3 (Final)


 Description   
ctb8:/opt/slocal/src#1 cat /etc/centos-release
CentOS release 6.3 (Final)
ctb8:/opt/slocal/src#2 yum install libcouchbase2-core.x86_64 libcouchbase-devel.x86_64
Loaded plugins: downloadonly, fastestmirror, security
Loading mirror speeds from cached hostfile
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package libcouchbase-devel.x86_64 0:2.4.1-1.el6 will be installed
---> Package libcouchbase2-core.x86_64 0:2.4.1-1.el6 will be installed
--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: libcouchbase2-core-2.4.1-1.el6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: libcouchbase2-core-2.4.1-1.el6.x86_64
--> Finished Dependency Resolution
Error: Package: libcouchbase2-core-2.4.1-1.el6.x86_64 (couchbase)
           Requires: libssl.so.10(libssl.so.10)(64bit)
Error: Package: libcouchbase2-core-2.4.1-1.el6.x86_64 (couchbase)
           Requires: libcrypto.so.10(libcrypto.so.10)(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
ctb8:/opt/slocal/src#3 rpm -qa | grep openssl
openssl-devel-1.0.0-25.el6_3.1.x86_64
openssl-1.0.0-25.el6_3.1.x86_64
ctb8:/opt/slocal/src#4 ldconfig -v | egrep 'libcrypto.so.10|libssl.so.10'
ldconfig: /etc/ld.so.conf.d/kernel-2.6.32-279.11.1.el6.x86_64.conf:6: duplicate hwcap 1 nosegneg
        libssl.so.10 -> libssl.so.1.0.0
        libcrypto.so.10 -> libcrypto.so.1.0.0


 Comments   
Comment by Mark Nunberg [ 23/Sep/14 ]
This is certainly odd. On my env (CentOS 6.5), I see that the current openssl version is 1.0.1, not 1.0.0:

Installed Packages
Name : openssl
Arch : x86_64
Version : 1.0.1e
Release : 16.el6_5.14
Size : 4.0 M
Repo : installed
From repo : updates
Summary : A general purpose cryptography library with TLS implementation
URL : http://www.openssl.org/
License : OpenSSL
Description : The OpenSSL toolkit provides support for secure communications between
            : machines. OpenSSL includes a certificate management tool and shared
            : libraries which provide various cryptographic algorithms and
            : protocols.

Available Packages
Name : openssl
Arch : i686
Version : 1.0.1e
Release : 16.el6_5.15
Size : 1.5 M
Repo : updates
Summary : A general purpose cryptography library with TLS implementation
URL : http://www.openssl.org/
License : OpenSSL
Description : The OpenSSL toolkit provides support for secure communications between
            : machines. OpenSSL includes a certificate management tool and shared
            : libraries which provide various cryptographic algorithms and
            : protocols.

Name : openssl
Arch : x86_64
Version : 1.0.1e
Release : 16.el6_5.15
Size : 1.5 M
Repo : updates
Summary : A general purpose cryptography library with TLS implementation
URL : http://www.openssl.org/
License : OpenSSL
Description : The OpenSSL toolkit provides support for secure communications between
            : machines. OpenSSL includes a certificate management tool and shared
            : libraries which provide various cryptographic algorithms and
            : protocols.

Comment by Mark Nunberg [ 23/Sep/14 ]
Note that you are running CentOS 6.3. This release is not supported any more by CentOS or Red Hat (see https://access.redhat.com/support/policy/updates/errata/). You can build the package from source or you can upgrade to a newer minor release. Upgrading to a newer CentOS release should be a fairly painless and transparent process.
Comment by kelvin [ 23/Sep/14 ]
Thanks! It could install in CentOS 6.5.




[CCBC-515] lcb_rget3 does not check for missing configuration Created: 17/Sep/14  Updated: 17/Oct/14  Resolved: 19/Sep/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.2
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The library does not check for a missing configuration in the rget3() API call, thus calling this function on a non-bootstrapped client will segfault rather than return with an error.

 Comments   
Comment by Mark Nunberg [ 17/Sep/14 ]
http://review.couchbase.org/#/c/41464/




[CCBC-514] helper headers have different install targets depending on build system Created: 16/Sep/14  Updated: 17/Oct/14  Resolved: 17/Sep/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.2
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The helper headers such as 'bsdio-inl' and friends have different install paths depending on whether autotools or cmake is being used.




[CCBC-513] cmake does not install all files needed Created: 15/Sep/14  Updated: 17/Sep/14  Resolved: 17/Sep/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.1
Fix Version/s: 2.4.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
ERROR: Error installing couchbase: [15/242]
        ERROR: Failed to build gem native extension.

    ~/.rbenv/versions/2.1.0/bin/ruby -r ./siteconf20140915-28269-a46nh0.rb extconf.rb
checking for lcb_set_bootstrap_callback(NULL, NULL) in -lcouchbase... yes
checking for mach/mach_time.h... no
checking for stdint.h... yes
checking for sys/time.h... yes
checking for fcntl.h... yes
checking for sys/socket.h... yes
checking for errno.h... yes
checking for st_index_t... yes
checking for clock_gettime()... yes
checking for gettimeofday()... yes
checking for QueryPerformanceCounter()... no
checking for gethrtime()... yes
checking for rb_hash_lookup2()... yes
checking for rb_thread_fd_select()... yes
checking for rb_thread_blocking_region()... yes
checking for rb_thread_call_without_gvl()... yes
checking for poll() in poll.h... yes
checking for ppoll() in poll.h... yes
checking for rb_fiber_yield()... yes
creating couchbase_config.h
creating Makefile

make "DESTDIR=" clean

make "DESTDIR="
compiling multithread_plugin.c
multithread_plugin.c:30:36: fatal error: libcouchbase/bsdio-inl.c: No such file or directory
 #include <libcouchbase/bsdio-inl.c>
                                    ^
compilation terminated.
Makefile:224: recipe for target 'multithread_plugin.o' failed
make: *** [multithread_plugin.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/arashm/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/couchbase-1.3.9 for inspection.
Results logged to /home/arashm/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0-static/couchbase-1.3.9/gem_make.out


 Comments   
Comment by Mark Nunberg [ 15/Sep/14 ]
http://review.couchbase.org/41421




[CCBC-512] Docs state that getl will prevent all future access Created: 11/Sep/14  Updated: 18/Sep/14  Resolved: 18/Sep/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.1
Fix Version/s: 2.4.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Under this link: http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.0/group___l_c_b___g_e_t.html

The text:
If this parameter is set then the server will in addition to retrieving the item also lock the item, making it so that other operations to access the same item will fail with an error (either LCB_KEY_EEXISTS or LCB_ETMPFAIL). The key will only be accessible again once:
-The lock timeout expires
-The item is unlocked with the cas returned in the response
-The item is modified with the cas returned in the response

Is not correct about "other operations". In fact, other operations (both read and modify) would receive no error. The text could be simply modified to be "other operations with the lock bit set"

 Comments   
Comment by Mark Nunberg [ 11/Sep/14 ]
http://review.couchbase.org/41380




[CCBC-508] hashkey needs to be marked as volatile and clear that it's experimental and libmemcached compat oriented Created: 10/Sep/14  Updated: 16/Sep/14  Resolved: 16/Sep/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.7, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.4.1
Fix Version/s: 2.4.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This isn't described correctly in the API documentation and needs to be.

Please do so in any source code as well so it's not accidentally discovered. This is not a feature of the system currently.

Recommended text for a comment:
/* Note that hashkey/groupid is not a supported feature of Couchbase Server and this client. It should be considered volatile and experimental. Using this could lead to an unbalanced cluster, inability to interoperate with the data from other languages, not being able to use the Couchbase Server UI to look up documents and other possible future upgrade/migration concerns. */

 Comments   
Comment by Mark Nunberg [ 10/Sep/14 ]
http://review.couchbase.org/#/c/41317/




[CCBC-507] Add test case for empty config Created: 08/Sep/14  Updated: 08/Sep/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Server 3.0.0 sometimes returns an empty config. We should test for this and ensure the parse fails rather than crashing the client.

 Comments   
Comment by Mark Nunberg [ 08/Sep/14 ]
See MB-12104. I think this wouldn't be an issue in the library as the parsing would fail and the library would attempt the next node/provider. Still useful to have a test for this, though.




[CCBC-506] Lower the default CONFDELAY threshold Created: 02/Sep/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0, 2.4.1, 2.4.2
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The default setting for this is now 10 seconds. This means that it may take up to this time to refetch a new configuration, causing outages in such use environments when such a delay is unacceptable. While this number is certainly adjustable there seems to be some confusion and difficulty in conveying exactly what this number means. Perhaps a lower default value is ok for this as well?

To recap, this interval specifies the throttle period during which a client instance will not request a new configuration. This is the interval from the _last_ received configuration, and basically helps us avoid requesting too many config updates in a short timespan.

I am thinking about something like 3 seconds for this?

 Comments   
Comment by Mark Nunberg [ 02/Sep/14 ]
Adding matt to watchers, as maybe he can contribute his thoughts
Comment by Subhashni Balakrishnan [ 15/Oct/14 ]
can we try to do some test runs with different values and decide based on that - but this only impacts in the cases where we don't get a config on NMV




[CCBC-505] Allow user-side modification of node's statuses Created: 02/Sep/14  Updated: 04/Sep/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to JCBC-539 Consider a way to expose target key n... Open

 Description   
This feature will allow users to get/set the node statuses in cases where existing application infrastructure can be better than the existing built in cluster configuration info with respect to determining if nodes are down , unavailable etc.

This helps increase responsiveness in applications when a node is "known" to be down, while also not forcing any possibly library-side heuristic upon the user.

 Comments   
Comment by Mark Nunberg [ 04/Sep/14 ]
There are actually two ways to implement this. One is at the network level where we can forcefully close a socket, one is at the vbucket level where we can simply set all the indices to -1 where the node's real index would be instead.




[CCBC-504] cbc hash does not print the replica information any more Created: 02/Sep/14  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0, 2.4.1
Fix Version/s: 2.4.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: supportability
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Before 2.4.0:

"patick" Server:"192.168.61.102:11210" vBucket:345 CouchAPI:"http://192.168.61.102:8092/TEST" Replicas:"192.168.61.101:11210"

2.4.0 and onwards

patick: [vBucket=345, Index=1] Server: 192.168.61.102:11210, CouchAPI: http://192.168.61.102:8092/TEST


The support team uses 'cbc hash' to debug issues. Please add back the replica information.

 Comments   
Comment by Mark Nunberg [ 02/Sep/14 ]
http://review.couchbase.org/#/c/41148/




[CCBC-503] need better documentation on how to use error classifiers Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.0, 2.4.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Documentation has a section on error handling, but there's nothing in the documentation other than a release note on error classifiers.

Please add some detail on what these are and how they're intended to be used.

 Comments   
Comment by Mark Nunberg [ 28/Aug/14 ]
http://docs.couchbase.com/couchbase-sdk-c-2.4/index.html#error-handling-and-diagnostics gives a pretty detailed explanation. Do you mean to add an additional section within the API docs, or something else?




[CCBC-502] Provide API to retrieve bucket name Created: 27/Aug/14  Updated: 17/Oct/14  Resolved: 03/Sep/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.2
Security Level: Public

Type: New Feature Priority: Minor
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This should be a simple cntl to retrieve the name of the bucket as a string :)




[CCBC-501] Hide no_verify SSL option Created: 25/Aug/14  Updated: 17/Oct/14  Resolved: 26/Aug/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Hide the ssl=no_verify option and rename the capath option to cert_path, as currently the server does not support third-party-signed certificates, but a simple self-signed certificate




[CCBC-500] ConnSpec parser fails to understand a string with a non-existent scheme. Created: 20/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Task Priority: Major
Reporter: Brett Lawson Assignee: Brett Lawson
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When passing a string that is simply an IP address (ie: `192.168.7.26:8091`), libcouchbase fails with a 'Invalid arguments' error.

 Comments   
Comment by Mark Nunberg [ 20/Aug/14 ]
this is by design. to use a raw address, the v2 structure should be used instead
Comment by Brett Lawson [ 20/Aug/14 ]
A raw address/port combination is a valid URI and a valid connection string, http scheme should be used by default if one is not provided. This is necessary for backwards compatibility and does not have any known side effects.
Comment by Mark Nunberg [ 20/Aug/14 ]
The heuristics here are a bit more complex, and we shouldn't be using HTTP bootstrap by default unless there is good reason to do so, thus for example a traditional 'simple IP' address should end up being CCCP, but if a port is provided, would be HTTP, etc.

In my opinion, if dealing with a higher level SDK, the branching should be done at the SDK level rather than within the client library itself (this way, the SDK has a chance to print out a warning about how using a scheme-less syntax is deprecated, etc). This would work by having the SDK determine if the string starts with a valid scheme, and if it does, use the 'v3' options. If it does not, use the 'v1' options
Comment by Brett Lawson [ 20/Aug/14 ]
Whilst I agree with you, the spec specifically defines a host/port combination as being valid for other reasons (such as easily supporting user upgrades). Breaking spec for a single SDK, especially since it's such a simple change (literally code was added to make this specifically not work) does not make sense.

P.S. Using couchbase:// by default does not permit backwards compatibility with pre-2.5 servers (or it should not be at least, again, for reasons I won't list here; some are mentioned in the spec). Thus defaulting to this on a 'feature' thats sole purpose is supporting lazy initial upgrades also makes no sense.




[CCBC-499] cbc pillowfight now longer working Created: 18/Aug/14  Updated: 20/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Nakomis Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: pillowfight
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Ubuntu 12.04.5 LTS (GNU/Linux 3.2.0-67-virtual i686) on Softlayer VM


 Description   
Have installed libcouchbase, following instructions here [1]. When I run `cbc pillowfight` I get the following error:

Unknown command pillowfight
Usage: cbc <command> [options]
command may be:
   help Show help
<......>

If I run `cbc pillowfight -h 127.0.0.1:8091 -b default -i 1000 -I 1000 -t 1 -Q 1 -s 0 -m 50 -M 5120` I get the error:
Unknown command pillowfight
Couldn't parse options: Unknown option
No such option: i
Usage:
  pillowfight [OPTIONS...]

   -P --password Bucket password [Default='']
   -u --username Username (currently unused) [Default='']
<....>

`cbc version` gives the following:
cbc:
  Runtime: Version=2.4.0, Changeset=3ad936575b1718a748e5dc6ff73dfe801a201b15
  Headers: Version=2.4.0, Changeset=3ad936575b1718a748e5dc6ff73dfe801a201b15
  IO: Default=libevent, Current=libevent
  Compression (snappy): .. NOT SUPPORTED
  SSL: .. SUPPORTED

This was previously working

Please let me know if you require any further information

Cheers

Martin


[1]: http://www.couchbase.com/communities/c-client-library

 Comments   
Comment by Mark Nunberg [ 18/Aug/14 ]
http://review.couchbase.org/#/c/40189/
Comment by Nakomis [ 20/Aug/14 ]
Thanks, I hadn't realized `cbc pillowfight` had been replaced by `cbc-pillowfight`. Using the new cbc-pillowfight, is it possible to run it against a remote machine? If I try `cbc-pillowfight -h <ip of remote server>:8091 -b default -i 1000 -I 1000 -t 1 -Q 1 -s 0 -m 50 -M 5120` I get the error "The -h/--host option is deprecated. Use connection string instead", however, I can't find a way to specify the connection string

Thanks
Comment by Mark Nunberg [ 20/Aug/14 ]
Use the -U option. More information on the connection string can be found here: http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.0/group___l_c_b___l_c_b_t.html#structlcb__create__st3




[CCBC-498] Config cache only gets updated every 10 seconds Created: 12/Aug/14  Updated: 12/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.2.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Patrick Varley Assignee: Patrick Varley
Resolution: Unresolved Votes: 0
Labels: config-cache, customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I believe that the config cache is only updated every 10seconds. I believe we advise users to use the config cache in cases where connections are being created and destroyed all the time such as apache and nginx setups, these also happen to be environments where the process is short lived.

* Is the 10 second timer useful?
* In these environments with cccp should we be using the config cache?


 Comments   
Comment by Mark Nunberg [ 12/Aug/14 ]
The config cache is updated whenever a client instance needs to fetch a new config; in most cases a new config fetch is throttled so that a request does not take place more than once every ten seconds.

The 10 second timer is specifically removed for config cache during the initial configuration, thereby making the case of the apache/nginx concerns moot. If you can please elaborate on some observations you've seen, I can perhaps explain more

There is a distinct advantage for using the config cache even in places where CCCP is enabled, since CCCP still incurs an additional latency overhead in getting the initial config as well as the chance that the node which we connected to for CCCP is not the same node which is the target for the single K/V op in the common use case. The original intent of the config cache was to allow quick bootstrap - and in this case it's always useful. For longer lived applications which are using the config cache to circumvent the connection limits, I would say that it should not be used in conjunction with CCCP




[CCBC-497] Segfaults when handling network errors during rebalance Created: 11/Aug/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Task Priority: Major
Reporter: Subhashni Balakrishnan Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When handling network errors on view requests, lcb segfaults

*** glibc detected *** ./sdkd_lcb: free(): invalid pointer: 0x00007fe800000001 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x760e6)[0x7fe85bea30e6]
/root/sources/libcouchbase/inst/lib/libcouchbase.so.2(+0xcc98)[0x7fe85c96cc98]
/root/sources/libcouchbase/inst/lib/libcouchbase.so.2(+0xd101)[0x7fe85c96d101]
#4 0x00007ffd09435c98 in request_free_headers (req=0x7ffc900186f0) at src/http/http.c:36
free(*cur);

 Comments   
Comment by Mark Nunberg [ 12/Aug/14 ]
http://review.couchbase.org/#/c/40542/

@subhashni, can you see if this fixes the issue?
Comment by Subhashni Balakrishnan [ 12/Aug/14 ]
Hey Mark it fixes the issue..I've tried this fix before.. but would be nice to figure out why the double free
Comment by Mark Nunberg [ 12/Aug/14 ]
The double free takes place because it calls lcb_http_exec_request (or similar) multiple times on the same http_request_t if the request is retried :)




[CCBC-496] Abstract openssl use to dynamically loaded plugin Created: 07/Aug/14  Updated: 07/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If an application is using a different version of openssl, the library's version may conflict with it. The solution is to use dynamic loading of the plugin at runtime so that the application's version remains in-tact.




[CCBC-495] Library does not consider hostname and IP of the same server as being the same node Created: 06/Aug/14  Updated: 06/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.0, 2.4.0
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Minor
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Connecting to "localhost" will not reuse the connection if a node named "127.0.0.1" appears in the config. Unfortunately fixing this would mean determining how to handle hostnames with different resolving IPs, and similar.

 Comments   
Comment by Mark Nunberg [ 06/Aug/14 ]
In practice this is not a big issue since at most this causes an extra connection to be wasted during initial bootstrap, but may end up causing some confusion during initial log analysis.




[CCBC-494]  LCB documentation here http://www.couchbase.com/communities/c-client-library is not updated Created: 05/Aug/14  Updated: 07/Aug/14  Resolved: 07/Aug/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: 2.4.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Venu Uppalapati Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
 LCB documentation here http://www.couchbase.com/communities/c-client-library is not updated
for example, using lcb_set_error_callback with LCB2.4 throws a warning at compile time but the examples in the docs still use it.

 Comments   
Comment by Mark Nunberg [ 07/Aug/14 ]
Simply linked to the API docs in this case. Those are more extensive




[CCBC-493] HELLO is still emitted by client when connecting Created: 05/Aug/14  Updated: 05/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: None
Security Level: Public

Type: Task Priority: Minor
Reporter: Brett Lawson Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Based on the conversation that I recall, HELLO was supposed to be disabled client-side for all releases until such time that server support properly implements it (due to it being subject to change). libcouchbase appears to emit a HELLO request when connecting to a node.

 Comments   
Comment by Mark Nunberg [ 05/Aug/14 ]
It might be a bit of an annoyance, but a well behaved server should respond with NOT_SUPPORTED or UNKNOWN_COMMAND. in this case the server replies perfectly well here. Can you foresee a situation where this would be problematic?
Comment by Brett Lawson [ 05/Aug/14 ]
We had decided to remove this from the clients due to the fact that the server may not disable this immediately, but the feature does not yet work 100%. Additionally, when the feature is implemented, it is entirely possible that it will work in a completely incompatible way to what is currently built, or that opcode may be reused for another purpose.
Comment by Mark Nunberg [ 05/Aug/14 ]
http://review.couchbase.org/#/c/40306/




[CCBC-492] API ref docs man page for cbc refers to cbcrc man page, which is not available there Created: 01/Aug/14  Updated: 04/Aug/14  Resolved: 04/Aug/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 01/Aug/14 ]
http://review.couchbase.org/#/c/40190/




[CCBC-491] Provide lcb_dump() to print out debugging information about current client state Created: 31/Jul/14  Updated: 31/Jul/14  Resolved: 31/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.1
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This will provide an lcb_dump() to print out various debug information about the library which may then be used to help diagnose issues.




[CCBC-490] get-with-replica 'FIRST' mode will not actually try all replicas Created: 30/Jul/14  Updated: 30/Jul/14  Resolved: 30/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The LCB_REPLICA_FIRST will actually only respond with the first replica that has been received, since the custom rget callback (in get.c) is never invoked. See CCBC-489 for another symptom of this bug




[CCBC-489] Get-with-replica will leak memory Created: 30/Jul/14  Updated: 30/Jul/14  Resolved: 30/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Each call to get-with-replica will leak memory because the packet handler does not properly invoke the customized callback, rather invoking the user callback directly.




[CCBC-488] get with replica should return immediate failure if the bucket has no replicas Created: 30/Jul/14  Updated: 30/Jul/14  Resolved: 30/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 30/Jul/14 ]
This issue is actually caused by a bug in the lcbvb_vbreplica() function which has a bad check for determining if the input index is valid. This only affects get-with-replica using the index parameter, as in all other cases the index is already bound-checked by other mechanisms.
Comment by Brett Lawson [ 30/Jul/14 ]
This also affects OBSERVE requests completing.
Comment by Brett Lawson [ 30/Jul/14 ]
In hindsight, the bug accidentally makes most cases work, unless you try to observe/getReplica for a replica higher than you have configured.




[CCBC-487] Provide API to control memory usage of the client Created: 30/Jul/14  Updated: 04/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The library reserves a sane default of memory for various buffers and such, especially in regards to pooling and the like. Since many of these buffers are per-connection, it may result in high memory usage when many connections and/or instances are used.

The idea here is to provide a "Factor" of memory to use rather than an absolute number because of the high number of settings that may possibly be adjusted within the library regarding memory usage; thus a factor of 1 (the default) will use the default settings, a factor of 0.5 would attempt to reserve only half that amount of space, and so on.




[CCBC-486] cbc flush command missing Created: 30/Jul/14  Updated: 30/Jul/14  Resolved: 30/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.4.0
Fix Version/s: 2.4.1
Security Level: Public

Type: Task Priority: Minor
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This will be re-added as 'mcflush' instead, to avoid confusion between this command and the bucket-flush (aka "RESTful flush")

 Comments   
Comment by Mark Nunberg [ 30/Jul/14 ]
http://review.couchbase.org/#/c/40064/




[CCBC-485] unable to set ssl and ca_path using lcb_cntl Created: 30/Jul/14  Updated: 04/Aug/14  Resolved: 30/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Subhashni Balakrishnan Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.0-beta/group___l_c_b_t___i_n_f_o.html#gab3df573dbbea79cfa8ce77f6f61563dc

says it is possible to set the values using lcb_cntl but those are read only args.

 Comments   
Comment by Mark Nunberg [ 30/Jul/14 ]
You need to set ca_path via the connstr in the creation options
Comment by Subhashni Balakrishnan [ 30/Jul/14 ]
Yes but the doc says both modes get, set maybe fix doc? http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.0-beta/group___l_c_b___c_n_t_l.html#gaacfed9463423501e2623290ac68ce390
Comment by Mark Nunberg [ 30/Jul/14 ]
This is fixed in the release docs :) http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.0/group___l_c_b___c_n_t_l.html#gaacfed9463423501e2623290ac68ce390
Comment by Subhashni Balakrishnan [ 30/Jul/14 ]
ah.. my bad
Comment by Mark Nunberg [ 30/Jul/14 ]
http://review.couchbase.org/#/c/40055/




[CCBC-484] Override I/O plugin functionality with default I/O impl for send/recv functions Created: 29/Jul/14  Updated: 29/Jul/14  Resolved: 29/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1, 2.4.0-beta
Fix Version/s: 2.4.0
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Because some existing SDKs (like Ruby) make assertions about LCB's behavior which no longer hold true for 2.4, we need to patch the I/O functions at runtime, or risk breaking those dependent SDKs.

See https://github.com/couchbase/couchbase-ruby-client/blob/master/ext/couchbase_ext/plugin_common.c#L56 for an example




[CCBC-483] Ignore CCCP config updates from not-my-vbucket responses if CCCP is disabled Created: 28/Jul/14  Updated: 29/Jul/14  Resolved: 29/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.2
Fix Version/s: 2.4.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If the CCCP provider is explicitly disabled, don't update from not-my-vbucket responses. This allows various configuration proxies (functioning over HTTP) to work properly

 Comments   
Comment by Mark Nunberg [ 28/Jul/14 ]
No good way to test this for now




[CCBC-482] Don't set "Last refresh" time for initial file config Created: 28/Jul/14  Updated: 28/Jul/14  Resolved: 28/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to

 Description   
When the client is initially bootstrapped from a file-based cache, it is possible that the cache might be stale; if so, don't set the initial timestamp for the config (which is used as the base to determine whether to throttle subsequent config requests). This allows the client to quickly refetch the configuration if the existing one is stale.




[CCBC-481] Use retry queue signalling instead of deadlines. Created: 25/Jul/14  Updated: 25/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Minor
Reporter: Brett Lawson Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, operations that need to be rescheduled for retry or waiting-on-config situations enter the retry queue and then are retried at a later date based on a timeout period. In many cases, this is inefficient and those operations should instead be signalled by external events occurring (such as a new config being received). In these cases, the retry queue should only be deadlined on the operation timeout, rather than an arbitrary retry interval. This change should additionally help make the handling of operation retries more deterministic.




[CCBC-480] Don't fail immediately for vbuckets which lack a master node Created: 24/Jul/14  Updated: 25/Jul/14  Resolved: 25/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-beta
Fix Version/s: 2.4.0
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Items whose master vbucket index contains a -1 should not be failed immediately to the user, since in this case a new configuration is never received. The proper behavior is to forward the item to a random node once and receive a not-my-vbucket with the proper configuration.

 Comments   
Comment by Mark Nunberg [ 25/Jul/14 ]
This has been addressed in a different manner, by scheduling the item immediately to the retry queue, but without actually forwarding it to a 'random' node. This approach eliminates the excessive network I/O in this case




3.0 API Changes (CCBC-462)

[CCBC-479] Use built-in C89 int/short/long/etc where possible Created: 16/Jul/14  Updated: 16/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There's no good reason to use fixed width types for simple functions that aren't expected to have a specified value. int/unsigned will be sufficient. Leave the fixed width types for things that actually expect large values.




[CCBC-478] setting large design doc terminates the program Created: 16/Jul/14  Updated: 16/Jul/14  Resolved: 16/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Bug Priority: Major
Reporter: sjoerd@weett.nl Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Node.js


 Description   
I have a design document which plans with dates. In CouchDB I used to import the libraries 'moment.js' and 'later.js'. When I upload the same design document to CouchBase, the nodejs program fails with this message:

FATAL ERROR:
    libcouchbase experienced an unrecoverable error and terminates the program
    to avoid undefined behavior.
    The program should have generated a "corefile" which may used
    to gather more information about the problem.
    If your system doesn't create "corefiles" I can tell you that the
    assertion failed in ../deps/lcb/src/http.c at line 268
16 Jul 09:38:40 - [nodemon] app crashed - waiting for file changes before starting...

Removing one library (reducing the code size of the design document) resolves the issue. 25kb of code works, 50kb of js code fails

 Comments   
Comment by Mark Nunberg [ 16/Jul/14 ]
The problem is that the code never calls ringbuffer_ensure_capacity before appending the extra bytes. Since normally the ringbuffer preallocates at minimum around 4k we never notice this with smaller body sizes; but with larger body sizes it seems to show itself.

The newer version (2.4.x) should not suffer from this problem since it does not use the ringbuffer from this purpose but rather lcb_string, whose allocation/resizing is done internally upon appending.
Comment by sjoerd@weett.nl [ 16/Jul/14 ]
I installed the dev version of couchbase node which uses the updated libocuchbase and can confim you are right!
Thanks for the awesome job on CouchBase!




[CCBC-477] Configuration provider may erroneously revert to HTTP Created: 14/Jul/14  Updated: 18/Jul/14  Resolved: 18/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1, 2.4.0-beta
Fix Version/s: 2.4.0
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The configuration provider may erroneously revert to CCCP in cases where the client sleeps in between operations. In this case the timer to request a new configuration expires during the sleep interval and the library then assumes that the provider has failed.

This is also caused by the CCCP provider not cancelling its timeout when a new configuration has been received, but only when the actual refresh has been paused.




[CCBC-476] Create EL7 repositories Created: 12/Jul/14  Updated: 17/Jul/14  Resolved: 17/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.0
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 15/Jul/14 ]
I've managed to build the RPMs. however I need a system to test this on...




[CCBC-475] Better error messag when failing to connect using SSL Created: 11/Jul/14  Updated: 18/Jul/14  Resolved: 18/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 3.0 Cluster
SSL


 Description   
When I try to connect to the cluster with .pem file that did not exist the error message suggested it was a network issue:

[patrick:~/Code/C/libcouchbase] master* 1 ± LCB_SSL_MODE=1 LCB_SSL_CACERT=bad.pem ~/Code/Python/set.py
Traceback (most recent call last):
  File "/Users/patrick/Code/Python/get-loop.py", line 5, in <module>
    cb = Couchbase.connect(bucket='beer-sample', host='192.168.71.101')
  File "/Library/Python/2.7/site-packages/couchbase/__init__.py", line 219, in connect
    **kwargs)
  File "/Library/Python/2.7/site-packages/couchbase/connection.py", line 153, in __init__
    self._do_ctor_connect()
  File "/Library/Python/2.7/site-packages/couchbase/connection.py", line 163, in _do_ctor_connect
    self._connect()
couchbase.exceptions._NetworkError_0x10 (generated, catch NetworkError): <RC=0x10[Network failure], There was a problem while trying to send/receive your request over the network. This may be a result of a bad network or a misconfigured client or server., C Source=(src/connection.c,878)>

It would be good if the error message was better.

get-loop.py

#!/usr/bin/python
import time
from couchbase import Couchbase

cb = Couchbase.connect(bucket='beer-sample', host='192.168.71.101')
while True:
    result = cb.get('new_holland_brewing_company-sundog')
    print result
    time.sleep(1)

 Comments   
Comment by Mark Nunberg [ 11/Jul/14 ]
http://review.couchbase.org/#/c/39320/




[CCBC-474] Failure to schedule commands may leak memory Created: 10/Jul/14  Updated: 31/Jul/14  Resolved: 31/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.1
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Failure to schedule some commands that allocate internal cookie structures may leak memory as the cookie structure is never properly freed

 Comments   
Comment by Mark Nunberg [ 31/Jul/14 ]
http://review.couchbase.org/#/c/40128/




[CCBC-473] cbc utility is not listed on man page index Created: 07/Jul/14  Updated: 13/Oct/14  Resolved: 13/Oct/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The cbc utility man page is not on the index so it's possible to get the man page only through installation.

 Comments   
Comment by Matt Ingenthron [ 07/Jul/14 ]
I could not find a .next to put the fix version on, please set that up.
Comment by Mark Nunberg [ 07/Jul/14 ]
Which index is this? - The new Doxygen pages will contain references to the relevant manpages on the side bar; so I guess this can be considered fixed in 2.4?
Comment by Matt Ingenthron [ 07/Jul/14 ]
The API reference off of the docs site. Given that it exists, it takes 2 minutes, we've somehow skipped this API reference publishing step for 2.3.1 and 2.3.2, it'd be really nice to fix it one-off for 2.3.2.
Comment by Mark Nunberg [ 07/Jul/14 ]
http://review.couchbase.org/#/c/39180/




[CCBC-472] server_nodes returns only nodes specified via "host" argument Created: 03/Jul/14  Updated: 09/Jul/14  Resolved: 09/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Bug Priority: Major
Reporter: Pavel Paulau Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Python 2.7.6
lcb 2.3.2
couchbase 1.2.2


 Description   
https://gist.github.com/pavel-paulau/5f02459f73ee207a18aa

I'm pretty sure it's used to work. And it obviously has nothing to do with python sdk...

 Comments   
Comment by Mark Nunberg [ 03/Jul/14 ]
This is going to be fixed in libcouchbase 2.4. Unfortunately the way 'nodes' was implemented was rather confusing and would be dependent on HTTP bootstrap (which we only do selectively)
Comment by Pavel Paulau [ 03/Jul/14 ]
Thanks for clarification.

Two questions:
1. When do you plan to release 2.4
2. Is there any temporary workaround? I need list of servers for view queries via 3rd party HTTP library...
Comment by Mark Nunberg [ 07/Jul/14 ]
2.4 should have a beta out sometime this week which should hopefully fix this issue.

As for a temporary workaround - I'll see what I can come up with. Are you OK with a patch? :)
Comment by Pavel Paulau [ 07/Jul/14 ]
I temporary workaround-ed the problem but look forward to permanent fix.
Comment by Mark Nunberg [ 07/Jul/14 ]
http://review.couchbase.org/#/c/39158/




[CCBC-471] Retried HTTP request may hang client Created: 06/Jul/14  Updated: 09/Jul/14  Resolved: 09/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If a request is retried as a result of an HTTP redirect, it will increment the pendops counter (which indicates how many pending operations remain until the event loop should stop and control be returned back to the user); however the counter is decremented only once -- when the request finishes.

 Comments   
Comment by Mark Nunberg [ 06/Jul/14 ]
http://review.couchbase.org/39143




[CCBC-470] Publish docs for C SDK July 2014 release Created: 01/Jul/14  Updated: 03/Jul/14  Resolved: 03/Jul/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.2
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Major
Reporter: Amy Kurtzman Assignee: Amy Kurtzman
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





3.0 API Changes (CCBC-462)

[CCBC-469] Change all type names to lcb_TYPENAME rather than lcb_typename_t Created: 01/Jul/14  Updated: 01/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The newer convention is done in most of the 2.4 codebase. Using this convention makes it easier to distinguish types inside source code and also allows the names to be more conformant to POSIX standards which technically prohibits the _t suffix from being used.

We can either remove the old names completely, or place them in deprecated.h.

Probably the most notable difference would be in the lcb_t -- which would need to be renamed to something like lcb_pBUCKET




3.0 API Changes (CCBC-462)

[CCBC-468] 3.0: Remove lcb_timer public API Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The timer API was never really used and should have always been private (its use came in before we started having 'interface attributes' within the library)




3.0 API Changes (CCBC-462)

[CCBC-467] 3.0: Remove 'sanity check' and 'struct IDs' Created: 28/Jun/14  Updated: 07/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 3.0-beta
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
These functions have not been widely used or maintained. Their purpose was to assist applications in verifying the structure sizes used by the library conformed to that of what their application was expecting. However in reality the structure sizes rarely changed, and when they did change, they only changed in compatible ways so that applications compiled against older versions would never break anyway.

Applications using newer versions of the structures would also never work anyway since the library itself would reject the newer versions at runtime.




3.0 API Changes (CCBC-462)

[CCBC-466] 3.0: Remove lcb_get_last_error Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This function is deprecated and its use can result in false positives, true negatives. Most internals do not set 'last_error', and because there may be multiple things going on within the library, getting the last error does not make sense. Only operation specific error fields should be used.




3.0 API Changes (CCBC-462)

[CCBC-465] 3.0: Remove lcb_error_callback Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This API call has no use other than indicating bootstrap success/failure, and has been deprecated in favor of lcb_bootstrap_callback in 2.4




3.0 API Changes (CCBC-462)

[CCBC-464] 3.0: Error code type should be lcb_STATUS, not lcb_error_t Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In conformance with standards and more consistent names, we should call it lcb_STATUS rather than error; also considering that some codes are not necessarily errors :)




3.0 API Changes (CCBC-462)

[CCBC-463] 3.0: Remove syncmode Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 3.0-beta
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-462] 3.0 API Changes Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Sub-Tasks:
Key
Summary
Type
Status
Assignee
CCBC-463 3.0: Remove syncmode Technical task Open Mark Nunberg  
CCBC-464 3.0: Error code type should be lcb_ST... Technical task Open Mark Nunberg  
CCBC-465 3.0: Remove lcb_error_callback Technical task Open Mark Nunberg  
CCBC-466 3.0: Remove lcb_get_last_error Technical task Open Mark Nunberg  
CCBC-467 3.0: Remove 'sanity check' and 'struc... Technical task Open Mark Nunberg  
CCBC-468 3.0: Remove lcb_timer public API Technical task Open Mark Nunberg  
CCBC-469 Change all type names to lcb_TYPENAME... Technical task Open Mark Nunberg  
CCBC-479 Use built-in C89 int/short/long/etc w... Technical task Open Mark Nunberg  

 Description   
This is an umbrella task for the various changes which will take place in the 3.0 API. Some of these changes may be introduced as volatile within the 2.4 series, while some involve explicitly removing older APIs.

As a secondary goal of the 3.0 API, we hope to make it mostly API and ABI compatible with the 2.0 series, so that application changes can be gradual and minimal.




[CCBC-461] Unmask NOT_MY_VBUCKET return codes. Created: 26/Jun/14  Updated: 28/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.0
Fix Version/s: 3.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
NOT_MY_VBUCKET error codes should be unmasked in the library and propagated to the user if we were unable to retry the operation for whatever reason. While typically the library _will_ retry the operation, masking this error under a different code makes it more difficult for the application to handle.

Historically we have mapped this error to 'LCB_ETIMEDOUT' which isn't always accurate and makes our timeout mechanism seem buggy (an error called "Timeout" should only be thrown after a specific amount of time has elapsed).

The alternative, LCB_NO_MATCHING_SERVER is also confusing as this is typically the return code _before_ an operation is scheduled - while a NOT_MY_VBUCKET is a definite reply from the server. While in raw C applications it may be possible to treat a NO_MATCHING_SERVER from an operation return value differently than one invoked from within a callback, for most common SDKs there is no distinction - and these two paths require different handling paths.




[CCBC-460] phrasing on the getting started page is odd Created: 26/Jun/14  Updated: 26/Jun/14  Resolved: 26/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
On our current page, it has this section:

We are building windows versions of libcouchbase with two different MS runtimes: 9th (Visual Studio 2008), 10th (Visual Studio 2010) and 11th (Visual Studio 2012) versions, to simplify linking for your application. Just choose the appropriate zip archive:

That's phrased a bit odd. Please fix.

 Comments   
Comment by Mark Nunberg [ 26/Jun/14 ]
Windows

Archives for Windows are included. Bundled in the archive are the header files (inside include), the dynamic library (bin/libcouchbase.dll), the import library (lib/libcouchbase.lib) and the debug variants (bin/libcouchbase_d.dll and lib/libcouchbase_d.lib). The cbc utility is included as well.

If you wish to link against libcouchbase from a Windows application, you must select the archive appropriate for the version of Visual Studio being used to compile your application.




[CCBC-459] Provide error category indicating server is loaded Created: 26/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1, 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We should make a new error classifier (LCB_ERRTYPE_SRVLOAD) and LCB_EIFSRVLOAD to indicate that the error reply would be solved by a backoff.

Currently we have EIFTMP which includes these errors, but also includes some errors which are not strictly related to having a backoff interval.




[CCBC-458] Allow to disable refresh-on-views-error Created: 25/Jun/14  Updated: 10/Jul/14  Resolved: 10/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0, 2.3.1, 2.3.2
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently when an HTTP error is received we automatically take this as a cue to refresh the configuration. While the side effects are generally harmless (because of throttling) they are troublesome to analyze in the logs and are a waste of network resources. Since this behavior exists primarily for views-only workload we should look into adding an option to disable this behavior and/or disable it by default.

We may also possibly implement the proper "Heuristics Checking" specified in a proposal somewhere too..

 Comments   
Comment by Mark Nunberg [ 10/Jul/14 ]
http://review.couchbase.org/39288




[CCBC-457] pillowfight fails to connect to non-standard port Created: 24/Jun/14  Updated: 24/Jun/14  Resolved: 24/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Bug Priority: Major
Reporter: Dave Rigby Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: OS X Mavericks, master branch (4d6a266)


 Description   
Running pillowfitght and attempting to connect to a cluster running on port 9000 fails. This works fine with the release23 branch (8e337a8).

I see the following in the logs when attempting to connect:

$ LCB_LOGLEVEL=5 bin/cbc-pillowfight -h localhost:9000
    Creating instance 00ms [I0] {1287} [INFO] (instance - L:451) Version=2.4.0-dp1-9-g13916c5, Changeset=13916c5e25b63bc2b54324ca7817d1e29d47d1b7
    0ms [I0] {1287} [INFO] (instance - L:452) Effective connection string: couchbase://localhost:9000/default?. Bucket=default
    0ms [I0] {1287} [DEBUG] (instance - L:189) Adding host localhost:9000 to initial CCCP bootstrap list
    0ms [I0] {1287} [DEBUG] (confmon - L:94) Provider FILE is DISABLED
    0ms [I0] {1287} [DEBUG] (confmon - L:91) Provider CCCP is ENABLED
    0ms [I0] {1287} [DEBUG] (confmon - L:91) Provider HTTP is ENABLED
    0ms [I0] {1287} [TRACE] (confmon - L:263) Start refresh requested
    0ms [I0] {1287} [TRACE] (confmon - L:243) Current provider is CCCP
    0ms [I0] {1287} [DEBUG] (lcbio_mgr - L:383) <localhost:9000> (HE=0x7fc369008e00) Creating new connection because none are available in the pool
    0ms [I0] {1287} [DEBUG] (lcbio_mgr - L:301) <localhost:9000> (HE=0x7fc369008e00) Starting connection on I=0x7fc368604db0
    0ms [I0] {1287} [INFO] (connection - L:423) <localhost:9000> (SOCK=0x7fc368604ed0) Starting. Timeout=2000000us
    3ms [I0] {1287} [TRACE] (connection - L:241) <localhost:9000> (SOCK=0x7fc368604ed0) Got event handler for new connection
    3ms [I0] {1287} [TRACE] (connection - L:288) <localhost:9000> (SOCK=0x7fc368604ed0) Scheduling asynchronous watch for socket.
    3ms [I0] {1287} [TRACE] (connection - L:241) <localhost:9000> (SOCK=0x7fc368604ed0) Got event handler for new connection
    3ms [I0] {1287} [INFO] (connection - L:99) <localhost:9000> (SOCK=0x7fc368604ed0) Connected
    3ms [I0] {1287} [DEBUG] (lcbio_mgr - L:255) <localhost:9000> (HE=0x7fc369008e00) Received result for I=0x7fc368604db0,C=0x7fc368604ed0; E=0x0
    3ms [I0] {1287} [DEBUG] (lcbio_mgr - L:206) <localhost:9000> (HE=0x7fc369008e00) Assigning R=0x7fc368604be0 SOCKET=0x7fc368604ed0
    3ms [I0] {1287} [DEBUG] (ioctx - L:75) <localhost:9000> (CTX=0x7fc36a100700,unknown) Pairing with SOCK=0x7fc368604ed0
    3ms [I0] {1287} [ERROR] (negotiation - L:162) <localhost:9000> (SASLREQ=0x7fc36a100000) Error: 0x10, IO Error
    3ms [I0] {1287} [ERROR] (cccp - L:130) <NOHOST:NOPORT> Got I/O Error=0x10
    3ms [I0] {1287} [INFO] (confmon - L:190) Provider 'CCCP' failed
    3ms [I0] {1287} [DEBUG] (ioctx - L:125) <localhost:9000> (CTX=0x7fc36a100700,sasl) Destroying. PND=0,ENT=0,SORC=1
    104ms [I0] {1287} [TRACE] (confmon - L:243) Current provider is HTTP
    104ms [I0] {1287} [TRACE] (htconfig - L:393) Starting HTTP Configuration Provider 0x7fc369003400
    104ms [I0] {1287} [ERROR] (htconfig - L:397) Not scheduling HTTP provider since no nodes have been configured for HTTP bootstrap
    104ms [I0] {1287} [INFO] (confmon - L:190) Provider 'HTTP' failed
    104ms [I0] {1287} [TRACE] (confmon - L:204) Maximum provider reached. Resetting index
    104ms [I0] {1287} [ERROR] (bootstrap - L:96) Failed to bootstrap client=0x7fc368603ab0. Code=0xa, Message=No more bootstrap providers remain

    Failed to connect: Error while establishing TCP connection


For comparison, on release23 I see the following (working):

    $ LCB_LOGLEVEL=5 bin/cbc-pillowfight -h localhost:9000
    Creating instance 00ms [I0] [TRACE] (confmon - L:56) Creating monitor...
    0ms [I0] [DEBUG] (confmon - L:91) Have 2 providers enabled
    0ms [I0] [TRACE] (confmon - L:253) Start refresh requested
    0ms [I0] [TRACE] (confmon - L:229) Current provider is 2
    0ms [I0] [DEBUG] (connmgr - L:262) Got request R=0x7fab53830400,localhost:11210
    0ms [I0] [INFO] (connmgr - L:221) Starting connection on I=0x7fab53405e00,C=0x7fab53405e18
    0ms [I0] [INFO] (connection - L:407) Starting connection (0x7fab53405e18) to localhost:11210
    2ms [I0] [ERROR] (connection - L:123) Connection=0x7fab53405e18 failedLCBERR=0x18, OS Err=61
    2ms [I0] [INFO] (connmgr - L:173) Received result for I=0x7fab53405e00,C=0x7fab53405e18; E=0x18
    2ms [I0] [DEBUG] (cccp - L:304) CCCP Socket connected
    2ms [I0] [ERROR] (cccp - L:141) Got I/O Error=0x18
    2ms [I0] [INFO] (confmon - L:175) Provider [2] failed
    103ms [I0] [TRACE] (confmon - L:229) Current provider is 3
    103ms [I0] [TRACE] (htconfig - L:283) Starting HTTP Configuration Provider 0x7fab53814600
    103ms [I0] [INFO] (connection - L:407) Starting connection (0x7fab53814648) to localhost:9000
    103ms [I0] [INFO] (connection - L:137) Connection=0x7fab53814648,localhost:9000 completed succesfully
    103ms [I0] [DEBUG] (htconfig - L:229) Successfuly connected to REST API localhost:9000
    104ms [I0] [TRACE] (htconfig - L:129) Received 200 bytes on HTTP stream
    104ms [I0] [TRACE] (htconfig - L:145) HTTP not yet done. Err=0x25
    112ms [I0] [TRACE] (htconfig - L:129) Received 8411 bytes on HTTP stream
    112ms [I0] [DEBUG] (htconfig - L:140) Generation 0 -> 1
    112ms [I0] [INFO] (confmon - L:151) Setting new configuration of type 3
    112ms [I0] [DEBUG] (file - L:242) Got updated configuration. Flushing to file
    112ms [I0] [DEBUG] (bootstrap - L:58) Instance configured!

    113ms [I0] [DEBUG] (connmgr - L:262) Got request R=0x7fab56018c00,192.168.0.81:12000
    113ms [I0] [INFO] (connmgr - L:221) Starting connection on I=0x7fab552006c0,C=0x7fab552006d8
    113ms [I0] [INFO] (connection - L:407) Starting connection (0x7fab552006d8) to 192.168.0.81:12000
    113ms [I0] [INFO] (connection - L:137) Connection=0x7fab552006d8,192.168.0.81:12000 completed succesfully
    113ms [I0] [INFO] (connmgr - L:173) Received result for I=0x7fab552006c0,C=0x7fab552006d8; E=0x0
    113ms [I0] [INFO] (connmgr - L:160) Assigning R=0x7fab56018c00,c=0x7fab552006d8
    113ms [I0] [DEBUG] (connmgr - L:262) Got request R=0x7fab5601be00,127.0.0.1:12002
    113ms [I0] [INFO] (connmgr - L:221) Starting connection on I=0x7fab55200d70,C=0x7fab55200d88
    113ms [I0] [INFO] (connection - L:407) Starting connection (0x7fab55200d88) to 127.0.0.1:12002
    113ms [I0] [INFO] (connection - L:137) Connection=0x7fab55200d88,127.0.0.1:12002 completed succesfully
    113ms [I0] [INFO] (connmgr - L:173) Received result for I=0x7fab55200d70,C=0x7fab55200d88; E=0x0
    113ms [I0] [INFO] (connmgr - L:160) Assigning R=0x7fab5601be00,c=0x7fab55200d88
    358ms [I0] [INFO] (connmgr - L:402) Reclaiming connection I=0x7fab552006c0,Cu=0x7fab560080d8,Cp=0x7fab552006d8 (192.168.0.81:12000)
    358ms [I0] [INFO] (connmgr - L:402) Reclaiming connection I=0x7fab55200d70,Cu=0x7fab56008690,Cp=0x7fab55200d88 (127.0.0.1:12002)

 Comments   
Comment by Mark Nunberg [ 24/Jun/14 ]
http://review.couchbase.org/38743




[CCBC-456] IOCP plugin aborts on init failure Created: 23/Jun/14  Updated: 09/Jul/14  Resolved: 09/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.1.2
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: sdeigm42 Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Windows 7 / 64bit


 Description   
In some of the C client library function (at least in lcb_iocp_new_iops in libcouchbase/plugins/io/iocp/iocp_iops.c) the abort() method is called in case of an failure. This causes my entire application to terminate instead of giving me the chance to print out an appropriate failure message.

I would suggest to replace the abort() function call with a return of an appropriate error code.

 Comments   
Comment by Mark Nunberg [ 07/Jul/14 ]
I've written up a fix for this. Indeed in the specific location there, you may do better with just an error message; however in general it might be better to leave the abort()s where they are; it's better to have a program predictably crash and possibly leaving a core dump than have it unpredictably hang.
Comment by sdeigm42 [ 07/Jul/14 ]
For us and our customers an abort and the termination of our process in case of an failure which can happen during normal processing is unacceptable since we don't have any chance to report errors, do cleanup operations and other things. Of course our software is not 100% error free and could also crash for other reasons but hopefully only for programming errors and not for other situations like not enough sockets available or other missing resources. In my opinion a basic client library should never terminate a process.

The main reason why we are evaluating couchbase is the high availability of the database due to the cluster approach. Since our software is working in a 24/7 environment with minimal downtime any possible termination through libcouchbase would be a show stopper for us to use the library/database.
Comment by Mark Nunberg [ 07/Jul/14 ]
I'm more curious to know under which circumstances the original abort was seen in the first place. Are you operating in a memory constrained environment?
Comment by sdeigm42 [ 07/Jul/14 ]
We were doing stress tests from a Windows 7 client with 8GB memory agains a Redhat couchbase server. At some point we got a lot of network errors and then our software terminated. Later on we used a debugger to figure out that libcouchbase called the abort() method after the Windows function CreateIoCompletionPort() failed. So we had no chance to figure out why the Windows method failed, but I assume the system was running out of open socket descriptors.
Comment by Mark Nunberg [ 07/Jul/14 ]
http://review.couchbase.org/#/c/38817/
Comment by Mark Nunberg [ 07/Jul/14 ]
I'd be interested to know if the library was leaking sockets. Are you running views as part of your stress test?

I'd also recommend testing the 2.4 developer preview (and beta which will be released soon) if possible. Should probably fix any socket leaks.

Lacking that, please try the 2.3.2 version (our latest stable release). 2.1.2 is pretty old.
Comment by sdeigm42 [ 07/Jul/14 ]
We don't think that your library is leaking sockets since most of the time the stress tests complete without any problems. When the test cases failed some lcb_get and some lbc_store commands failed and lcb_strerror returned "Network error" and shortly after that our process aborted.

We are just running one view - and that just at the end of the test - to collect all keys that have to be deleted.

We will update to a newer version of libcouchbase as you suggested.

Just to summarize again: we have no problems with the library itself. Today we weren't even able to reproduce the problem at all. But we can not live with the theoretical problem that a limited resource is not available and our software terminates due to this fact. Since our software runs in a realtime environment where we get multiple inbound and outbound socket connections for various protocols (SQL databases, SOAP connections, HTTP connections ...) we are dependent on the underlying client libraries to report an error if some resource is not available. Otherwise we would have to control the overall socket usage of all these subsystems which is not feasible.
Comment by Mark Nunberg [ 07/Jul/14 ]
In your particular case, be aware the library creates a new TCP connection each time you query a view. This will be fixed in 2.4

I can't guarantee the code is free of other places where it may abort or crash due to allocation failure, though I believe the most common cases have been caught. If you see a certain place where this may happen, please report it.
Comment by sdeigm42 [ 08/Jul/14 ]
Ok, I will update to version 2.4 as soon as possible and if I encounter any new aborts I will report them.

Thanks for your great support!!




[CCBC-455] Restore support for 'cbcrc' file Created: 20/Jun/14  Updated: 23/Jun/14  Resolved: 23/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-454] Provide clearer API to obtain REST hosts (and other types of hosts for external use) Created: 19/Jun/14  Updated: 25/Jun/14  Resolved: 25/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently we have lcb_get_server_list, lcb_get_host, and lcb_get_port. All these functions are confusing and rather clumsy to use. We should rather have something that reflects how the server works today with its varied connection options.

Folks will most likely wish to use either the views or rest admin ports.

 Comments   
Comment by Mark Nunberg [ 25/Jun/14 ]
http://review.couchbase.org/38810




[CCBC-453] pillowfight: Add 'pause-at-end' option Created: 19/Jun/14  Updated: 24/Jun/14  Resolved: 24/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Dave Rigby Assignee: Dave Rigby
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
For certain load testing, it is useful to see what the cost of established connections is on the application and/or server. Therefore add a new option to pillowfight to pause at the end of execution (with all connections held open) to be able to measure the cost (memory, threads, FDs etc) of keeping them open.


 Comments   
Comment by Dave Rigby [ 19/Jun/14 ]
http://review.couchbase.org/#/c/38473/




[CCBC-452] Rename 'dsn' module to something more correct Created: 17/Jun/14  Updated: 25/Jun/14  Resolved: 25/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 24/Jun/14 ]
http://review.couchbase.org/#/c/38770/




[CCBC-451] info log instance configuration at time of creation Created: 16/Jun/14  Updated: 18/Jun/14  Resolved: 18/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
It is useful when reviewing logs to diagnose an issue to see what parameters were provided and tunables have been adjusted. This would be very helpful from a field serviceability perspective.

This is related to CCBC-447.

 Comments   
Comment by Mark Nunberg [ 17/Jun/14 ]
http://review.couchbase.org/#/c/38365/
Comment by Mark Nunberg [ 17/Jun/14 ]
0ms [I0] [INFO] (instance - L:414) Version=2.3.0, Changeset=b09f8ce84be9b3bbcde1ba1be66e3f281d37bed2
0ms [I0] [INFO] (instance - L:415) Effective connection string: couchbase://localhost/default?console_log_level=2&
0ms [I0] [INFO] (lcbio_mgr - L:303) <localhost:11210> (HE=0x1a3c260) Starting connection on I=0x1a3c8b0
0ms [I0] [INFO] (connection - L:421) <localhost:11210> (SOCK=0x1a3c9f0) Starting. Timeout=2000000us
1ms [I0] [INFO] (connection - L:99) <localhost:11210> (SOCK=0x1a3c9f0) Connected
1ms [I0] [INFO] (lcbio_mgr - L:257) <localhost:11210> (HE=0x1a3c260) Received result for I=0x1a3c8b0,C=0x1a3c9f0; E=0x0
1ms [I0] [INFO] (lcbio_mgr - L:240) <localhost:11210> (HE=0x1a3c260) Assigning R=0x1a3c210 SOCKET=0x1a3c9f0
1ms [I0] [INFO] (ioctx - L:75) <localhost:11210> (CTX=0x1a3e2a0,unknown) Pairing with SOCK=0x1a3c9f0
2ms [I0] [INFO] (ioctx - L:125) <localhost:11210> (CTX=0x1a3e2a0,sasl) Destroying. PND=0,ENT=1,SORC=1
2ms [I0] [INFO] (ioctx - L:75) <localhost:11210> (CTX=0x1a46790,unknown) Pairing with SOCK=0x1a3c9f0
2ms [I0] [INFO] (ioctx - L:125) <localhost:11210> (CTX=0x1a46790,unknown) Destroying. PND=0,ENT=1,SORC=1
2ms [I0] [INFO] (lcbio_mgr - L:463) <localhost:11210> (HE=0x1a3c260) Reclaiming connection I=0x1a3c8b0,C=0x1a3c9f0
3ms [I0] [INFO] (confmon - L:162) Setting new configuration. Received via CCCP
3ms [I0] [INFO] (ioctx - L:75) <localhost:11210> (CTX=0x1a3e370,unknown) Pairing with SOCK=0x1a3c9f0
3ms [I0] [INFO] (server - L:457) <localhost:11210> (SRV=0x1a4af80,IX=0) Setting initial timeout=2499ms
foo The key does not exist on the server (0xd)
4ms [I0] [INFO] (ioctx - L:125) <localhost:11210> (CTX=0x1a3e370,memcached) Destroying. PND=0,ENT=0,SORC=1
Comment by Matt Ingenthron [ 17/Jun/14 ]
Honestly, that's not very user friendly. It looks a bit too much like in the weeds debugging logging.

Can you remove the line number and module? Also lines like these seem like they shouldn't be info level and haven't anything user actionable.:
1ms [I0] [INFO] (lcbio_mgr - L:257) <localhost:11210> (HE=0x1a3c260) Received result for I=0x1a3c8b0,C=0x1a3c9f0; E=0x0
1ms [I0] [INFO] (lcbio_mgr - L:240) <localhost:11210> (HE=0x1a3c260) Assigning R=0x1a3c210 SOCKET=0x1a3c9f0

Added Brett and Don as watchers as well... would be good to get more feedback.
Comment by Brett Lawson [ 17/Jun/14 ]
I agree that quite a few of those should fall under DEBUG logging. Additionally, having the line number helps debug, but I also agree it should not show in usual log messages, perhaps a flag to enable or disable that kind of information?
Comment by Mark Nunberg [ 17/Jun/14 ]
Logging is off by default. The information displayed to the screen and their respective levels are:
(1) Still a work in progress.
(2) Only displayed during connection state changes, such that in steady mode there should never be any messages printed to the screen.

Perhaps we can make a google doc elsewhere for more suggestions on levels/displays/formatting/etc. which can pertain to the logging library. For starters the file/line information has been invaluable in helping diagnose issues when given logs from users. Yes they are a bit of line noise to the user but give an immense amount of insight as to what the application is doing from a developers perspective.
Comment by Matt Ingenthron [ 17/Jun/14 ]
Understood that it's off by default, but I think we should be thinking about CONFIG or INFO level as something that is intended to be used as *on* by default.

I also understand what you're saying on line numbers, but this should be thought about from the end user's UX, not what is most useful to us. Unfortunately, this one currently reads more like tracing than logging of important events.

One possible approach is that you have multiple log lines in the same place in the file. In other words, you may log a bit at INFO level and a lot more at DEBUG level. Another approach, as Brett recommends, is different log formatters.
Comment by Mark Nunberg [ 18/Jun/14 ]
The individual issue has been fixed. We should have a comb-through of the logging system in the future to determine a better way to handle this. One thing that comes to mind is extending the logging API - and making it volatile again (it has been volatile in 2.3.x) to be able to extend it with more flexibility.
Comment by Mark Nunberg [ 18/Jun/14 ]
Don, that is exactly what we already have. My key concern is about the callback and what exactly would be passed to it. For example the naive approach is that there is only a level/function/file/message; however with a library like libcouchbase we also add:

* Subsystem
* Instance ID

and we'd like to add:
* Host:Port
* Associated "context pointer"

The library is complex and asynchronous which means that there are:
(1) Many things happening
(2) The things aren't happening in a specified order
(3) There are possibly multiple concurrent instances of the same "thing" happening

A good example would be issuing multiple concurrent view requests. The requests may have the same destination and the same instance associated with them and would only differ by the local TCP port as well as the context pointer associated with them.

Another good (and very common) example is having multiple instances of the library operating within the same thread. We've actually run into this issue previously and added a new parameter to the callback just before the 2.3.1 release adding this field.

Of course will all this information I may consider moving to a struct-based callback rather than passing a million arguments; or maybe a different approach altogether.

My comments about the "volatility of the interface" refers mainly to this callback, rather than how a user may enable logging. How a user may enable logging should be considered committed - though with that some ambiguity might arise as well; for example the environment variable _only_ affects how logging is handled _if_ the application did not install its own custom callback, etc. I don't think it's a good idea to change how users may enable, disable, or filter logging levels though.




[CCBC-450] Provide additional error codes for various failures Created: 14/Jun/14  Updated: 16/Jun/14  Resolved: 16/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently we only return NETWORK_ERROR or CONNECT_ERROR upon various types of network failures. At the user's option we should also be able to return more detailed codes reflecting what actually happened to the socket. This feature should be compatible with older versions and thus disable by default.

 Comments   
Comment by Mark Nunberg [ 16/Jun/14 ]
http://review.couchbase.org/38297
http://review.couchbase.org/38298




[CCBC-449] pillowfight isn't listed as a subcommand of 'cbc Created: 13/Jun/14  Updated: 13/Jun/14  Resolved: 13/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Dave Rigby Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: OS X


 Description   
When running cbc with no arguments, it shows a list of possible subcommands. However pillowfight isn't present in the list:

      $cbc
      Error: Unknown command "cbc"
      Usage: cbc command [options]
      command may be:
         help show this help or for given command
         cat output keys to stdout
         cp store files to the cluster
         create store files with options
         observe observe key state
         flush remove all keys from the cluster
         hash hash key(s) and print out useful info
         lock lock keys
         unlock unlock keys
         rm remove keys
         stats show stats
         verify verify content in cache with files
         version show version
         verbosity specify server verbosity level
         view execute couchbase view (aka map/reduce) request
         admin execute request to management REST API
         bucket-create create data bucket on the cluster
         bucket-delete delete data bucket
         bucket-flush flush data bucket
      Use 'cbc command --help' to show the options



 Comments   
Comment by Dave Rigby [ 13/Jun/14 ]
http://review.couchbase.org/#/c/38223/
Comment by Dave Rigby [ 13/Jun/14 ]
@Mark - assigning back to you to set the fixVersion - I assume it'll be 2.3.2 but wanted to double-check.




[CCBC-448] Explicitly search /usr/local for newer OS X Created: 12/Jun/14  Updated: 13/Jun/14

Status: Reopened
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Wei-Li Liu Assignee: Wei-Li Liu
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Mac OS X


 Description   
I clone libcouchbase repo and build.with autotool on my mac
The instruction is :

$ ./config/autorun.sh
$ ./configure
$ make
$ make check
$ make install

On the 2nd step (./configure) I see
====================================================
checking for libev3... no
checking for libev4... no
checking for libevent >= 1.4... no
checking for libevent2... no
checking for libuv... no
configure: error: Failed to locate usable event library. You must install either libev or libevent dev package or use --disable-plugins option
====================================================


However, I do have libevent installed
=====================================================
Wei-Lis-MacBook-Pro:libcouchbase wei-li$ brew install libevent
Warning: libevent-2.0.21 already installed
Wei-Lis-MacBook-Pro:/ wei-li$ sudo find . -name libevent
Password:
find: ./dev/fd/3: Not a directory
find: ./dev/fd/4: Not a directory
./Users/wei-li/repo/couchbase/libcouchbase/plugins/io/libevent
./usr/local/Cellar/libevent
./usr/local/Library/LinkedKegs/libevent
./usr/local/opt/libevent
====================================================
configure script wasn't able to find the package
 






 Comments   
Comment by Mark Nunberg [ 12/Jun/14 ]
Please ensure you have the development libraries for libevent installed as well. It seems you have this available from homebrew, however your homebrew installation is not inside the default linker/compiler search paths.

Find an 'event.h' :)
Comment by Dave Rigby [ 13/Jun/14 ]
@Wei-Li

To be more specific, brew installs things in /usr/local/ by default, and so you need to add them to your search path if building from source. Try:

    CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure && make
Comment by Mark Nunberg [ 13/Jun/14 ]
or 'brew link libevent'. Can you find 'libevent.dylib'?
Comment by Dave Rigby [ 13/Jun/14 ]
@Mark: "brew link libevent" just puts things in /usr/local (at least on my box), which the lcb configure scripts don't look in by default (hence me specifying it).
Comment by Mark Nunberg [ 13/Jun/14 ]
mnunberg@mbp15 ~ $ cpp --verbose
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin12.5.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.8.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 224.1 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0 -I /usr/include -fdebug-compilation-dir /Users/mnunberg -ferror-limit 19 -fmessage-length 134 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.8.0 -fobjc-dispatch-method=mixed -fobjc-default-synthesize-properties -fencode-extended-block-signature -fdiagnostics-show-option -fcolor-diagnostics -traditional-cpp -o - -x c -
clang -cc1 version 5.0 based upon LLVM 3.3svn default target x86_64-apple-darwin12.5.0
ignoring duplicate directory "/usr/include"
  as it is a non-system directory that duplicates a system directory
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
Comment by Mark Nunberg [ 13/Jun/14 ]
It would be odd if you had to add -I/usr/local/include -- in any event, see what homebrew does (brew install libcouchbase?)
Comment by Dave Rigby [ 13/Jun/14 ]
@Mark - /usr/local/include has been removed in 5.1:

$ cpp --verbose
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.2.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 236.3 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -I /usr/include -fdebug-compilation-dir /Users/dave/Documents/Code -ferror-limit 19 -fmessage-length 233 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fdiagnostics-show-option -fcolor-diagnostics -vectorize-slp -traditional-cpp -o - -x c -
clang -cc1 version 5.1 based upon LLVM 3.4svn default target x86_64-apple-darwin13.2.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /usr/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks (framework directory)
End of search list.
Comment by Wei-Li Liu [ 13/Jun/14 ]
Adding the search path when build works for me

CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure && make

I have to switch to root user to resolve some permission issue after that steps. Eventually it builds successfully .
I wasn't sure if libcouchbase configure scripts doesn't look in by default. Now It is confirmed I will resolve this

Comment by Mark Nunberg [ 13/Jun/14 ]
Libcouchbase doesn't explicitly look in /usr/local because on any sane system that path is _already_ part of the default compiler search path. You should keep this open and rename the title to something like "explicitly search /usr/local for newer OS X"
Comment by Wei-Li Liu [ 13/Jun/14 ]
Done




[CCBC-447] Include lcb version in log output Created: 11/Jun/14  Updated: 16/Jun/14  Resolved: 16/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Dave Rigby Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
LCB_LOGLEVEL is very helpful for debugging issues, but it would be even more helpful if it printed the lcb version on startup, similar to the Java SDK.



 Comments   
Comment by Mark Nunberg [ 16/Jun/14 ]
http://review.couchbase.org/38301




[CCBC-446] Convey more clearly that 'CONTIG' key types imply the header is provided as well. Created: 08/Jun/14  Updated: 17/Oct/14  Resolved: 18/Jun/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: docs, library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Matt Ingenthron [ 08/Jun/14 ]
What is a 'contig' key type? Is this part of the public API?
Comment by Mark Nunberg [ 08/Jun/14 ]
https://github.com/couchbase/libcouchbase/blob/packet-ng/include/libcouchbase/api3.h

Yes, it's part of the public API. All these routines are currently deemed volatile as they are subject to change, however the intent is to eventually promote this api "Version 3 API" (the lcb 2.x API). This is an alternate API which provides much more clarity and conciseness with respect to scheduling and creating new commands, however it is not fully implemented. The older "2.x" API simply wraps this new API internally.
Comment by Matt Ingenthron [ 09/Jun/14 ]
With the thought that we're now going to do a major release, does it make sense to make this the new API as COMMITTED and deprecate (but not remove) the old one? We won't often have major releases and we should aim for clarity by having the 3.0 release the new API. Also, it's best if we don't have too many changes. I'm sure there's a schedule impact. We can discuss when we next get together.
Comment by Mark Nunberg [ 18/Jun/14 ]
http://review.couchbase.org/38405
Comment by Mark Nunberg [ 18/Jun/14 ]
Just in case yall were wondering, this is all highly volatile API, but it's there as this will form the basis of a 3.0 API when we actually do a 3.0




[CCBC-445] Provide 'sync destructor' functionality Created: 05/Jun/14  Updated: 12/Jun/14  Resolved: 12/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
in packet-ng, lcb_destroy() does not actually free I/O resources but decrements their reference counts. Some resources such as pending socket connects are "cancelled" asynchronously, with the actual 'free' event taking place on the next tick of the event loop rather than immediately.

The most common case is when the library has failed to connect or when there are pending config requests. While this is typically harmless (as the amount of memory "leaked" is fairly small) it is likely most annoying when debugging an application for memory leaks. This feature provides a means by which to offer "synchronous destroy" functionality. Unfortunately this cannot become the default for lcb_destroy() as async-based models also use lcb_destroy() and as such will end up with an infinite loop.

This feature will be implemented via lcb_cntl() and via an environment variable, thus code may compile against this feature and simply ignore an error at runtime if it cannot be set.




[CCBC-444] Can't handle out-of-order responses to concurrent GET requests. Created: 04/Jun/14  Updated: 20/Jun/14  Resolved: 05/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Bug Priority: Major
Reporter: Kevin Kingdon Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Centos 6.4, 64-bit, Couchbase-Server 2.2

Attachments: Text File purge_single_server.txt    

 Description   
We are using libcouchbase in an event-driven mode using libev, sharing the event loop between libcouchbase and our application. IWe instantiate a single library instance and send multiple "single get" requests before the callback from the first request. There is a chance that Couchbase server will send an out-of-order response to one of the GET requests. When that happens, libcouchbase returns an internal error: "Unknown implicit send message op=0" After debugging in the libcouchbase code, we see that the code only handles out-of-order responses for GETQ or GATQ or NOP opcodes. This implies that the application must wait after submitting a single get request to libcouchbase until the callback for that get request is invoked before submitting a second get request.

We believe that libcouchbase should handle the out-of-order responses. The data is there in the transaction log ring buffer to match the response to the request and issue the callback. If the application issues multiple concurrent GET requests, no immediate error is returned and the internal ring buffers grow to accommodate the outstanding requests. Indeed, if no out-of-order response happens, everything works. Forcing a single request at a time cuts our Couchbase transactions per second in half from around 9K tps to less than 5K tps.

 Comments   
Comment by Mark Nunberg [ 04/Jun/14 ]
Perhaps CCBC-435 is related to your issue.

In any event the next version (i.e. 2.4) will feature command lookup by opaque only and not rely on any implicit ordering of responses - thus not relying on request ordering at all.

The current behavior for memcached is to always serve requests in order - so unless you're running a modified memcached this should not be happening in the first place. But again, please see the CCBC-435 issue which I believe is the cause of what you're seeing now.
Comment by Kevin Kingdon [ 05/Jun/14 ]
I believe that we will still hit the internal error inside of lcb_server_purge_implicit_responses(). We will hit the switch statement starting on line 707 in server.c and take the default case. (The opcode of the request throwing the error is PROTOCOL_BINARY_CMD_GET.) The change you made doesn't appear to affect this code path because we were already not hitting the internal error condition in lcb_proto_parse_single. The switch statement anticipates receiving a response to a later request before a response to an earlier request, but only for a multi-get. Single-get responses must be received in-order or we will hit the default switch case and fail.

I don't know why the response for the later request is received from Couchbase server prior to the response to the earlier request. I'm not sure what constitutes a stale request, but the requests in our application are closely spaced in time -- we are issuing 9K requests per second to libcouchbase in our application. Our application log files don't show a significant time gap between the earlier request (whose response was skipped) and the later request whose response was received first.

Our application is expecting a callback of some kind for each request accepted by libcouchbase. When the internal error callback is made, all of the requests outstanding in libcouchbase at the time are lost and no callback to our app is made. The only workaround we have is to time out the request in our app and respond with an error to our app's remote client. If we don't destroy the library instance and recreate it before issuing new requests, the next few requests return a "network error" to the request callback. We are able to successfully retry those requests. Even if we destroy the library instance after the internal error, we still would lose the requests in the cmd_log ring buffer at the time of the error.
Comment by Mark Nunberg [ 05/Jun/14 ]
If there's a bug in the library, it may be in the assignment of the 'opaque' value, or in comparing the timestamps between the requests. In any event, the issue is resolved in the upcoming major release. See https://github.com/couchbase/libcouchbase/blob/1c602553e4502f3eec8f605f910ccffa283b2d9c/src/mc/mcreq.c#L540-553

If you can confirm you're getting responses out of order though, it'd be a bug in memcached
Comment by Kevin Kingdon [ 05/Jun/14 ]
-- "If you can confirm you're getting responses out of order though, it'd be a bug in memcached "

We are currently testing with a 2-node Couchbase cluster. In this configuration, individual GET requests can be sent to different nodes, based on the respective key hashes. Each node would respond in-order to the requests that node received, but collectively, the two nodes could respond out-of-order with respect to the order in which requests were submitted to libcouchbase. It is a testament to the ultra-fast responsiveness of Couchbase that we don't see a higher percentage of out-of-order responses.
Comment by Mark Nunberg [ 05/Jun/14 ]
Ok, but the code in question only cares about responses received on an individual server. Each server can be thought of as an individual pipeline that is isolated from all other servers. While it is certainly possible and common to receive responses out of order from the perspective of an application (i.e. key1 is mapped to server1 and key2 is mapped to server2, and server2 responds before server1 does) this should not affect the I/O logic of the application itself which is always scoped to the individual server in question.
Comment by Mark Nunberg [ 06/Jun/14 ]
Just checking on this, can you try with an older version of the library (i.e. 2.2.0) and see if that helps? - see http://packages.couchbase.com/clients/c/index.html for a listing of older versions
Comment by Kevin Kingdon [ 06/Jun/14 ]
Unfortunately, we had to move to 2.3.0 to get the CCBC-270 fix. We use libev. We were hitting this issue when we start our application while Couchbase server is still warming up. Our app shares the event loop with libcouchbase.

I plan on working on this issue some more next week. I will do a private build of 2.3.1 with additional instrumentation to track our app's requests through the libcouchbase library. Hopefully that will provide an additional clue. We tried serializing requests to libcouchbase, but that has a severe performance impact, primarily because we end up serializing both view-based and memcached-based requests into the same application queue. I would rather not have to do a lot of rework at the app level to work around this, but we are now experiencing lost requests and consequent memory leaks. The subsystem that is waiting for responses from the "database" module times out and sends an error response to the remote client, but the "database" module that provides the application wrapper for Couchbase is leaking the requests it assumes would come back in libcouchbase callbacks.
Comment by Mark Nunberg [ 06/Jun/14 ]
Yours sounds like a very typical use case - though the "unknown implicit send message" is rare - we haven't seen this in 2.3.x. Typically the "high risk" factors are when you have a rebalance or failover -- then you end up with relocating commands and timeouts (which should still work, but is obviously a less travelled code path)

Also, have you tried to use the built in logging feature (just set LCB_LOGLEVEL in the environment, with a value of 1-5 (5 being the highest)) to see what might possibly be going wrong?

You can also try to compile the libev plugin for 2.3.x and use it with 2.2 - it's worth a shot
Comment by Kevin Kingdon [ 19/Jun/14 ]
I was able to find the root cause of this issue. It is in the libev plugin. Occasionally, (especially when many of requests are being sent at once), the EV_WRITE event for a connection is not triggered, and the library never sends one request to the server. When a later request is sent and its response is received, libcouchbase is still expecting a response to the unsent request. This triggers the internal-error path during the ensuing attempt to clean up. The fix is to force an EV_WRITE event when updating the LCB_WRITE_EVENT flag for the connection. Fixed code below:

static int lcb_io_update_event(struct lcb_io_opt_st *iops,
                               lcb_socket_t sock,
                               void *event,
                               short flags,
                               void *cb_data,
                               void (*handler)(lcb_socket_t sock,
                                               short which,
                                               void *cb_data))
{
    struct libev_cookie *io_cookie = iops->v.v0.cookie;
    struct libev_event *evt = event;
    int events = EV_NONE;

    if (flags & LCB_READ_EVENT) {
        events |= EV_READ;
    }
    if (flags & LCB_WRITE_EVENT) {
        events |= EV_WRITE;
    }

    if (events == evt->ev.io.events && handler == evt->handler) {
        /* no change! */
        return 0;
    }

    ev_io_stop(io_cookie->loop, &evt->ev.io);
    evt->data = cb_data;
    evt->handler = handler;
    ev_init(&evt->ev.io, handler_thunk);
    ev_io_set(&evt->ev.io, sock, events);
// KWK -- redundant -- ev_io_stop(io_cookie->loop, &evt->ev.io);
    ev_io_start(io_cookie->loop, &evt->ev.io);
// KWK
    // If we are setting the write flag, force a write event to become pending.
    // Otherwise, occasionally we miss triggering this
    if (flags & LCB_WRITE_EVENT) {
        ev_feed_event(io_cookie->loop, &evt->ev.io, EV_WRITE);
    }
// KWK

    return 0;
}
Comment by Kevin Kingdon [ 19/Jun/14 ]
BTW, This now passes my libcouchbase stress test where I have up to 800 'get' requests outstanding with libcouchbase at one time. (I am issuing a total of 300K get request distributed to 10 lcb instances running in 10 separate threads, each with its own event loop.)
Comment by Mark Nunberg [ 19/Jun/14 ]
This sounds quite strange. Does this happen if you use a different libev backend? AFAIK libev should use level-triggered events rather than edge-triggered ones. Perhaps this is a bug inside libcouchbase somewhere else where the write event is mistakenly unscheduled?
Comment by Kevin Kingdon [ 20/Jun/14 ]
I reverted my patch to the libev plugin and added back some of my instrumentation of libcouchbase. I am including a trace below of a sequence of 'get' requests and responses. The 'seq' is the packet/request 'opaque' value. When the 'parse_single' is called with a packet 'seq' greater than the expected request 'seq', the internal error will occur. The 'seq' not sent is 4742. I don't yet know why the conn->output ringbuffer's read_head moves from seq(4742) to seq(4745) without seq(4742) being written.

v0_handle:try-write:conn(0x7f8694015d18):seq(4738)
parse_single:conn(0x7f8694015d18):seq(4735)=req-seq(4735)
parse_single:conn(0x7f8694015d18):seq(4736)=req-seq(4736)
single_get:conn(0x7f8694015d18):seq(4742)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4742)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4742)
parse_single:conn(0x7f8694015d18):seq(4737)=req-seq(4737)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4742)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4742)
single_get:conn(0x7f8694015d18):seq(4743)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4742)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4742)
single_get:conn(0x7f8694015d18):seq(4744)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4742)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4742)
parse_single:conn(0x7f8694015d18):seq(4738)=req-seq(4738)
parse_single:conn(0x7f8694015d18):seq(4739)=req-seq(4739)
parse_single:conn(0x7f8694015d18):seq(4740)=req-seq(4740)
single_get:conn(0x7f8694015d18):seq(4745)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4745)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4745)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4745)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4745)
single_get:conn(0x7f8694015d18):seq(4746)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4745)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4745)
v0_handler:try-write:conn(0x7f8694015d18):seq(4745)
single_get:conn(0x7f8694015d18):seq(4748)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4748)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4748)
single_get:conn(0x7f8694015d18):seq(4750)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4748)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4748)
single_get:conn(0x7f8694015d18):seq(4751)
lcb_sockrw_set_want:write-wanted:conn(0x7f8694015d18):seq(4748)
lcb_sockrw_apply_want:write-scheduled:conn(0x7f8694015d18):seq(4748)
v0_handler:try-write:conn(0x7f8694015d18):seq(4748)
parse_single:conn(0x7f8694015d18):seq(4745)>req-seq(4742)
Comment by Kevin Kingdon [ 20/Jun/14 ]
Per your suggestion, I re-ran the stress test with LIBEV_FLAGS=1, LIBEV_FLAGS=2, and LIBEV_FLAGS=4, to try various libev backends. In each case, libcouchbase failed to send one or more 'get' requests to the couchbase server.
Comment by Kevin Kingdon [ 20/Jun/14 ]
OK, I think I've found the root cause of the issue, and also have an explanation for why my workaround actually helps. In 2.3.1, there is a per-connection timer that fires and checks for stale requests in the command log. The code in purge_single_server() is called to check for and fail any stale requests. The problem in the purge_single_server() implementation is that it incorrectly assumes that the connection's output ring-buffer has the same request at the read-head as does the command-log ring-buffer. But this is not the case when some requests have been sent to the server and not yet received responses; in this case, the output ring-buffer has consumed the transmitted requests and is either now empty or has some later requests yet-to-be sent. In this latter case, the purge_single_server() code is broken -- it is attempting to remove an unsent request from the output ring buffer that may not even be present. In the particular run-through I examined, the earliest request in the command-log hadn't timed out, so no command failure was triggered, but the code near the bottom of purge_single_server() reset the output ring buffer without preserving the unsent requests. (The # of bytes to be sent was less than the number of bytes in the command-log ring buffer, triggering the reset without copying the unsent requests.)

The workaround I proposed works, I believe, because the i/o watcher is made pending before the timer watcher becomes pending. So the i/o callback is earlier in the pending list than the timer callback. The i/o callback tries to send the output ring buffer, and would nearly always succeed (always worked in my tests). When the timer fires, the output ring buffer is empty, and the broken purge_single_server() handling of the output ring buffer is avoided.

Fixing this in 2.3.1 would be difficult (for me, in any case). It would require either ignoring the output ring buffer, which might result in later responses for stale requests, or it would require tracking the relationship between requests in the command log and their position in the output ring buffer. I know the code has been entirely reworked for 2.4, so this may no longer be an issue.
Comment by Kevin Kingdon [ 20/Jun/14 ]
I've attached a modified version of purge_single_server() that attempts to fix the issue with the command-log ring-buffer being out-of-sync with the connection output ring-buffer.




[CCBC-443] Add DSN/URI/Connection string for creation options Created: 03/Jun/14  Updated: 10/Jun/14  Resolved: 10/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This will add a URI feature as mentioned in this document: https://docs.google.com/document/d/172ausWsYt3eYYOZ1lYHVS8ccbrrVJaGwHIRsf_O_Hyc/edit




[CCBC-442] Allow lcb_cntl_setstring() to pass string key-value pairs Created: 03/Jun/14  Updated: 03/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This feature will allow setting of common options via key-value string pairs rather than through "funny looking" hex codes

 Comments   
Comment by Mark Nunberg [ 03/Jun/14 ]
http://review.couchbase.org/#/c/37788/




[CCBC-441] lcb_arithmetic does not set expiration in packet header Created: 03/Jun/14  Updated: 03/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Brett Lawson [ 03/Jun/14 ]
https://gist.github.com/brett19/af735a0c16ce3f1fc813/revisions
Comment by Mark Nunberg [ 03/Jun/14 ]
http://review.couchbase.org/#/c/37785/




[CCBC-440] Apply new shuffle algorithm to packet-ng Created: 03/Jun/14  Updated: 18/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to CCBC-433 hostlist_randomize skips the first ho... Resolved

 Description   
See CCBC-433

 Comments   
Comment by Mark Nunberg [ 18/Jun/14 ]
http://review.couchbase.org/37790




[CCBC-439] Improvement needed on cbc help text Created: 03/Jun/14  Updated: 04/Sep/14  Resolved: 04/Sep/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.2
Security Level: Public

Type: Bug Priority: Minor
Reporter: Perry Krug Assignee: Perry Krug
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The current output of help for cbc does not show the arguments required for each command:
[root@cb1 ~]# cbc help create
Usage: create [options] [arguments]

-? Print this help text (--help)
-h val Hostname to connect to (--host)
-b val Bucket to use (--bucket)
-u val Username for the rest port (--user)
-P val password for the rest port (--password)
-T Enable command timings (--enable-timings)
-t val Specify timeout value (--timeout)
-D Behave like legacy memcached client (default false) (--dumb)
-S val Force SASL authentication mechanism ("PLAIN" or "CRAM-MD5") (--sasl)
-C val Specify transport for bootstrapping the connection: "HTTP" or "CCCP" (default) (--config-transport)
-Z val Path to cached configuration (--config-cache)
-f val Flag for the new object (--flag)
-e val Expiry time for the new object (--exptime)
-a Fail if the object exists (--add)

It has also been requested that cbc and/or libcouchbase have a 'man' page which might help in parallel here.

 Comments   
Comment by Mark Nunberg [ 04/Sep/14 ]
http://review.couchbase.org/#/c/41195/




[CCBC-438] Properly handle datatype in response structure (strip/leave compression settings) Created: 02/Jun/14  Updated: 09/Jun/14  Resolved: 09/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-437] cbc: add datatype and ssl options Created: 02/Jun/14  Updated: 12/Jun/14  Resolved: 12/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 12/Jun/14 ]
https://github.com/couchbase/libcouchbase/commit/aadec1086ff3b346bb0917567ad893203c973cd2




[CCBC-436] Create tests for existence of ctls Created: 02/Jun/14  Updated: 03/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 03/Jun/14 ]
http://review.couchbase.org/#/c/37789/




[CCBC-435] Stale commands may be conflated with new responses Created: 31/May/14  Updated: 02/Jul/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0-dp2
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Gantt: start-finish
Relates to

 Description   
When a response for a new command is received, the library may conflate this with an older version of the same command. This is because the "purge_implicit_responses" function is being called _after_ the request data is being retrieved for the response.

 Comments   
Comment by Mark Nunberg [ 31/May/14 ]
http://review.couchbase.org/#/c/37712/




[CCBC-434] pillowfight should output errors by default Created: 30/May/14  Updated: 14/Jun/14  Resolved: 14/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Perry Krug Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When running pillowfight, there is no indication of error when a node dies, etc.



 Comments   
Comment by Mark Nunberg [ 02/Jun/14 ]
As far as I can tell, cbc will output actual operation errors if and when they occur.
Generic cluster topology change events would be reflected in logging (which must be enabled in the environment) and not as programmatic output from pillowfight.
Comment by Perry Krug [ 02/Jun/14 ]
Then this is a valid bug that the errors from cbc are not being reported when running pillowfight. I ran it, "kill -9" memcached and got no output on the command prompt that I was running it on.
Comment by Mark Nunberg [ 02/Jun/14 ]
I don't see how this is a bug, you need to run with logging. pillowfight is a tool to test throughput between client and server, the client can't possibly catch and figure out whenever a node is down. Have you run with logging? Do you want logging to be enabled for pillowfight by default?
Comment by Perry Krug [ 02/Jun/14 ]
The client can catch and report when it gets a timeout or a connection reset can it not?

How do I enable logging for pillowfight via the command-line?
Comment by Mark Nunberg [ 02/Jun/14 ]
A node failing over doesn't necessarily mean the command will time out if the client can retry to send it when the node comes back up before the timeout expires. libcouchbase is operation-driven which means it can only know and programmatically report when an operation fails.

You can enable logging by LCB_LOGLEVEL in the environment, using a number between 1-5 as the value, with higher numbers being more verbose. I'd suggest =2
Comment by Perry Krug [ 02/Jun/14 ]
Mark, please read my previous notes...I was not talking about failover at all, rather a node actually _failing_.

Trying with that environment variable set does work well, thank you. I would think we want at least LCB_LOGLEVEL=1 to be set by default. You have to remember, not everyone running this command is an expert C programme ;-)
Comment by Matt Ingenthron [ 03/Jun/14 ]
+1 to some logging on pillowfight by default. It's new functionality in 2.3, so understandable that it's not there, but would be good to add.
Comment by Mark Nunberg [ 12/Jun/14 ]
http://review.couchbase.org/#/c/38211/




[CCBC-433] hostlist_randomize skips the first host in the list. Created: 30/May/14  Updated: 16/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.1
Fix Version/s: 2.3.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Patrick Varley Assignee: Patrick Varley
Resolution: Fixed Votes: 0
Labels: connection, customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
Relates to
relates to CCBC-440 Apply new shuffle algorithm to packet-ng Resolved

 Comments   
Comment by Mark Nunberg [ 02/Jun/14 ]
Pending +V from submitter.




[CCBC-432] The code example in Configuration Sources is wrong Created: 28/May/14  Updated: 09/Jun/14  Resolved: 09/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.0
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Minor
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: cccp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://docs.couchbase.com/couchbase-sdk-c-2.3/#configuration-sources


 Description   
The example code for disabling HTTP is wrong and will not compile:

lcb_config_transport transports[] = {
    LCB_CONFIG_TRANSPORT_CCCP,
    LCB_CONFIG_TRANSPORT_LIST_END
};

cb_config_transport is wrong it should be cb_config_transport_t:

lcb_config_transport_t transports[] = {
    LCB_CONFIG_TRANSPORT_CCCP,
    LCB_CONFIG_TRANSPORT_LIST_END
};

 Comments   
Comment by Mark Nunberg [ 09/Jun/14 ]
https://github.com/couchbaselabs/docs-ng/pull/131




[CCBC-431] Replace vbucket_* function usages with new lcbvb prefix universally. Created: 25/May/14  Updated: 24/Jun/14  Resolved: 24/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-430] Ensure SSL is properly setup in threaded environments Created: 25/May/14  Updated: 09/Jul/14  Resolved: 09/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Also enusre SSL_library_init() etc is done correctly as well.

 Comments   
Comment by Mark Nunberg [ 07/Jul/14 ]
http://review.couchbase.org/39202




[CCBC-429] Select plugin will needlessly busy-poll when timeouts are active. Created: 24/May/14  Updated: 25/May/14  Resolved: 25/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0, 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 24/May/14 ]
http://review.couchbase.org/37564




[CCBC-428] Use terse (/pools/default/bs/$bucket) rather than compat (/pools/default/bucketsStreaming/default) for streaming config Created: 24/May/14  Updated: 26/May/14  Resolved: 26/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Use the terse URI which is far more efficient on the cluster. This would allow to have the benefits of the library despite using an older cluster.

This should be implemented in a way so that older clusters will not suffer the need to make a new TCP connection if the first request (to bs) fails.

 Comments   
Comment by Mark Nunberg [ 24/May/14 ]
http://review.couchbase.org/#/c/37575/




[CCBC-427] Use common code for decoding and parsing HTTP responses. Created: 24/May/14  Updated: 25/May/14  Resolved: 25/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently we have the streaming HTTP parser which we use for bucket configuration and another HTTP parser which we use for parsing the lcb_make_http_request() functionality. We should merge these into a single parser.




add support for SSL to libcouchbase in support of Couchbase Server 3.0 (CCBC-344)

[CCBC-426] Make data connections use SSL encryption if enabled Created: 21/May/14  Updated: 25/May/14  Resolved: 25/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Technical task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 24/May/14 ]
http://review.couchbase.org/37339




add support for SSL to libcouchbase in support of Couchbase Server 3.0 (CCBC-344)

[CCBC-425] implement SSL I/O Core with OpenSSL for both event models Created: 21/May/14  Updated: 25/May/14  Resolved: 25/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Technical task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-424] libcouchbase not updated in brew recipes ... Created: 21/May/14  Updated: 22/May/14  Resolved: 22/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.1
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Trivial
Reporter: adonoho Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: libcouchbase
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: OS X v10.9.2


 Description   
libcouchbase v2.3.1 is current. As of this morning, 20140521, brew is installing v2.3.0.


awd$ brew upgrade libcouchbase
==> Upgrading 1 outdated package, with result:
libcouchbase 2.3.0
==> Upgrading libcouchbase
==> Downloading http://packages.couchbase.com/clients/c/libcouchbase-2.3.0.tar.gz
######################################################################## 100.0%
==> ./configure --prefix=/usr/local/Cellar/libcouchbase/2.3.0 --disable-examples --disable-tests --disable-couchbasemock
==> make install
🍺 /usr/local/Cellar/libcouchbase/2.3.0: 118 files, 1.6M, built in 16 seconds


Anon,
Andrew


 Comments   
Comment by Dave Rigby [ 21/May/14 ]
Have you tried updating brew?

    brew update; brew upgrade libcouchbase

Comment by adonoho [ 21/May/14 ]
Dave,

That solves the problem.

Please consider updating the documentation to reflect the fact that one must update brew to get the latest recipes. (To those of us who have only installed brew to support libcouchbase, these kinds of handholding steps matter. I only ever use brew when a new libcouchbase update occurs.)

Thank you.

Anon,
Andrew
Comment by Matt Ingenthron [ 21/May/14 ]
Mark: when you have a moment, please add such a line to the docs.
Comment by Mark Nunberg [ 21/May/14 ]
https://github.com/couchbaselabs/docs-ng/pull/127




[CCBC-423] Don't request configuration from nodes about to be ejected Created: 20/May/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1, 2.4.3
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on CCBC-420 vbucket: Don't map item to server wit... Resolved

 Comments   
Comment by Mark Nunberg [ 20/May/14 ]
In addition to not remapping keys to "Eject Wait" nodes, we should also not try to solicit them for config requests.




[CCBC-422] Build Ubuntu 14.04 packages and provide them in a repo Created: 20/May/14  Updated: 26/Jun/14  Resolved: 26/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Brent Woodruff Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In our installation documentation, we reference several repositories one can use to install libcouchbase on Ubuntu 10.04, 11.10, and 12.04:

http://docs.couchbase.com/couchbase-sdk-c-2.3/index.html#ubuntu

Ubuntu 14.04 has been released, making it the most recent LTS release. The 14.04 LTS release is to be maintained through 2019. Please prepare an Ubuntu 14.04 package of libcouchbase and provide it via an apt repository.

 Comments   
Comment by Mark Nunberg [ 20/May/14 ]
I'll need to set up a VM environment and see exactly why we need so many varied ubuntu dists.




[CCBC-421] Allow option to disable relocation of commands on failover Created: 20/May/14  Updated: 01/Jul/14  Resolved: 02/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: None
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Won't Fix Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Provide option to disable rescheduling of commands to other nodes when topologies have changed. Packet-ng should handle this more gracefully, but it would still be helpful to avoid non-idempotent operations from being retried.

Currently we don't have the facilities to inspect (in 2.3.x) which operations should be retried and which shouldn't be, but we should add this as an extensible option in any event.

 Comments   
Comment by Mark Nunberg [ 02/Jun/14 ]
Has been resolved in a VF, but we're not gonna roll it into 2.3.x




[CCBC-420] vbucket: Don't map item to server with no vbuckets Created: 20/May/14  Updated: 20/May/14  Resolved: 20/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.7, 2.2.0, 2.3.1
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks CCBC-423 Don't request configuration from node... Open
Relates to

 Description   
in found_incorrect_master, we sometimes end up mapping keys to servers based on their presence inside the serverList array. However we should really be ignoring this node if it doesn't have any vBuckets mapped to it.

See https://www.couchbase.com/issues/browse/MB-10543?focusedCommentId=82995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-82995

 Comments   
Comment by Mark Nunberg [ 20/May/14 ]
Matt, Alk -- I've placed you both here as watchers for this ticket.
Comment by Mark Nunberg [ 20/May/14 ]
http://review.couchbase.com/#/c/37322/
Comment by Mark Nunberg [ 20/May/14 ]
packet-ng does not use found_incorrect_master - so this is not _entirely_ applicable to it.




[CCBC-419] Crash in libcouchbase 2.3.0 Created: 20/May/14  Updated: 16/Jun/14  Resolved: 16/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.0.3
Fix Version/s: 2.3.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Haster Assignee: Mark Nunberg
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I have 2-node claster and if all nodes are in pending state and I try to set some data to claster
my application crashes with such stack:
(gdb) bt
#0 0x0000003435c0e9dd in raise () from /lib64/libpthread.so.0
#1 0x00002b887dc2c5b6 in skgesigOSCrash () from /u01/app/oracle/product/11.2.0/client/lib/libclntsh.so.11.1
#2 0x00002b887deca609 in kpeDbgSignalHandler () from /u01/app/oracle/product/11.2.0/client/lib/libclntsh.so.11.1
#3 0x00002b887dc2c7c6 in skgesig_sigactionHandler () from /u01/app/oracle/product/11.2.0/client/lib/libclntsh.so.11.1
#4 <signal handler called>
#5 0x0000003435030265 in raise () from /lib64/libc.so.6
#6 0x0000003435031d10 in abort () from /lib64/libc.so.6
#7 0x00002b88814980a5 in ringbuffer_consumed (buffer=0x7073, nb=30646) at src/ringbuffer.c:263
#8 0x00002b888149a00c in lcb_purge_single_server (server=0x1daa9168, error=LCB_ETIMEDOUT) at src/server.c:330
#9 0x00002b888149d927 in lcb_server_timeout_handler (sock=<optimized out>, which=<optimized out>, arg=0x6) at src/timeout.c:38
#10 0x00002b8884e49348 in event_base_loop () from /import/home/hasg_test/IBES_INSTANCE/Linux/5.0/i686/HAS_SERVER_051.10__0/64/bin/lib/libevent-2.0.so.5
#11 0x00002b888149e893 in lcb_wait (instance=0x1db96800) at src/wait.c:60
#12 0x00000000004312d0 in Couchbase::set_values (this=0x1d731568, elements=..., timeout=100) at couchbase_loader_source/couchbase.cpp:268
#13 0x000000000043e9e1 in couchbase_loader::writer_thread::execute (this=0x1d7314e0) at couchbase_loader_source/writer_thread.cpp:49
#14 0x00002b88836a48d3 in threads::thread_proc(void*) () from /import/home/hasg_test/IBES_INSTANCE/Linux/5.0/i686/HAS_SERVER_051.10__0/64/bin/libhas_common.so
#15 0x0000003435c0673d in start_thread () from /lib64/libpthread.so.0
#16 0x00000034350d40cd in clone () from /lib64/libc.so.6
#17 0x0000000000000000 in ?? ()


 Comments   
Comment by Haster [ 20/May/14 ]
Sorry, I;ve given this error on 2.0.3 version. But I thinks that it will reproduce and on 2.3.0 too.
Now I can't set all nodes on my cluster to pending state, so I can't check it
Comment by Patrick Varley [ 11/Jun/14 ]
Haster have you reproduce this on 2.3.1? I know we have fixed a number of bugs around ringbuffer_consumed since 2.0.3
Comment by Haster [ 12/Jun/14 ]
Patrick, I can't reproduce this bug in new version. So, I think you can drop it.




[CCBC-418] Not-My-Vbucket may result in immediate timeouts Created: 19/May/14  Updated: 26/Jun/14  Resolved: 26/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.3.1
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Bug Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
[EDIT] remove previous description after analysis

This is specific to the case when there is only a single node and a new node is being added. In this case we will get a not-my-vbucket during the first relocation. Since the node has not yet been added (the current I/O handler must first return to the event loop) the client has nowhere that it can forward the command to, and ends up dispatching it with a not-my-vbucket.

 Comments   
Comment by Mark Nunberg [ 19/May/14 ]
This will probably only be fixed in packet-ng
Comment by Mark Nunberg [ 26/Jun/14 ]
Note that an ETIMEDOUT may still be issued, however the client can requeue the operation upon receipt of such error.




[CCBC-417] TTP/TTR calculation in observe response handling is broken. Created: 15/May/14  Updated: 15/May/14  Resolved: 15/May/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Venu Uppalapati Assignee: Mark Nunberg
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
TTP/TTR calculation in observe response handling is broken
https://github.com/couchbase/libcouchbase/blob/master/src/handler.c#L533-536

On server side only the persist_time is sent as part of the CAS.
https://github.com/couchbase/ep-engine/blob/master/src/ep_engine.cc#L4334-4341

 Comments   
Comment by Venu Uppalapati [ 15/May/14 ]
This is a known limitation on server side. Server side does not currently have a implementation for time taken for replication.




[CCBC-416] Map in-situ AUTH_ERROR to NETWORK_ERROR or retry Created: 14/May/14  Updated: 03/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0, 2.3.1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to

 Description   
Authentication errors may be received during a rebalance with CCCP (See CCBC-357). We should be able to distinguish these errors from ones received as an actual authentication failure. This is handled in packet-ng, but not in 2.3.x

The point of the issue is to help developers not worry about receiving this error during normal application flow - since there is no actionable path for such an error.

 Comments   
Comment by Mark Nunberg [ 03/Jun/14 ]
http://review.couchbase.org/#/c/37800/




[CCBC-415] IPv6 Functionality Created: 14/May/14  Updated: 02/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: New Feature Priority: Major
Reporter: Haster Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If I put some ipv6 address (for example "::1") function lcb_host_parse(...) returns LCB_INVALID_HOST_FORMAT error.
It is needed to add ipv6 support


 Comments   
Comment by Mark Nunberg [ 14/May/14 ]
I'm not sure how well ipv6 works up the stack tbh. I'm curious though - did this work in lcb 2.2?
Comment by Haster [ 14/May/14 ]
Mark, I don't know =)
But I suggest that it don't work in 2.2 version,
But in libcouchbase there is some things, related to ipv6, such as function lcb_behavior_set_ipv6 in cntl.c

Maybe it's not a bug but future request or task =)




[CCBC-414] cccp_cookie is never assigned to cccp_provider Created: 14/May/14  Updated: 18/Jun/14  Resolved: 20/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We never assign cccp_cookie to cccp->cmdcookie, thereby not allowing us to set an ignore flag when the object is being released.

 Comments   
Comment by Mark Nunberg [ 20/May/14 ]
http://review.couchbase.org/#/c/37321/




[CCBC-413] Noisy warnings when extracting libcouchbase tarball Created: 13/May/14  Updated: 02/Jul/14  Resolved: 02/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: 2.3.2
Security Level: Public

Type: Bug Priority: Minor
Reporter: Michael Leib Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS 6, latest rpms installed from yum, tar --version reports "tar (GNU tar) 1.23"


 Description   
Got the latest 2.31 libcouchbase-2.3.1.tar.gz

gunzip the .gz => ok
tar xvf *.tar => not ok

...
libcouchbase-2.3.1/cmake/Modules/GenerateConfigDotH.cmake
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
libcouchbase-2.3.1/cmake/Modules/GetLibcouchbaseFlags.cmake
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
libcouchbase-2.3.1/cmake/Modules/GetPlatformCCInfo.cmake
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
...

etc, etc

 Comments   
Comment by Mark Nunberg [ 13/May/14 ]
Despite the annoying messages from the output (I've yet to resolve this), the tar command does succeed. I've just verified this on a Centos system (6.5) and it all builds and extracts successfuly.
Comment by Mark Nunberg [ 13/May/14 ]
A quick google suggests this is the result of creating the archive on OS X: http://superuser.com/questions/318809/linux-os-x-tar-incompatibility-tarballs-created-onos-x-give-errors-when-unt




[CCBC-412] config cache: Documentation suggests using deprecated function lcb_create_compat() Created: 13/May/14  Updated: 22/May/14  Resolved: 22/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.1
Fix Version/s: 2.3.2
Security Level: Public

Type: Task Priority: Critical
Reporter: Dave Rigby Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
As of lcb 2.3.1, the `lcb_create_compat` function is deprecated - see http://docs.couchbase.com/couchbase-sdk-c-2.3/#release-notes-for-couchbase-client-library-c-231-ga-09-may-2014. However the documentation at http://docs.couchbase.com/couchbase-sdk-c-2.3/#file-based-configuration-cache still suggests using the deprecated method.

`lcb_cntl` with the operation `LCB_CNTL_CONFIGCACHE` is now the recommended method of enabling config cache. The documentation should be updated to reflect this.




 Comments   
Comment by Mark Nunberg [ 19/May/14 ]
https://github.com/couchbaselabs/docs-ng/pull/127




[CCBC-411] 2.3.1 Release notes are missing. Created: 12/May/14  Updated: 12/May/14  Resolved: 12/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.1
Fix Version/s: None
Security Level: Public

Type: Task Priority: Critical
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: releasenote
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://docs.couchbase.com/couchbase-sdk-c-2.3/#release-notes

 Comments   
Comment by Mark Nunberg [ 12/May/14 ]
https://github.com/couchbaselabs/docs-ng/pull/122
Comment by Amy Kurtzman [ 12/May/14 ]
The release notes are available on the docs site now.

Please note that the announcements of the SDK releases and the publication of the release notes on the docs website is not always coordinated and will often lag behind the announcement. The announcement does mention that possibility and includes a link to the release notes on GitHub. In the future, please consider checking with me about the documentation release status before filing an issue about it not being on the docs website.




[CCBC-410] Typo in CCCP Struct Code Example in 2.3 C SDK Documentation Will Result in Compilation Error Created: 09/May/14  Updated: 12/May/14  Resolved: 12/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Morrie Schreibman Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: usability
Remaining Estimate: 0.5h
Time Spent: Not Specified
Original Estimate: 0.5h


 Description   
The code error which I found may be seen at

http://docs.couchbase.com/couchbase-sdk-c-2.3/index.html#integration-with-libcouchbase

The code example given:
.
  lcb_t instance;
  struct lcb_create_st options;
  lcb_config_transport_t enabled_transports = {
  LCB_CONFIG_TRANSPORT_CCCP,
  LCB_CONFIG_TRANSPORT_LIST_END
  };

  memset(&options, 0, sizeof(options));
  options.version = 2;
  options.v.v2.mchosts = "example.com:11210";
  options.v.v2.transports = enabled_transports;
...
will not compile (at least it will not compile under gcc). The problem is on the 3rd line. The correct code is
.
  lcb_t instance;
  struct lcb_create_st options;
  lcb_config_transport_t enabled_transports [ ] = {
  LCB_CONFIG_TRANSPORT_CCCP,
  LCB_CONFIG_TRANSPORT_LIST_END
  };

  memset(&options, 0, sizeof(options));
  options.version = 2;
  options.v.v2.mchosts = "example.com:11210";
  options.v.v2.transports = enabled_transports;
.

-while this may seem trivial, my experience is that customers frequently paste in source code examples without studying them in detail, and then get themselves in trouble later. So this is a doc bug which should be fixed.


 Comments   
Comment by Mark Nunberg [ 12/May/14 ]
https://github.com/couchbaselabs/docs-ng/pull/122




select io mode should not define "fd_set" struct array. Only one "fd_set" variable is OK. (CCBC-407)

[CCBC-408] FORWARDPORT: => Select fixups for 2.4/3.0 Created: 06/May/14  Updated: 18/Jun/14  Resolved: 25/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 21/May/14 ]
http://review.couchbase.org/37388




[CCBC-407] select io mode should not define "fd_set" struct array. Only one "fd_set" variable is OK. Created: 05/May/14  Updated: 18/Jun/14  Resolved: 08/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.0
Fix Version/s: 2.3.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: whl0630 Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: memory-use
Σ Remaining Estimate: 20m Remaining Estimate: 20m
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: 20m Original Estimate: 20m
Environment: no relation with os env

Sub-Tasks:
Key
Summary
Type
Status
Assignee
CCBC-408 FORWARDPORT: => Select fixups for 2.4... Technical task Resolved Mark Nunberg  

 Description   
The fd_set struct related definition (e.g. fd_set readfds[FD_SETSIZE]; ) in "plugins\io\select\plugin-select.c" should be only one item, not array, like: fd_set readfds. Because fd_set has been a set array.

In our os env, FD_SETSIZE is 51200, which means "readfds[FD_SETSIZE]" will consume very large memory.



 Comments   
Comment by Mark Nunberg [ 06/May/14 ]
Thank you for the input. I'll fix it ASAP.
Comment by Mark Nunberg [ 06/May/14 ]
http://review.couchbase.org/36758




[CCBC-406] instancepool example does not run Created: 04/May/14  Updated: 27/Jul/14  Resolved: 27/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.3.0
Fix Version/s: 2.4.0
Security Level: Public

Type: Bug Priority: Trivial
Reporter: Shiv Dayal Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: documentation
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Ubunutu 14.04 GCC 4.8.2


 Description   
Inf function pool_create in pool.c following should have a case 2

switch(options->version) {
    case 0:
        io = options->v.v0.io;
        break;
    case 1:
        io = options->v.v1.io;
        break;
    default:
        /*
         * I don't know the layout of the option structure (its created
         * after I wrote the example) so I don't now if an io ops structure
         * is provided or not.
         */
        return LCB_EINTERNAL;
    }

like case 2:
        io = options->v.v2.io;

otherwise it says cannot create pool: internal libcouchbase error because it throws error.




[CCBC-405] Throw appropriate error on empty key Created: 02/May/14  Updated: 10/Jul/14  Resolved: 10/Jul/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Bug Priority: Major
Reporter: Jon Moses Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: OSX 10.9, LCB 2.3.0

$ cbc version
cbc built from: libcouchbase 2.3.0_15_g0b04b36 (rev. 0b04b36c1ef8d568271ea402b582e4339b81c8f3)
    using libcouchbase: 2.3.0 (libevent)
    using libyajl: 2.0.4

Attachments: File minimal.c    

 Description   
If I issue a query that returns 0x07, the next two queries, regardless of validity will return 0x10 instead of the expected results.

See 'minimal.c' attached

 Comments   
Comment by Sergey Avseyev [ 02/May/14 ]
Expected output:
$ example/minimal/minimal
STORED "foo" CAS: 17097673194045177856
GOT "foo" CAS: 17097673194045177856 FLAGS:0x0 SIZE:3
bar
GET ERROR: Invalid input/arguments (0x7)
GOT "foo" CAS: 17097673194045177856 FLAGS:0x0 SIZE:3
bar


Actual output:
$ example/minimal/minimal
STORED "foo" CAS: 580788420889542656
GOT "foo" CAS: 580788420889542656 FLAGS:0x0 SIZE:3
bar
GET ERROR: Invalid input/arguments (0x7)
ERROR: Network failure (0x10), (null)


Matt, do you know whether the server defines behaviour on empty key (zero length)
Comment by Mark Nunberg [ 02/May/14 ]
The error callback is not and never has been a reliable way to programmatically extract the error from an operation. For what it's worth, 2.2.0 exhibits the same behavior:

+ LD_LIBRARY_PATH=/sources/libcouchbase-2.0.7/inst/lib ./minimal
Using libcouchbase 2.0.7
STORED "foo" CAS: 858236688785346304
GOT "foo" CAS: 858236688785346304 FLAGS:0x0 SIZE:3
bar
GET ERROR: Invalid arguments (0x7)
GOT "foo" CAS: 858236688785346304 FLAGS:0x0 SIZE:3
bar
+ LD_LIBRARY_PATH=/sources/libcouchbase-2.1.3/inst/lib ./minimal
Using libcouchbase 2.1.3
STORED "foo" CAS: 5565267957526957824
GOT "foo" CAS: 5565267957526957824 FLAGS:0x0 SIZE:3
bar
GET ERROR: Invalid arguments (0x7)
ERROR: Network error (0x10), (null)
+ LD_LIBRARY_PATH=/sources/libcouchbase-2.2.0/inst/lib ./minimal
Using libcouchbase 2.2.0
STORED "foo" CAS: 8308748484727672576
GOT "foo" CAS: 8308748484727672576 FLAGS:0x0 SIZE:3
bar
GET ERROR: Invalid arguments (0x7)
ERROR: Network error (0x10), (null)
+ LD_LIBRARY_PATH=/sources/libcouchbase-2.3.0/inst/lib ./minimal
Using libcouchbase 2.3.0
STORED "foo" CAS: 3755695023363067648
GOT "foo" CAS: 3755695023363067648 FLAGS:0x0 SIZE:3
bar
GET ERROR: Invalid input/arguments (0x7)
GET ERROR: Network failure (0x10)
ERROR: Network failure (0x10), (null)


The issue seems to be in the client assuming a specific behavior of error_callback which is not guaranteed.
Comment by Jon Moses [ 02/May/14 ]
I'm not sure if the behavior expresses with a different initial error than 0x07, that's just the one that I reproduced from my data.
Comment by Mark Nunberg [ 02/May/14 ]
If you're scheduling commands asynchronously, then the failed commands in the pipeline will end up being invoked with the error received. This is actually correct behavior since the server is closing the socket.

Observe:

recvmsg(6, {msg_name(0)=NULL, msg_iov(1)=[{"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 65495}], msg_controllen=0, msg_flags=0}, 0) = 0
write(2, "GET ERROR: Invalid input/argumen"..., 41GET ERROR: Invalid input/arguments (0x7)
) = 41
epoll_ctl(3, EPOLL_CTL_DEL, 6, {EPOLLIN, {u32=6, u64=6}}) = 0
close(6) = 0
write(2, "ERROR: Network failure (0x10), ("..., 38ERROR: Network failure (0x10), (null)
) = 38


The first command gets EINVAL, and the error callback is invoked from it. The subsequent read from the socket returns 0 indicating the server has closed the connection which causes the remaining commands to fail with LCB_NETWORK_ERROR.
Comment by Mark Nunberg [ 02/May/14 ]
Proper solution is to return an error (either at lcb level or sdk level) if the key is empty.
Comment by Mark Nunberg [ 10/Jul/14 ]
http://review.couchbase.org/39284




[CCBC-404] Segfault in lcb2.3 in connmgr invoke_request() Created: 02/May/14  Updated: 07/May/14  Resolved: 07/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0
Fix Version/s: 2.3.1
Security Level: Public

Type: Task Priority: Critical
Reporter: Dave Rigby Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS 6u5

Issue Links:
Dependency

 Description   
Customer running lcb2.3 (on top of Python SDK) and reported:

<quote>
TimeoutError: <Key=u'DEV_509075a7b22c93664cc9d853ca7f7a722100c3e1', RC=0x17[Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout], Operational Error, Results=1, C Source=(src/multiresult.c,148)>

And then it crashed with core dumps.
</quote>

I dug into this a little bit, and I see essentially the same backtrace on both core files:

(gdb) bt 5
#0 invoke_request (req=0x7ffaa84792c0) at src/connmgr.c:126
#1 0x00007ffb2993de0f in connection_available (he=0x7ffab402c700) at src/connmgr.c:163
#2 0x00007ffb2994cb5b in timer_callback (sock=<value optimized out>, which=<value optimized out>, arg=0x7ffab4008d40) at src/timer.c:45
#3 0x00007ffb26b81b44 in ?? ()
#4 0x00007ffb26d940e0 in ?? ()

The `conn` field looks garbage:

(gdb) p *req
$9 = {llnode = {next = 0x0, prev = 0x0}, callback = 0x7ffb29949bb0 <socket_connected>, key = "inbox-31:11210", '\000' <repeats 1035 times>, he = 0x7ffab402c700, timer = 0x7ffaa8c13530, state = 1, conn = 0x18, err = LCB_SUCCESS, data = 0x7ffaa872fd80}

Digging further, this came from the he->idle list with is supposedly empty, but has size ==1 :

(gdb) p he->ll_idle
$11 = {next = 0x7ffab402c700, prev = 0x7ffab402c700, size = 1}




 Comments   
Comment by Dave Rigby [ 07/May/14 ]
http://review.couchbase.org/#/c/36601/
Comment by Mark Nunberg [ 07/May/14 ]
Reopen if this does not resolve the issue.




Handle API/ABI modifications from 2.x => 3.0/2.4 (CCBC-377)

[CCBC-403] Restore 'memcached compat' functionality. Created: 01/May/14  Updated: 25/Jun/14  Resolved: 25/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Technical task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This functionality is broken in 2.2.0+

We should check if we want to re-add this before GA, or leave it alone.

 Comments   
Comment by Matt Ingenthron [ 01/May/14 ]
what does `memcached compat` mean?
Comment by Mark Nunberg [ 01/May/14 ]
Basically the ability to connect to a dumb memcached cluster without a vbucket config etc.

The question is:

(1) Do we want to support this functionality? I can see it being "cute" but I'm not sure how useful it would be
(2) Does anything rely on this functionality? My answer is 'NO' considering that it's been broken for quite some time.
Comment by Matt Ingenthron [ 01/May/14 ]
It was useful for isolating issues in the past. I think the answer for #1 is probably not, but I wouldn't be unhappy if we maintained it and not to my knowledge on #2.
Comment by Mark Nunberg [ 01/May/14 ]
Yeah, my thoughts are similar. It shouldn't be too hard to readd it either; It's just not the time to do it yet -- and mark it @volatile or similar.
Comment by Mark Nunberg [ 24/Jun/14 ]
http://review.couchbase.org/#/c/38752/




[CCBC-402] Provide API for exposing buffer/pool settings (rdb, netbuf) Created: 01/May/14  Updated: 30/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-401] Remove 2.3.0 'v2' creation structure ABI fields, replace with lcb_cntl() Created: 30/Apr/14  Updated: 30/Apr/14  Resolved: 30/Apr/14

Status: Closed
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.3.0
Fix Version/s: 2.3.1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
See CCBC-392 for a discussion of why this is necessary




[CCBC-400] Improve API doc navbar layout Created: 29/Apr/14  Updated: 26/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Trivial
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

- If I click on "Introduction" from the "Couchbase C Client" the left hand menu folds up.
- Introduction is in a funny place in the list on the left.





[CCBC-399] Promote logging api to 'uncommitted' Created: 29/Apr/14  Updated: 03/Jun/14  Resolved: 03/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Logging API should not be volatile. If our current API is indeed volatile then let us make one which isn't.

 Comments   
Comment by Mark Nunberg [ 03/Jun/14 ]
http://review.couchbase.org/#/c/37801/




[CCBC-398] Ensure all API calls have interface attributes. Created: 29/Apr/14  Updated: 01/May/14  Resolved: 01/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.0-dp1
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We should be clear to document any functions to have the proper 'stability attributes'; this would require about an 30 mins work to simply skim over the list.

 Comments   
Comment by Mark Nunberg [ 01/May/14 ]
http://review.couchbase.org/#/c/36543/




[CCBC-397] Create a back-compat file for lcb_cntl operations Created: 29/Apr/14  Updated: 01/May/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Minor
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Provide a file which may be used by any C client application.

This may be dropped into other clients as 'lcb_cntl_compat.h' and would conditionally define any cntls not already defined. We could make this auto generated via a script. This would avoid other clients having to #ifdef this stuff themselves.




[CCBC-396] Make it simpler to execute tests against a real cluster Created: 27/Apr/14  Updated: 28/Apr/14  Resolved: 28/Apr/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 2.4.0-dp1
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently we must run tests using special environment variables. Considering that 'check-all' is now our wrapper, we should now add 'real cluster' params to the check-all script.




[CCBC-395] Add configuration cache API via lcb_cntl() Created: 25/Apr/14  Updated: 18/Jun/14  Resolved: 29/Apr/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.3.1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to CCBC-392 ABI compatibility broken on 2.3 when ... Resolved

 Description   
This would add a simple api to use lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_CONFIGCACHE, "filename"); which would be applied to an already-created instance.

 Comments   
Comment by Mark Nunberg [ 25/Apr/14 ]
http://review.couchbase.org/#/c/36330/
Comment by Mark Nunberg [ 25/Apr/14 ]
Fixed in 2.3. Pending merged in 2.4




[CCBC-394] Crash on win32 in lcb_get_replica Created: 25/Apr/14  Updated: 25/Jul/14  Resolved: 01/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.3.0
Fix Version/s: 2.3.1
Security Level: Public

Type: Task Priority: Blocker
Reporter: Haster Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
How to reproduce:
1) Start claster with 3 nodes
2) Start a loop of getting some vakue from claster ()first use lcb_get and if NETWORK_ERROR or CONNECTION_ERROR then try to use lcb_get_replica)
3) Stop one node - as a result we have crash

There is a call stack below:
libcouchbase.dll!ringbuffer_ensure_capacity(ringbuffer_st * buffer=0x00000000, unsigned long size=0x00000018) Line 62 + 0x4
libcouchbase.dll!lcb_server_buffer_start_packet_ex(lcb_server_st * c=0x066b0770, lcb_command_data_st * ct=0x12edfce0, ringbuffer_st * buff=0x00000000, ringbuffer_st * buff_cookie=0x066b0794, const void * data=0x12edfd00, unsigned long size=0x00000018) Line 199 + 0x11 libcouchbase.dll!lcb_server_start_packet_ex(lcb_server_st * c=0x066b0770, lcb_command_data_st * ct=0x12edfce0, const void * data=0x12edfd00, unsigned long size=0x00000018) Line 216 + 0x6 bytes C
libcouchbase.dll!lcb_get_replica(lcb_st * instance=0x035ed8c0, const void * command_cookie=0x0357bb88, unsigned long num=0x00000001, const lcb_get_replica_cmd_st * const * items=0x12edfd38) Line 210 C
cache_impl.dll!0a7e2361()
[Frames below may be incorrect and/or missing, no symbols loaded for cache_impl.dll]
cache_impl.dll!0a7e3b1d()


 Comments   
Comment by Haster [ 30/Apr/14 ]
Hi!
I made some investigation of this problem and found that then I called lcb_get_replica in lcb_server_start_packet_ex the buff is assigned with NULL and after that lcb_server_buffer_start_packet_ex->ringbuffer_ensure_capacity is called.
But in lcb_get function is used lcb_server_start_packet instead of lcb_server_start_packet_ex.
In that function is additional check:
        if (!c->connection.output) {
            c->connection.output = calloc(1, sizeof(ringbuffer_t));
            ringbuffer_initialize(c->connection.output, 8092);
        }

Maybe it is enough to add such check to lcb_server_start_packet_ex. function?
I don't know what about memory leaks in that situation...
Comment by Mark Nunberg [ 30/Apr/14 ]
Yes, this should be enough. memory will not leak because anything assigned to ->output will be cleaned up either by the io system or by the server's connection dtor.
Comment by Mark Nunberg [ 30/Apr/14 ]
http://review.couchbase.org/#/c/36525/




[CCBC-393] Add apidoc section about environment variables Created: 25/Apr/14  Updated: 25/Jun/14  Resolved: 25/Jun/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.0-beta
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 24/Jun/14 ]
http://review.couchbase.org/#/c/38777/




[CCBC-392] ABI compatibility broken on 2.3 when using config cache. Created: 24/Apr/14  Updated: 01/May/14  Resolved: 01/May/14

Status: Resolved
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0
Fix Version/s: 2.3.1
Security Level: Public

Type: Task Priority: Critical
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Fixed Votes: 0
Labels: