[CCBC-397] Create a back-compat file for lcb_cntl operations Created: 29/Apr/14  Updated: 25/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Provide a file which may be used by any C client application.

This may be dropped into other clients as 'lcb_cntl_compat.h' and would conditionally define any cntls not already defined. We could make this auto generated via a script. This would avoid other clients having to #ifdef this stuff themselves.




[CCBC-331] memcached buckets may receive operation failures during rebalance Created: 20/Feb/14  Updated: 25/Oct/14

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Subhashni Balakrishnan Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When updating the config received on http stream, the client returns an error LCB_CLIENT_ETMPFAIL for memcached buckets. Please clarify if this design is intentional.

https://github.com/couchbase/libcouchbase/blob/master/src/bconf_provider.c#L183


 Comments   
Comment by Mark Nunberg [ 20/Feb/14 ]
I'd argue this is by design as a memcached bucket is usually intended for lightweight caching purposes. Rather than rescheduling all commands we just fail them all out. The error message might need to be a bit more helpful?
Comment by Matt Ingenthron [ 18/Mar/14 ]
This came up during a test of an upgrade. Usually during upgrade with memcached buckets, we just fire the request at the node we're aware of. That does mean that there is a possible lack of consistency between two actors on where to fire a request, based on the config they're using and the ketama hashing, but that's okay for a cache.

I don't think we'd expect failing out requests though in this case.

From reading the code, it looks like maybe we just need to do nothing during this replace_config() if it's for memcached buckets? Looking at that, right now if we have an add node case (which comes up in this test), we'll push a bunch of errors up to the app that would have otherwise been successful.
Comment by Mark Nunberg [ 18/Mar/14 ]
Because we support hashkey, we have no way to route requests to a server without that kind of context. With Couchbase buckets we already have the vbucket so we don't need to re-hash; however this isn't the case with memcached buckets as the destination bucket depends on the hash of the actual key which we forget about once the operation is scheduled.

In order to make this work, we'd either need to disable support for hashkey or keep the (hash)key around for future operations.
Comment by Matt Ingenthron [ 18/Mar/14 ]
I think the right thing to do here is to use the previously hashed location. Anything different would be a change over what we do in other clients and moxi. it'd also be a change over what we did in previous releases, I believe.

Looking at it from a cache perspective, when adding a node to the cluster the old node has a cached value, and the new node will eventually have a cached value. The old location would slowly LRU out and the requests would get to the new place eventually.

So if we don't rehash, we actually will likely get correct behavior, see what I mean? It's different than vbucket situations, but memcached buckets are different than couchbase buckets.
Comment by Mark Nunberg [ 18/Mar/14 ]
For storage, the stored data will never be read back out again - so it's pointless to send the data there.

For retrieval, a potentially stale version of the data may be retrieved (assuming it hasn't been updated on the "correct" node yet). I think the "Correct" behavior in that light is to retry retrieval operations but not storage operations. This would still "fail" operations, but I think it's the correct thing to do.

If you agree, please change the title of this ticket to reflect this - and we'll try to fix it -- maybe in this release or in the next.
Comment by Matt Ingenthron [ 18/Mar/14 ]
What was the behavior before the integration of confmon?

For storage, it may be pointless to send the data there, but it's more disruptive if we start returning errors to the caller when there was no error. There was a topology change which has a defined behavior.

For retrieval, the potentially stale is the expected behavior with memcached buckets. I believe that's the behavior in the current release too.

Is it a large change to just not do anything in that section of code? It doesn't seem like it. We'd just let the operations go to the node they're hashed for and then we'd have (approximately) the same behavior as in the current release, right?
Comment by Mark Nunberg [ 18/Mar/14 ]
2.3 has not changed the way we handle this. This has always been the behavior -- I'm just explaining why the behavior has been the way it has been.

See: https://github.com/mnunberg/libcouchbase/blob/2.2.0/src/bconf_provider.c#L320-339 (2.2.0)
See: https://github.com/mnunberg/libcouchbase/blob/2.0.7/src/instance.c#L734-744 (2.0.7)

You'd need a bit more code out there to relocate properly to a memcached bucket (we relocate currently based on the vbucket destination -- in this case we'd need to remember the _old_ destination). Also, this does not help if a server was _removed_ from the cluster (in which case the prior "Destination Index" may no longer exist).
Comment by Matt Ingenthron [ 19/Mar/14 ]
I didn't realize this was the behavior through release 2.2. We should definitely release note this.

I don't think we need to relocate, as previously mentioned. We don't do that elsewhere, but we don't have failures either. Also, not worried as much about the remove case.
Comment by Mark Nunberg [ 25/Oct/14 ]
See http://review.couchbase.org/#/c/42445/ for a prospective fix. I'm not sure when this change will actually make it through; just that it exists.




[CCBC-357] Authentication Errors during rebalance Created: 07/Apr/14  Updated: 25/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0-dp2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to MB-11875 memcached should respond with differe... Resolved

 Description   
Authentication errors may be delivered to the application during a rebalance and while CCCP is being employed.

 Comments   
Comment by Mark Nunberg [ 14/May/14 ]
This _may_ have been fixed in 2.3.1
Comment by Mark Nunberg [ 14/May/14 ]
And.. apparently not. This has been a fairly difficult one to track down and reproduce.
Comment by Mark Nunberg [ 14/May/14 ]
A possible explanation is that since we depend on network errors and not-my-vbucket replies for new configuration information, we might end up in a situation where we don't end up with a wrong mapping until well after a node has been removed (including the 10 second grace period). Need more info to confirm.
Comment by Mark Nunberg [ 02/Jun/14 ]
This one will be open for quite a while. It may have been resolved in packet-ng, however the difficulty to reproduce this means we can't really verify for certain




[CCBC-531] Document the behaviour of Arithmetic/Counter operations max/min values Created: 24/Oct/14  Updated: 25/Oct/14

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: 2.4.4
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Ian McCloy Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
http://docs.couchbase.com/sdk-api/couchbase-c-client-2.4.0/group___l_c_b___a_r_i_t_h.html

The following should be documented in the API reference. In libcouchbase the values that are incremented and decremented are unsigned 64bit int values. So valid values are integers from 0 to 18,446,744,073,709,551,615 If you have the maximum value 18,446,744,073,709,551,615 and increment it by 100 it will wrap to 0 and keep incrementing, this will set the value to 99. If the value is 0 and you decrement the value by 1 it will stay at 0, it does not wrap.

 Comments   
Comment by Mark Nunberg [ 25/Oct/14 ]
http://review.couchbase.org/42444




[CCBC-532] Incr / Decr with TTL doesn't change TTL value Created: 23/Oct/14  Updated: 25/Oct/14

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 2.4.4
Security Level: Public

Type: Bug Priority: Major
Reporter: Ian McCloy Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Ubuntu 12.04.5
Python 2.7.3
CB 3.0
LCB Version= 2.4.3
Python SDK - 2014-10-06 10:33:08.652762 __version__ = '1.2.4'

Setting a new TTL value to a document during an increment or decrement call doesn't change the TTL value for the document. *Am still testing this against other SDKs/LCB as well*



Have this is a try/except block to print any errors, no errors are printed.

=========

print "Setting foo now"

c.set("foo", 18446744073709551615, ttl=300)

try:
  data = c.get("foo")

except CouchbaseError as e:
    print "Couldn't retrieve value for key", e
    # Rethrow the exception, making the application exit
    raise

====

print out the key_exptime

then run

====


print "Incrementing with TTL"

try:
  c.incr("foo", amount=100, ttl=10)

except CouchbaseError as e:
    print "Couldn't incr value for key", e
    # Rethrow the exception, making the application exit
    raise

=====

The value is incremented as expected.
print out key_exptime and it hasn't changed and doc still exists.

 Comments   
Comment by Mark Nunberg [ 23/Oct/14 ]
#!/usr/bin/env python
from couchbase import Couchbase
from time import sleep

cb = Couchbase.connect(bucket='default')
cb.delete('counter', quiet=True)
cb.incr('counter',amount=1, initial=100, ttl=1)
sleep(2)
cb.get('counter')

===============
Traceback (most recent call last):
  File "ttl.py", line 9, in <module>
    cb.get('counter')
  File "/Users/mnunberg/Source/pycbc-stable/couchbase/connection.py", line 500, in get
    return _Base.get(self, key, ttl, quiet, replica, no_format)
couchbase.exceptions._NotFoundError_0xD (generated, catch NotFoundError): <Key=u'counter', RC=0xD[The key does not exist on the server], Operational Error, Results=1, C Source=(src/multiresult.c,282)>
Comment by Mark Nunberg [ 23/Oct/14 ]
Apparently, TTL only works if `create` is true, per the protocol specs.

https://github.com/couchbase/libcouchbase/blob/master/src/operations/counter.c#L57-66
Comment by Mark Nunberg [ 24/Oct/14 ]
Will fix this in LCB




[CCBC-530] Pillowfight performance drops to near-0 during rebalance Created: 23/Oct/14  Updated: 23/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screen Shot 2014-10-23 at 11.54.58 AM.png    

 Description   
Running this pillowfight command which should be very light in usage:
cbc-pillowfight -U couchbase://ec2-54-176-27-210.us-west-1.compute.amazonaws.com/event_stream?retry_backoff=0.0 -t 4 -B 1 -I 300000 -m 1024 -M 1024 -c -1

I rebalance one node out of the cluster and the performance becomes very erratic during the rebalance and then returns to very smooth once completed. See attached screenshot for what I'm observing.

According to Mark, an improvement could be made to have the client keep using the ffmap as it gets NMV responses instead of throwing it away each time.






[CCBC-526] Configuration source may revert to HTTP (port 8091) during excessive errors Created: 16/Oct/14  Updated: 17/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.0, 2.4.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Perry Krug Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In AWS, I am running this command:
cbc-pillowfight -U couchbase://ec2-54-176-251-0.us-west-1.compute.amazonaws.com/event_stream -t 128 -B 10000 -I 10000 -m 1024 -M 1024 -n -c -1

I am observing many:
Operation(s) failed: [0x17] Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout

I assume this is related to resource exhaustion and is not the main focus of this bug...but I think it is related so I included it.

The main concern is that within a few seconds of running this command, I observe a large increase in the number of 8091 connections to my Couchbase Server cluster. With the exact command above, the increase is in the range of 60-80 connections before I terminate the command. When adding a "-v" to capture logs, it's only around 6-10 connections increase, but is still more than I would expect and hopefully representative of the problem. With "-vv or -vvv" I am unable to reproduce the behavior which I assume is due to some race condition/context switching.

Logs are here: http://customers.couchbase.com.s3.amazonaws.com/perry/libcouchbase (sorry, it's rather big, the increase of connections should be relatively near the end)

 Comments   
Comment by Mark Nunberg [ 16/Oct/14 ]
I'm moving this to minor because in any event the client will use the "terse" URI (starting from lcb version 2.4.0), which is much more resource friendly to the cluster.

The reason, in this case, that it's falling back to port 8091 is because the connection via the 11210 port is timing out, and the client will switch to the next available bootstrap method.

Note that interaction with port 8091 can be disabled entirely by using 'bootstrap_on=cccp' in the connection string. Note that this behavior is not the default in order to support older servers (which do not have CCCP) and memcached buckets (which do not support CCCP).




[CCBC-525] Make failure tests more deterministic Created: 16/Oct/14  Updated: 17/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Subhashni Balakrishnan
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
From a community user:
I'm trying to build libcouchbase from sources on Ubuntu 12.04 and have several tests failing. Here's the output:

[ RUN ] MockUnitTest.testReconfigurationOnNodeFailover
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:252: Failure
Value of: iter->second
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
tests/iotests/t_netfail.cc:316: Failure
Value of: lcb_get_num_nodes(instance)
Actual: 10
Expected: 9
[ FAILED ] MockUnitTest.testReconfigurationOnNodeFailover (9225 ms)
[ RUN ] MockUnitTest.testBufferRelocationOnNodeFailover
tests/iotests/t_netfail.cc:414: Failure
Value of: rv.error
Actual: 23
Expected: LCB_SUCCESS
Which is: 0
[ FAILED ] MockUnitTest.testBufferRelocationOnNodeFailover (36752 ms)
[ RUN ] MockUnitTest.testSaslMechs
[ OK ] MockUnitTest.testSaslMechs (1853 ms)
[ RUN ] MockUnitTest.testMemcachedFailover
tests/iotests/t_netfail.cc:536: Failure
Value of: lcb_get_num_nodes(instance)
Actual: 10
Expected: 9
[ FAILED ] MockUnitTest.testMemcachedFailover (1743 ms)

Tried building master, release 2.4.2 and 2.3.2.

 Comments   
Comment by Mark Nunberg [ 16/Oct/14 ]
Unfortunately these tests are not fully deterministic; all I can say is that they pass "most of the time", and the reason why the failures appear are due to timing issues between the mock and the client library.

Development of a new, more deterministic mock server is underway, but it will be quite some time until it's properly integrated within the client.

Generally, ensuring there is sufficient CPU and memory resources on the machine running the tests will ensure they have a high pass rate, but this is not guaranteed.
Comment by Mark Nunberg [ 16/Oct/14 ]
For further reference, our tests are executed on our Jenkins builders for each commit. They typically have at least one test that fails on one or another configuration; this will typically be a result of insufficient CPU resources which cause the mock to become slow. See: http://sdkbuilds.couchbase.com/job/lcb-linux-cmake/ for the latest test results.

Comment by Mark Nunberg [ 17/Oct/14 ]
This isn't specific to 2.4.2, but all versions of the library. Assigning to subhashni :) - I know she is working on something




[CCBC-506] Lower the default CONFDELAY threshold Created: 02/Sep/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0, 2.4.1, 2.4.2
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The default setting for this is now 10 seconds. This means that it may take up to this time to refetch a new configuration, causing outages in such use environments when such a delay is unacceptable. While this number is certainly adjustable there seems to be some confusion and difficulty in conveying exactly what this number means. Perhaps a lower default value is ok for this as well?

To recap, this interval specifies the throttle period during which a client instance will not request a new configuration. This is the interval from the _last_ received configuration, and basically helps us avoid requesting too many config updates in a short timespan.

I am thinking about something like 3 seconds for this?

 Comments   
Comment by Mark Nunberg [ 02/Sep/14 ]
Adding matt to watchers, as maybe he can contribute his thoughts
Comment by Subhashni Balakrishnan [ 15/Oct/14 ]
can we try to do some test runs with different values and decide based on that - but this only impacts in the cases where we don't get a config on NMV




[CCBC-423] Don't request configuration from nodes about to be ejected Created: 20/May/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1, 2.4.3
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on CCBC-420 vbucket: Don't map item to server wit... Resolved

 Comments   
Comment by Mark Nunberg [ 20/May/14 ]
In addition to not remapping keys to "Eject Wait" nodes, we should also not try to solicit them for config requests.




[CCBC-507] Add test case for empty config Created: 08/Sep/14  Updated: 08/Sep/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Server 3.0.0 sometimes returns an empty config. We should test for this and ensure the parse fails rather than crashing the client.

 Comments   
Comment by Mark Nunberg [ 08/Sep/14 ]
See MB-12104. I think this wouldn't be an issue in the library as the parsing would fail and the library would attempt the next node/provider. Still useful to have a test for this, though.




[CCBC-505] Allow user-side modification of node's statuses Created: 02/Sep/14  Updated: 04/Sep/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Critical
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to JCBC-539 Consider a way to expose target key n... Open

 Description   
This feature will allow users to get/set the node statuses in cases where existing application infrastructure can be better than the existing built in cluster configuration info with respect to determining if nodes are down , unavailable etc.

This helps increase responsiveness in applications when a node is "known" to be down, while also not forcing any possibly library-side heuristic upon the user.

 Comments   
Comment by Mark Nunberg [ 04/Sep/14 ]
There are actually two ways to implement this. One is at the network level where we can forcefully close a socket, one is at the vbucket level where we can simply set all the indices to -1 where the node's real index would be instead.




[CCBC-503] need better documentation on how to use error classifiers Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.4.0, 2.4.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Documentation has a section on error handling, but there's nothing in the documentation other than a release note on error classifiers.

Please add some detail on what these are and how they're intended to be used.

 Comments   
Comment by Mark Nunberg [ 28/Aug/14 ]
http://docs.couchbase.com/couchbase-sdk-c-2.4/index.html#error-handling-and-diagnostics gives a pretty detailed explanation. Do you mean to add an additional section within the API docs, or something else?




[CCBC-496] Abstract openssl use to dynamically loaded plugin Created: 07/Aug/14  Updated: 07/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If an application is using a different version of openssl, the library's version may conflict with it. The solution is to use dynamic loading of the plugin at runtime so that the application's version remains in-tact.




[CCBC-495] Library does not consider hostname and IP of the same server as being the same node Created: 06/Aug/14  Updated: 06/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.0, 2.4.0
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Minor
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Connecting to "localhost" will not reuse the connection if a node named "127.0.0.1" appears in the config. Unfortunately fixing this would mean determining how to handle hostnames with different resolving IPs, and similar.

 Comments   
Comment by Mark Nunberg [ 06/Aug/14 ]
In practice this is not a big issue since at most this causes an extra connection to be wasted during initial bootstrap, but may end up causing some confusion during initial log analysis.




[CCBC-487] Provide API to control memory usage of the client Created: 30/Jul/14  Updated: 04/Aug/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The library reserves a sane default of memory for various buffers and such, especially in regards to pooling and the like. Since many of these buffers are per-connection, it may result in high memory usage when many connections and/or instances are used.

The idea here is to provide a "Factor" of memory to use rather than an absolute number because of the high number of settings that may possibly be adjusted within the library regarding memory usage; thus a factor of 1 (the default) will use the default settings, a factor of 0.5 would attempt to reserve only half that amount of space, and so on.




[CCBC-342] Default installation should use libevent plugin Created: 05/Mar/14  Updated: 01/Aug/14

Status: Reopened
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
10:11 < mnunberg> avsej: also. it's not using the libevent backend
10:11 <+avsej> it signs the repository during import
10:11 <+avsej> https://github.com/couchbaselabs/lcbpackage/blob/master/rakefiles/repositories.rake#L143
10:11 < mnunberg> cbc built from: libcouchbase 2.3.0 (rev. 7b11fe7e4f8f23f24861fe82287d7eb10de2c6b4) using libcouchbase: 2.3.0 (select)
10:12 <+avsej> did you install libcouchbase2-libevent?
10:12 < mnunberg> uhmm..
10:12 < mnunberg> i thought the default behavior was for it to pull in libevent
10:12 < mnunberg> no i didn't install it explicitly
10:12 <+avsej> there should be file like ubuntu/config/distributions
10:13 <+avsej> mnunberg, it won't install it implicitly either
10:13 < mnunberg> hrm, that's not good
10:13 <+avsej> so if you need libevent just install it
10:13 <+avsej> libcouchbase2-libevent
10:13 < mnunberg> but if that's hwo previous versions did it, then ok
10:13 < mnunberg> it doesn't matter to me now
10:13 <+avsej> it is documented http://www.couchbase.com/communities/c-client-library
10:14 < mnunberg> another thing i'd like to do in 3.0 is a blanket dependency on libevent. it's the most well tested and stable of our event loops
10:14 < mnunberg> if you don't want libevent, compile from source
10:17 <+avsej> what if people are using libevent1 in their libraries? they will end up having two versions in their app?
10:18 <+avsej> i don't see issue at all
10:18 <+avsej> it is always better to rely on the repository maintainer rather then us
10:18 <+avsej> are you sure we will push new libcouchbase for each new release of libevent?
10:18 < mnunberg> most people installing libcouchbase are using it as a dependency for ruby php python etc.
10:19 < mnunberg> so we should give the most optimal runtime defaults
10:19 <+avsej> right
10:19 <+avsej> in this case they won't use libcouchbase IO at all
10:19 <+avsej> this is why we have libcouchbase2-core package
10:19 <+avsej> without any dependencies and additional blobs
10:19 < mnunberg> not really. most folks are still using the default I/O plugin, whichever it is that ships
10:19 <+avsej> for ruby people just want libcouchbase
10:20 <+avsej> and for python they also have their own
10:20 < mnunberg> that's the "Most minimal packaging configuration", not "Most optimal runtime configuration" :)
10:20 < mnunberg> python optionally elts you use your own I/O but it's not the default
10:20 < mnunberg> for normal sycnrhonous operatin you want the fastest there is
10:20 < mnunberg> and that's what most use cases are
10:21 <+avsej> so you mean that select implementation is the slowest? it adds a little overhead, and we don't have c10k on the client
10:21 < mnunberg> avsej: it's also buggier
10:22 < mnunberg> 14
10:22 < mnunberg> err..
10:23 < mnunberg> mnunberg@mbp15 ~/Source/lcb-master $ git log 2.0.0..HEAD --pretty=oneline plugins/io/select/ | wc -l 14
10:23 < mnunberg> mnunberg@mbp15 ~/Source/lcb-master $ git log 2.0.0..HEAD --pretty=oneline plugins/io/libevent/ | wc -l 4
10:23 < mnunberg> mnunberg@mbp15 ~/Source/lcb-master $ git log 2.0.0..HEAD --pretty=oneline plugins/io/libev/ | wc -l 7
10:24 < mnunberg> on the other hand select is simpler to debug
10:24 < mnunberg> eh, i don't know
10:25 <+avsej> in any event people have easy what to substitute it, and so far select is good as minimal implementation
10:25 <+avsej> the recommended way to install libcouchbase is 'yum install libcouchbase-devel libcouchbase2-libevent'
10:26 <+avsej> so I don't see any issue
10:26 < mnunberg> that's not the most intuitive way of installing a package
10:26 < mnunberg> if my documentation says "I need dependency FOO" i'll go ahead and query my package manager and install "FOO"
10:26 < mnunberg> i'm not gonna jump to some other document telling me what to install
10:27 <+avsej> why not fix documentation then
10:27 < mnunberg> because people don't read documentation
10:27 < mnunberg> as a rule, people only read documentation enough to get them up and running
10:27 <+avsej> just recently i was compiling nginx with some module which has dependency on gd
10:27 <+avsej> and first thing I was tried is yum install libgd-devel which failed
10:28 <+avsej> the correct way was gd-devel
10:29 <+avsej> in any event libevent is not good enough as default for ruby and I against to have this as mandatory dependency
10:29 <+avsej> libevent doesn't allow to release GIL while it is doing IO
10:29 <+avsej> the ruby IO plugin does
10:30 <+avsej> you probably can rename the packages, but please leave ability to install libcouchbase2-core, i.e. without any plugins
10:30 < mnunberg> it's the default for python and php. i think the way i wanted this to begin with was a libcouchbase2 metapackage
10:30 < mnunberg> i.e. libcouchbase2 would install libcouchbase2 and libevent
10:31 < mnunberg> and for things that had "Special" requirements, they could use '-core'
10:31 <+avsej> why not remove libcouchbase2 package at all
10:31 < mnunberg> we don't have such a package afaict
10:31 <+avsej> and update docs to install libcouchbase2-libevent if they depend on it
10:32 <+avsej> right
10:32 <+avsej> no package no problem
10:32 < mnunberg> going back to my point
10:32 < mnunberg> people don't read docs
10:32 < mnunberg> especially if _not_ installing libevent doesn't actually give any kind of error or issues until much later when it's in production
10:32 < mnunberg> and then it's like "Oh, we're crashing"
10:32 <+avsej> https://github.com/couchbase/libcouchbase/blob/master/packaging/deb/control#L39 remove this line
10:32 < mnunberg> so then we ask "Ok, tell me what I/O plugin you're using"
10:32 < mnunberg> "Oh, you're using select. You should use libevent"
10:32 <+avsej> or rather move it to libcouchbase2-libevent sectio
10:33 <+avsej> https://github.com/couchbase/libcouchbase/blob/master/packaging/deb/control#L12 after this one
10:33 < mnunberg> avsej: not this release. these are just plans for a 3.0 or whatever
10:33 < mnunberg> i'd rather not touch or change behavior
10:34 <+avsej> rpm also has equivalent https://github.com/couchbase/libcouchbase/blob/master/packaging/rpm/libcouchbase.spec.in#L34
10:34 <+avsej> i'm okay with solution which is about moving alias 'libcouchbase2' from 'libcouchbase2-core' to 'libcouchbase2-libevent'
10:34 <+avsej> totally agree with it
10:35 <+avsej> seems like the easiest solution
10:35 < mnunberg> avsej: yeah, we'll need to schedule it for next release, like .future
10:35 < mnunberg> let me do this on JIRA
10:35 <+avsej> could you copy this log there?
10:35 <+avsej> or extract the main idea
10:35 < mnunberg> ok

 Comments   
Comment by Mark Nunberg [ 01/Aug/14 ]
Select is stable enough now
Comment by Matt Ingenthron [ 01/Aug/14 ]
I actually don't agree with this. We've had years of using libevent by default. I'm mostly concerned about environments where people don't check the underlying plugin use. Might be good to talk through this on Monday.




[CCBC-402] Provide API for exposing buffer/pool settings (rdb, netbuf) Created: 01/May/14  Updated: 30/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.4.0-dp1
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-461] Unmask NOT_MY_VBUCKET return codes. Created: 26/Jun/14  Updated: 28/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.0
Fix Version/s: 3.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
NOT_MY_VBUCKET error codes should be unmasked in the library and propagated to the user if we were unable to retry the operation for whatever reason. While typically the library _will_ retry the operation, masking this error under a different code makes it more difficult for the application to handle.

Historically we have mapped this error to 'LCB_ETIMEDOUT' which isn't always accurate and makes our timeout mechanism seem buggy (an error called "Timeout" should only be thrown after a specific amount of time has elapsed).

The alternative, LCB_NO_MATCHING_SERVER is also confusing as this is typically the return code _before_ an operation is scheduled - while a NOT_MY_VBUCKET is a definite reply from the server. While in raw C applications it may be possible to treat a NO_MATCHING_SERVER from an operation return value differently than one invoked from within a callback, for most common SDKs there is no distinction - and these two paths require different handling paths.




3.0 API Changes (CCBC-462)

[CCBC-479] Use built-in C89 int/short/long/etc where possible Created: 16/Jul/14  Updated: 16/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There's no good reason to use fixed width types for simple functions that aren't expected to have a specified value. int/unsigned will be sufficient. Leave the fixed width types for things that actually expect large values.




[CCBC-317] Consolidate lcb_list_t and lcb_clist_t Created: 03/Feb/14  Updated: 08/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: 2.3.0-dp2
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We should split our our list implementation to a node type (next/prev) and a list type (which also contains size and some other metadata).




3.0 API Changes (CCBC-462)

[CCBC-467] 3.0: Remove 'sanity check' and 'struct IDs' Created: 28/Jun/14  Updated: 07/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 3.0-beta
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
These functions have not been widely used or maintained. Their purpose was to assist applications in verifying the structure sizes used by the library conformed to that of what their application was expecting. However in reality the structure sizes rarely changed, and when they did change, they only changed in compatible ways so that applications compiled against older versions would never break anyway.

Applications using newer versions of the structures would also never work anyway since the library itself would reject the newer versions at runtime.




3.0 API Changes (CCBC-462)

[CCBC-469] Change all type names to lcb_TYPENAME rather than lcb_typename_t Created: 01/Jul/14  Updated: 01/Jul/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The newer convention is done in most of the 2.4 codebase. Using this convention makes it easier to distinguish types inside source code and also allows the names to be more conformant to POSIX standards which technically prohibits the _t suffix from being used.

We can either remove the old names completely, or place them in deprecated.h.

Probably the most notable difference would be in the lcb_t -- which would need to be renamed to something like lcb_pBUCKET




3.0 API Changes (CCBC-462)

[CCBC-468] 3.0: Remove lcb_timer public API Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The timer API was never really used and should have always been private (its use came in before we started having 'interface attributes' within the library)




[CCBC-462] 3.0 API Changes Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Sub-Tasks:
Key
Summary
Type
Status
Assignee
CCBC-463 3.0: Remove syncmode Technical task Open Mark Nunberg  
CCBC-464 3.0: Error code type should be lcb_ST... Technical task Open Mark Nunberg  
CCBC-465 3.0: Remove lcb_error_callback Technical task Open Mark Nunberg  
CCBC-466 3.0: Remove lcb_get_last_error Technical task Open Mark Nunberg  
CCBC-467 3.0: Remove 'sanity check' and 'struc... Technical task Open Mark Nunberg  
CCBC-468 3.0: Remove lcb_timer public API Technical task Open Mark Nunberg  
CCBC-469 Change all type names to lcb_TYPENAME... Technical task Open Mark Nunberg  
CCBC-479 Use built-in C89 int/short/long/etc w... Technical task Open Mark Nunberg  

 Description   
This is an umbrella task for the various changes which will take place in the 3.0 API. Some of these changes may be introduced as volatile within the 2.4 series, while some involve explicitly removing older APIs.

As a secondary goal of the 3.0 API, we hope to make it mostly API and ABI compatible with the 2.0 series, so that application changes can be gradual and minimal.




3.0 API Changes (CCBC-462)

[CCBC-466] 3.0: Remove lcb_get_last_error Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This function is deprecated and its use can result in false positives, true negatives. Most internals do not set 'last_error', and because there may be multiple things going on within the library, getting the last error does not make sense. Only operation specific error fields should be used.




3.0 API Changes (CCBC-462)

[CCBC-465] 3.0: Remove lcb_error_callback Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This API call has no use other than indicating bootstrap success/failure, and has been deprecated in favor of lcb_bootstrap_callback in 2.4




3.0 API Changes (CCBC-462)

[CCBC-464] 3.0: Error code type should be lcb_STATUS, not lcb_error_t Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0-beta
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In conformance with standards and more consistent names, we should call it lcb_STATUS rather than error; also considering that some codes are not necessarily errors :)




3.0 API Changes (CCBC-462)

[CCBC-463] 3.0: Remove syncmode Created: 28/Jun/14  Updated: 28/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 3.0-beta
Fix Version/s: None
Security Level: Public

Type: Technical task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-336] observe operations may not be retried when NMV received Created: 24/Feb/14  Updated: 26/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.0.0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.3.0-dp2
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
During investigation of an issue, Mark Nunberg found that observe replies with NMV may not be retried in libcouchbase. Further, the differentiation between NMV and -1 (i.e., no node in this position) isn't clear from the way the library currently handles things.

The correct behavior is that on any NMV, the library will retry using an updated configuration. This will happen until timeout.

As a workaround, check for the state with a get() or retry the observe() at an application level until successful. It's recommended to retry with some kind of rate limiting and a maximum.


 Comments   
Comment by Mark Nunberg [ 26/Jun/14 ]
I've filed CCBC-461 which should hopefully provide visibility into NOT_MY_VBUCKET.

For the most common case of observe, which is the durability polling, the not-my-vbucket is effectively retried under the hood (in the sense that the polling will simply retry the observe command to all nodes) and is generally not a problem.

Since the packet format for observe differs from any other operation, the normal retry handling code would need to be fiddled with to make a special case for OBSERVE.

Any idea of how important this is?




[CCBC-251] replace the jira logo for CCBC with something modern Created: 20/Aug/13  Updated: 26/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Trivial
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[CCBC-400] Improve API doc navbar layout Created: 29/Apr/14  Updated: 26/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Trivial
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

- If I click on "Introduction" from the "Couchbase C Client" the left hand menu folds up.
- Introduction is in a funny place in the list on the left.





[CCBC-448] Explicitly search /usr/local for newer OS X Created: 12/Jun/14  Updated: 13/Jun/14

Status: Reopened
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.3.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Wei-Li Liu Assignee: Wei-Li Liu
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Mac OS X


 Description   
I clone libcouchbase repo and build.with autotool on my mac
The instruction is :

$ ./config/autorun.sh
$ ./configure
$ make
$ make check
$ make install

On the 2nd step (./configure) I see
====================================================
checking for libev3... no
checking for libev4... no
checking for libevent >= 1.4... no
checking for libevent2... no
checking for libuv... no
configure: error: Failed to locate usable event library. You must install either libev or libevent dev package or use --disable-plugins option
====================================================


However, I do have libevent installed
=====================================================
Wei-Lis-MacBook-Pro:libcouchbase wei-li$ brew install libevent
Warning: libevent-2.0.21 already installed
Wei-Lis-MacBook-Pro:/ wei-li$ sudo find . -name libevent
Password:
find: ./dev/fd/3: Not a directory
find: ./dev/fd/4: Not a directory
./Users/wei-li/repo/couchbase/libcouchbase/plugins/io/libevent
./usr/local/Cellar/libevent
./usr/local/Library/LinkedKegs/libevent
./usr/local/opt/libevent
====================================================
configure script wasn't able to find the package
 






 Comments   
Comment by Mark Nunberg [ 12/Jun/14 ]
Please ensure you have the development libraries for libevent installed as well. It seems you have this available from homebrew, however your homebrew installation is not inside the default linker/compiler search paths.

Find an 'event.h' :)
Comment by Dave Rigby [ 13/Jun/14 ]
@Wei-Li

To be more specific, brew installs things in /usr/local/ by default, and so you need to add them to your search path if building from source. Try:

    CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure && make
Comment by Mark Nunberg [ 13/Jun/14 ]
or 'brew link libevent'. Can you find 'libevent.dylib'?
Comment by Dave Rigby [ 13/Jun/14 ]
@Mark: "brew link libevent" just puts things in /usr/local (at least on my box), which the lcb configure scripts don't look in by default (hence me specifying it).
Comment by Mark Nunberg [ 13/Jun/14 ]
mnunberg@mbp15 ~ $ cpp --verbose
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin12.5.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.8.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 224.1 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0 -I /usr/include -fdebug-compilation-dir /Users/mnunberg -ferror-limit 19 -fmessage-length 134 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.8.0 -fobjc-dispatch-method=mixed -fobjc-default-synthesize-properties -fencode-extended-block-signature -fdiagnostics-show-option -fcolor-diagnostics -traditional-cpp -o - -x c -
clang -cc1 version 5.0 based upon LLVM 3.3svn default target x86_64-apple-darwin12.5.0
ignoring duplicate directory "/usr/include"
  as it is a non-system directory that duplicates a system directory
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
Comment by Mark Nunberg [ 13/Jun/14 ]
It would be odd if you had to add -I/usr/local/include -- in any event, see what homebrew does (brew install libcouchbase?)
Comment by Dave Rigby [ 13/Jun/14 ]
@Mark - /usr/local/include has been removed in 5.1:

$ cpp --verbose
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.2.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 236.3 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -I /usr/include -fdebug-compilation-dir /Users/dave/Documents/Code -ferror-limit 19 -fmessage-length 233 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fdiagnostics-show-option -fcolor-diagnostics -vectorize-slp -traditional-cpp -o - -x c -
clang -cc1 version 5.1 based upon LLVM 3.4svn default target x86_64-apple-darwin13.2.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /usr/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks (framework directory)
End of search list.
Comment by Wei-Li Liu [ 13/Jun/14 ]
Adding the search path when build works for me

CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./configure && make

I have to switch to root user to resolve some permission issue after that steps. Eventually it builds successfully .
I wasn't sure if libcouchbase configure scripts doesn't look in by default. Now It is confirmed I will resolve this

Comment by Mark Nunberg [ 13/Jun/14 ]
Libcouchbase doesn't explicitly look in /usr/local because on any sane system that path is _already_ part of the default compiler search path. You should keep this open and rename the title to something like "explicitly search /usr/local for newer OS X"
Comment by Wei-Li Liu [ 13/Jun/14 ]
Done




[CCBC-415] IPv6 Functionality Created: 14/May/14  Updated: 02/Jun/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: New Feature Priority: Major
Reporter: Haster Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If I put some ipv6 address (for example "::1") function lcb_host_parse(...) returns LCB_INVALID_HOST_FORMAT error.
It is needed to add ipv6 support


 Comments   
Comment by Mark Nunberg [ 14/May/14 ]
I'm not sure how well ipv6 works up the stack tbh. I'm curious though - did this work in lcb 2.2?
Comment by Haster [ 14/May/14 ]
Mark, I don't know =)
But I suggest that it don't work in 2.2 version,
But in libcouchbase there is some things, related to ipv6, such as function lcb_behavior_set_ipv6 in cntl.c

Maybe it's not a bug but future request or task =)




[CCBC-324] Core dump script analyser Created: 13/Feb/14  Updated: 24/Apr/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Major
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Reading core dumps can be painful without access to the machine that created them.

On the server we have a script that allows a user to grab some useful information about the core dumps from memcached.

I think it would be useful to have the same thing for the library.

 Comments   
Comment by Mark Nunberg [ 24/Apr/14 ]
can you link to that script so I can possibly modify it?
Comment by Patrick Varley [ 24/Apr/14 ]
I made one of these today:

backtrace="thread apply all bt"
echo $backtrace > bt
gdb -q -batch -n -x bt /usr/bin/python --core core.2652 >BT.gdb

A link to the server one which looks a lot smarter:
https://github.com/couchbase/ep-engine/blob/master/management/cbanalyze-core

Comment by Matt Ingenthron [ 24/Apr/14 ]
On solaris, you type pstack <corefile>. And, you can do it on any machine (symbols are included). :)




[CCBC-359] add feature test ensuring that E2BIG is returned on append above 20MB Created: 07/Apr/14  Updated: 07/Apr/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0, 2.3.0-dp2
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on MB-10778 Append do not return the correct erro... Closed

 Description   
When continuing to append beyond the maximum value of 20MByte, we should verify that applications receive the correct error response.




[CCBC-277] Provide pause_http_request, resume_http_request Created: 07/Oct/13  Updated: 25/Feb/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
These functions will stop and resume the given request's activity on the event loop. It would be nice if we could implement something like this: https://twistedmatrix.com/documents/11.1.0/core/howto/producers.html#auto8




[CCBC-302] cbc verify exit code Created: 12/Dec/13  Updated: 25/Feb/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.2.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Trivial
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The exit value for cbc verify is 0 even when the files differ. For example...

ingenthr-mbp:~ ingenthr$ dd if=/dev/urandom of=afile count=1024 bs=1024
1024+0 records in
1024+0 records out
1048576 bytes transferred in 0.082984 secs (12635886 bytes/sec)
ingenthr-mbp:~ ingenthr$ cbc create -f 3735928559 afile < afile
Stored "afile" CAS:cae1d52a4f000000
ingenthr-mbp:~ ingenthr$ cbc verify afile
ingenthr-mbp:~ ingenthr$ dd if=/dev/urandom of=afile count=1024 bs=1024
1024+0 records in
1024+0 records out
1048576 bytes transferred in 0.084461 secs (12414917 bytes/sec)
ingenthr-mbp:~ ingenthr$ cbc verify afile
Content differ: "afile"
ingenthr-mbp:~ ingenthr$ echo $?
0


By the way, that should be "contents differ".




[CCBC-292] Documentation/blog on how to use cbc-pillowfight Created: 12/Nov/13  Updated: 25/Feb/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: docs
Affects Version/s: 2.2.0
Fix Version/s: None
Security Level: Public

Type: Task Priority: Minor
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: customer, pillowfight, stats
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There is no documentation on how to use and read the output of cbc-pillowfight.
An article explaining how to use pillowfight and cbstats timings together would be useful.

 Comments   
Comment by Matt Ingenthron [ 12/Nov/13 ]
Note that pillowfight is a subcommand to cbc, and thus it's in that man page:
http://www.couchbase.com/autodocs/couchbase-c-client-2.2.0/cbc.1.html

Agreed that the output could be better described.




[CCBC-257] allow for setting bootstrap nodes via a configuration file, including dynamic updates Created: 27/Aug/13  Updated: 10/Jan/14

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.1.1
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: unibrow
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, bootstrap parameters may be supplied only via arguments to the constructor. A feature should be added to allow for URLs to be used for bootstrap to be configured via a properties file or something along those lines. This should be able to be edited, and then picked up, during a given client object's lifetime.

 Comments   
Comment by Perry Krug [ 27/Aug/13 ]
And this would need integration with the higher level languages (php/python/ruby/etc)
Comment by Sergey Avseyev [ 27/Aug/13 ]
Higher level languages have their own config systems. It isn't hard to pick any format you have in your application and initialize the instances. IMHO it isn't task of libcouchbase. We are trying to keep it simple and do not bloat with unnecessary features.

For example, you want to write command line utilities for couchbase, and you'd like to use libcouchbase to implement this task. Obviously having some rc file in home directory to configure these command line utilities will be awesome. And you must develop or integrate some solution for your particular project. We have done it for CBC, see man page cbcrc(4)
http://www.couchbase.com/autodocs/couchbase-c-client-2.1.1/cbcrc.4.html

We are building network library which will allow you to communicate you over couchbase protocols. There are better libraries for parsing configurations, I don't think we need to implement and maintain one in libcouchbase.

There is one place already, which used to keep configuration --the cluster itself, if we will bundle another one into libcouchbase, it will be be eventually out of sync with cluster configuration. And as soon as it will be our solution we will have to make it consistent and implement something which will keep it up to date
Comment by Perry Krug [ 27/Aug/13 ]
Thanks Sergey, but maybe we're not seeing the same thing here.

This bug is simply referring to the need for updating the internal list of bootstrap nodes in libcouchbase without updating the config string and restarting the application. Matt can provide more insight into the goal here.

My comment about the higher level languages was just to remind ourselves that whatever we build into libcouchbase needs a way of being controlled from those as well.
Comment by Mark Nunberg [ 10/Jan/14 ]
The idea here is to add another 'dynamic file' which would be used to contain the bootstrap nodes? I feel this would honestly be better managed by an external configuration system rather than in the library itself, as it would not be in sync with a common "properties" format for whatever HLL was wrapping LCB.
Comment by Perry Krug [ 10/Jan/14 ]
Mark, The problem we're trying to solve is the occasional need to update the "bootstrap" list of nodes without changing code or restarting the running instances of the library. Think about when the entire set of nodes of a cluster are removed and swapped with different IP's...the next time an instance starts up, it needs to get the updated list.

I don't think we're particularly tied to an extra file, but keep in mind that libcouchbase is sometimes used all by itself, so I believe it also needs to provide this facility as well as the higher level wrappers such as php/python/etc.

Does that make more sense?




[CCBC-259] libuv plugin does not track real work vs idle work. Created: 27/Aug/13  Updated: 28/Aug/13

Status: Open
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: 2.1.2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Brett Lawson Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The libuv plugin for libcouchbase does not properly keep track of 'active work' vs 'idle work' when dispatching libuv watchers. This causes an issue where the libuv event loop will continue idling even though there is no real work to complete. This particularly affects the node.js driver as a script that no longer has any work left will never exit if a Couchbase connection is still open (even though the connection is idle).




[CCBC-100] Allow Incremental Row Callbacks Created: 11/Sep/12  Updated: 09/Aug/13

Status: In Progress
Project: Couchbase C client library libcouchbase
Component/s: library
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
View query results may return large amounts of data. We should offer a row callback which returns a JSON string whose contents is a single row.

There are awesome JSON libraries to do this task, and it's something that I've found myself doing in the Perl client.. so why not move it into libcouchbase and let everyone be happy? :).

Basic idea:
Two callbacks are put into place. The first is a 'row callback' which receives rows, and the second is a complete callback, at which time any other non-row data is returned as a JSON object.

So for example

{ "total_rows" : 30,
  "rows" : [
   { "id" : "foo" },
   { "id" : bar" },
     ....

The row callback will first receive the string '{ "id" : "foo" }', then the second time around, '{"id":"bar"}'
The complete callback will return:
' { "total_rows" : 30 } '





Generated at Sat Oct 25 23:30:30 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.