[PYCBC-237] Add page_size option to RowProcessor to limit get_multi sizes Created: 06/Mar/14  Updated: 12/May/14

Status: Open
Project: Couchbase Python Client Library
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This option would be delivered to the RowProcessor implementation (if it's an instance of RowProcessor) to limit the number of documents fetched for a multi-get when dealing with many view results.

Currently if we get several thousand results in a single fetch, it means the include_docs handling with also try to do a get_multi on that many documents.

This will waste much memory and slow down the application.

 Comments   
Comment by Mark Nunberg [ 12/May/14 ]
Note this is possible to work around via user code by implementing a custom RowProcessor




[PYCBC-214] bucket flush needs to be added Created: 06/Jan/14  Updated: 08/Jan/14

Status: Open
Project: Couchbase Python Client Library
Component/s: docs, library
Affects Version/s: 1.1.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
For a given bucket, there needs to be an easy way to invoke the bucket's RESTful flush(). This should be at the bucket level and authentication level. In other words, for a given bucket that I'm doing set()/get() with, I should be able to flush(). That flush() must use the REST interface for the bucket, not broadcast the libcouchbase/memached flush operation.

Note that it is safe for other client instances to be connected during this RESTful flush() operation. Also, it may be worth mentioning in the docs that flush is not atomic on the cluster and will propagate asynchronously.

 Comments   
Comment by Mark Nunberg [ 08/Jan/14 ]
We currently lack an admin interface; we'd be doing this via the admin interface if we were to support it.




[PYCBC-179] add_multi documentation could use improvement Created: 29/Aug/13  Updated: 03/Sep/13

Status: Open
Project: Couchbase Python Client Library
Component/s: docs
Affects Version/s: 1.0.0
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Manfre Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: usability
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The docs don't mention that add_multi will raise KeyExistsError for the last already existing key, if any number of them already exist. If this is intentional behavior, mentioning it in the docs would be useful and that the KeyExistsError.all_results is where most people will need to look for the OperationResult list.

This behavior seems odd for the API. Assuming there is not a connection/server error, always returning the OperationResult dict would be more intuitive and avoid the code pattern of:

try:
  results = bucket.add_multi(keys)
except KeyExistsError as e:
  # ignore exception because I'm going to process the results anyway
  results = e.all_results

# now process/verify 'results'

---
If add_multi always returned an OperationResult dict, the code would simply be:

results = bucket.add_multi(keys)
# process/verify 'results' without the unnecessary exception

 Comments   
Comment by Mark Nunberg [ 03/Sep/13 ]
This may be fixed in subsequent versions, but the *multi API is expected to function identically to the single API with the exception of the return type being a dict rather than a Result object.

What I may do in the future is simply make all exception throwing optional (i.e. at least those exceptions caused by libcouchbase - the library may still throw exceptions in true "exceptional" situations).

Does that sound better?
Comment by Michael Manfre [ 03/Sep/13 ]
Given the API decision of "multi == single, except make the return type make sense for the function", how about changing that to "multi == single, except the return type and raised exceptions make sense for the function"? That API guideline change would open up two options to fix the multi functions.

1. (Preferred Fix) Always return a dict for the multi functions and don't raise the key errors. This is how memcached handles get_multi. Errors should still be raised in true "exceptional" situations.

2. Change the key error to also includes a list of keys that failed, instead of just all_results. While not ideal as always getting a result dict back, this would at least avoid all code from having to duplicate the work done by the couchbase driver, which already knows which keys failed.

----
If you make all exception throwing optional, please do so on a per function basis and not on a per connection basis. Per connection settings are only useful when you control all code that may use the connection, which may prevent using 3rd party packages that made a different 'quiet' choice when writing their code.
Comment by Mark Nunberg [ 03/Sep/13 ]
I'd like to get rid of exception throwing in _all_ situations except for true "unrecoverable" exceptions (for example, invalid format or some other serious error). Otherwise I do agree.

The problem with trying to figure out which keys "failed" is that currently only one exception (i.e. the first failure) is thrown - while the other ones still exist. Because Python cannot really return values when an exception has been thrown, those values are stored in the '.all_results' field of the exception object.

Note that you can use the Item API (though this is not present in 1.0.0, it will be available in 1.1.0) to go back to 'normal' behavior.

Also note that internally, the Couchbase driver doesn't really know which key failed - there is only _one_ failure flag internally for key failures - so that there isn't a huge performance hit if a lot of keys fail.

Feel free to rename this issue as "Don't raise exceptions on key failures", as this involves a lot of annoying boilerplate in any event. What we really should have done is something like:

# Raise on any error
cb.set(key, value, raise=True)

# Raise only on specific failures
cb.set(key, value, raise=[0x18, 0x27, 0x13])

# Don't raise (default)
cb.set(key, value)




[PYCBC-173] syncwait in design_create() expects "views" key Created: 13/Aug/13  Updated: 13/Aug/13

Status: Open
Project: Couchbase Python Client Library
Component/s: library
Affects Version/s: 1.1.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Volker Mische Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When you call design_create() with syncwait > 0 it polls the design document whether they are there or not. The polling seems to check the "views" key/value. The problem is that the design document might not contain any "views", but only spatial views that use "spatial" as key.




[PYCBC-144] Support PyPy Created: 24/Jun/13  Updated: 24/Oct/13

Status: In Progress
Project: Couchbase Python Client Library
Component/s: library
Affects Version/s: 0.10.0
Fix Version/s: None
Security Level: Public

Type: Epic Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by PYCBC-183 Cannot run on PyPy Resolved

 Description   
I've made some experiments in supporting PyPy.

Supposedly it's CPython compatible -- but not really.

I've started a branch on my own github called https://github.com/mnunberg/couchbase-python-client/tree/pypy

As of now there is no ETA for support - but I'd like to see it working with PyPy as well.

 Comments   
Comment by Mark Nunberg [ 30/Jul/13 ]
Why is this resolved? PyPy is still not supported
Comment by Jai Gupta [ 26/Sep/13 ]
Is this planned to be fixed?
Comment by Mark Nunberg [ 26/Sep/13 ]
I'd like to have it open, but it will require a substantial amount of work to make it work with PyPy. PyPy's C extension layer is sorta broken, so it would require various workarounds in our code in order to play nice with whatever PyPy has.
Comment by Jai Gupta [ 27/Sep/13 ]
When we tested Couchbase python library last time, it used to work with PyPy. I think at that time it was in pure Python. I am not sure though.

Now that Couchbase python library is stable, it does not support PyPy. It is bad for our project.

Is there any workaround? Or, should we run older version of Couchbase python library?

PyPy makes huge difference to our application.
Comment by Mark Nunberg [ 20/Oct/13 ]
So i've done some more work regarding PyPy support. It's leaky and buggy, but 1.2 will have better support than previously. The core problem is that a lot of the stuff the extension does is implemented in pure C. It's easier to maintain and links better with libcouchbase.

* Currently PyPy offers 'cpyext' which is buggy, not to mention being slow. The support in place now relies on CPyExt.

* Changing the code to be pure python would be a bad idea since the overhead doesn't come from a single algorithm but from the overall structure and options provided in our code, not to mention the interaction with libcouchbase. Additionally, the C structure is proven quite well with multiple use cases in Twisted and gevent.

In this category is included interactions with 'cffi' which would be cumbersome and buggy, since all the libcouchbase structures would need to be enumerated properly.

A possible workaround in this case is to maintain the C logic "as-is" and implement the problematic bits in Python, perhaps bridging between the two using cffi - in the case of cffi, we'd only need to bridge the C structures internal to the extension (which are simpler and less numerous than what libcouchbase has)
Comment by Mark Nunberg [ 24/Oct/13 ]
I've started a new project to implement much of the functionality using the 'cffi' module which is supported by PyPy:

https://github.com/couchbaselabs/couchbase-python-cffi




[PYCBC-133] Provide class-level methods by which to not print out long values Created: 13/Jun/13  Updated: 13/Jun/13

Status: Open
Project: Couchbase Python Client Library
Component/s: library
Affects Version/s: 0.11.1
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Mark Nunberg [ 13/Jun/13 ]
If a value is several long KBs and it has an exception, the screen will not be happy :)




[PYCBC-204] Release notes for the Windows SDK should have a link to the libcouchbase version used. Created: 04/Nov/13  Updated: 04/Nov/13

Status: Open
Project: Couchbase Python Client Library
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: None
Security Level: Public

Type: Task Priority: Minor
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: customer, documentation
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://docs.couchbase.com/couchbase-sdk-python-1.1/#release-notes-for-couchbase-python-sdk-110-ga-1-october-2013


 Description   
The Python SDK for Windows includes libcouchbase.
The release notes should say which version of libcouchbase it uses and a link to that version's release note.

 Comments   
Comment by Mark Nunberg [ 04/Nov/13 ]
You can determine this at runtime :) - Connection.lcb_version()




Generated at Tue Sep 16 11:27:52 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.