[PYCBC-144] Support PyPy Created: 24/Jun/13  Updated: 24/Oct/13

Status: In Progress
Project: Couchbase Python Client Library
Component/s: library
Affects Version/s: 0.10.0
Fix Version/s: None
Security Level: Public

Type: Epic Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
is duplicated by PYCBC-183 Cannot run on PyPy Resolved

I've made some experiments in supporting PyPy.

Supposedly it's CPython compatible -- but not really.

I've started a branch on my own github called https://github.com/mnunberg/couchbase-python-client/tree/pypy

As of now there is no ETA for support - but I'd like to see it working with PyPy as well.

Comment by Mark Nunberg [ 30/Jul/13 ]
Why is this resolved? PyPy is still not supported
Comment by Jai Gupta [ 26/Sep/13 ]
Is this planned to be fixed?
Comment by Mark Nunberg [ 26/Sep/13 ]
I'd like to have it open, but it will require a substantial amount of work to make it work with PyPy. PyPy's C extension layer is sorta broken, so it would require various workarounds in our code in order to play nice with whatever PyPy has.
Comment by Jai Gupta [ 27/Sep/13 ]
When we tested Couchbase python library last time, it used to work with PyPy. I think at that time it was in pure Python. I am not sure though.

Now that Couchbase python library is stable, it does not support PyPy. It is bad for our project.

Is there any workaround? Or, should we run older version of Couchbase python library?

PyPy makes huge difference to our application.
Comment by Mark Nunberg [ 20/Oct/13 ]
So i've done some more work regarding PyPy support. It's leaky and buggy, but 1.2 will have better support than previously. The core problem is that a lot of the stuff the extension does is implemented in pure C. It's easier to maintain and links better with libcouchbase.

* Currently PyPy offers 'cpyext' which is buggy, not to mention being slow. The support in place now relies on CPyExt.

* Changing the code to be pure python would be a bad idea since the overhead doesn't come from a single algorithm but from the overall structure and options provided in our code, not to mention the interaction with libcouchbase. Additionally, the C structure is proven quite well with multiple use cases in Twisted and gevent.

In this category is included interactions with 'cffi' which would be cumbersome and buggy, since all the libcouchbase structures would need to be enumerated properly.

A possible workaround in this case is to maintain the C logic "as-is" and implement the problematic bits in Python, perhaps bridging between the two using cffi - in the case of cffi, we'd only need to bridge the C structures internal to the extension (which are simpler and less numerous than what libcouchbase has)
Comment by Mark Nunberg [ 24/Oct/13 ]
I've started a new project to implement much of the functionality using the 'cffi' module which is supported by PyPy:


[PYCBC-244] Python SDK connection object host list usage not clear Created: 06/May/14  Updated: 02/Jul/14

Status: In Progress
Project: Couchbase Python Client Library
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Improvement Priority: Major
Reporter: Ian McCloy Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


The documentation for the connection object has the following ->
host (string or list) – the hostname or IP address of the node. This can be a list or tuple of multiple nodes; the nodes can either be simple strings, or (host, port) tuples (in which case the port parameter from the method arguments is ignored).


It wasn't clear to me how the sdk expected the list of hosts to be provided, only by reading the C client could I see it was expecting a semi-colon separated list of hosts. I would recommend that the documentation specifically says a semi-colon separated list of hosts and provide an example which has a list of hosts. Square bracket standard python lists also seem to work. Perhaps these can also be shown as an example.

Comment by Matt Ingenthron [ 06/May/14 ]
Ian: Thanks so much for the feedback on the docs. I've seen these across a number of the projects and it's very helpful to us to get this kind of feedback.
Comment by Mark Nunberg [ 06/May/14 ]
Actually the intent here is to provide a natural "List" for Python containing various host-port pairs. The semicolon is only supposed to accidentally work, because libcouchbase happens to handle it, and the client, internally, joins the various hosts in the list using the semicolon. I will try to offer some examples for how this can be done.
Comment by Mark Nunberg [ 07/May/14 ]

[PYCBC-254] Lock TTL=0 fails. Created: 29/Jul/14  Updated: 30/Jul/14

Status: In Progress
Project: Couchbase Python Client Library
Component/s: library
Affects Version/s: 1.2.2
Fix Version/s: 1.2.3
Security Level: Public

Type: Bug Priority: Major
Reporter: Patrick Varley Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

The documentation says "If set to 0 it will use the default lock timeout on the server."


However when I use ttl=0 I get this error:

Using PYCBC=1.2.2, LCB=('2.1.2', 131330)
Locking with: lock = cb.lock("lock", ttl=0)
Traceback (most recent call last):
  File "./lock.py", line 12, in <module>
    lock = cb.lock("lock", ttl=0)
  File "/Library/Python/2.7/site-packages/couchbase/connection.py", line 594, in lock
    return _Base.lock(self, key, ttl=ttl)
couchbase.exceptions.ArgumentError: <Lock must have an expiry, C Source=(src/get.c,107)>

Code example:

import couchbase
from couchbase import Couchbase, Connection

print "Using PYCBC={0}, LCB={1}".format(couchbase.__version__, Connection.lcb_version())
cb = Couchbase.connect(bucket='default', host='localhost')

print "Setting"
doc = cb.set("lock","test")
print doc.cas
print "Locking with: " + 'lock = cb.lock("lock", ttl=0)'
lock = cb.lock("lock", ttl=0)
print lock.cas

Comment by Mark Nunberg [ 29/Jul/14 ]
hrm, I'll consider this a documentation bug since lock should always be used with a timeout to make the code easier to read (most things that "Lock", if not passed a timeout, lock indefinitely, while in the server there is always a maximum limit)
Comment by Mark Nunberg [ 30/Jul/14 ]

Generated at Thu Jul 31 10:39:57 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.