[PYCBC-144] Support PyPy Created: 24/Jun/13  Updated: 24/Oct/13

Status: In Progress
Project: Couchbase Python Client Library
Component/s: library
Affects Version/s: 0.10.0
Fix Version/s: None
Security Level: Public

Type: Epic Priority: Major
Reporter: Mark Nunberg Assignee: Mark Nunberg
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
is duplicated by PYCBC-183 Cannot run on PyPy Resolved

I've made some experiments in supporting PyPy.

Supposedly it's CPython compatible -- but not really.

I've started a branch on my own github called https://github.com/mnunberg/couchbase-python-client/tree/pypy

As of now there is no ETA for support - but I'd like to see it working with PyPy as well.

Comment by Mark Nunberg [ 30/Jul/13 ]
Why is this resolved? PyPy is still not supported
Comment by Jai Gupta [ 26/Sep/13 ]
Is this planned to be fixed?
Comment by Mark Nunberg [ 26/Sep/13 ]
I'd like to have it open, but it will require a substantial amount of work to make it work with PyPy. PyPy's C extension layer is sorta broken, so it would require various workarounds in our code in order to play nice with whatever PyPy has.
Comment by Jai Gupta [ 27/Sep/13 ]
When we tested Couchbase python library last time, it used to work with PyPy. I think at that time it was in pure Python. I am not sure though.

Now that Couchbase python library is stable, it does not support PyPy. It is bad for our project.

Is there any workaround? Or, should we run older version of Couchbase python library?

PyPy makes huge difference to our application.
Comment by Mark Nunberg [ 20/Oct/13 ]
So i've done some more work regarding PyPy support. It's leaky and buggy, but 1.2 will have better support than previously. The core problem is that a lot of the stuff the extension does is implemented in pure C. It's easier to maintain and links better with libcouchbase.

* Currently PyPy offers 'cpyext' which is buggy, not to mention being slow. The support in place now relies on CPyExt.

* Changing the code to be pure python would be a bad idea since the overhead doesn't come from a single algorithm but from the overall structure and options provided in our code, not to mention the interaction with libcouchbase. Additionally, the C structure is proven quite well with multiple use cases in Twisted and gevent.

In this category is included interactions with 'cffi' which would be cumbersome and buggy, since all the libcouchbase structures would need to be enumerated properly.

A possible workaround in this case is to maintain the C logic "as-is" and implement the problematic bits in Python, perhaps bridging between the two using cffi - in the case of cffi, we'd only need to bridge the C structures internal to the extension (which are simpler and less numerous than what libcouchbase has)
Comment by Mark Nunberg [ 24/Oct/13 ]
I've started a new project to implement much of the functionality using the 'cffi' module which is supported by PyPy:


[PYCBC-244] Python SDK connection object host list usage not clear Created: 06/May/14  Updated: 02/Jul/14

Status: In Progress
Project: Couchbase Python Client Library
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Improvement Priority: Major
Reporter: Ian McCloy Assignee: Mark Nunberg
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


The documentation for the connection object has the following ->
host (string or list) – the hostname or IP address of the node. This can be a list or tuple of multiple nodes; the nodes can either be simple strings, or (host, port) tuples (in which case the port parameter from the method arguments is ignored).


It wasn't clear to me how the sdk expected the list of hosts to be provided, only by reading the C client could I see it was expecting a semi-colon separated list of hosts. I would recommend that the documentation specifically says a semi-colon separated list of hosts and provide an example which has a list of hosts. Square bracket standard python lists also seem to work. Perhaps these can also be shown as an example.

Comment by Matt Ingenthron [ 06/May/14 ]
Ian: Thanks so much for the feedback on the docs. I've seen these across a number of the projects and it's very helpful to us to get this kind of feedback.
Comment by Mark Nunberg [ 06/May/14 ]
Actually the intent here is to provide a natural "List" for Python containing various host-port pairs. The semicolon is only supposed to accidentally work, because libcouchbase happens to handle it, and the client, internally, joins the various hosts in the list using the semicolon. I will try to offer some examples for how this can be done.
Comment by Mark Nunberg [ 07/May/14 ]

Generated at Wed Jul 23 07:19:33 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.