Connection issue with Couchbase 6.5.1 and Python 3.0.0

Python SDK
1

I am trying to use async API of Couchbase. I have following code with FastAPI farmework:

from acouchbase.cluster import Cluster, Bucket
from couchbase.cluster import PasswordAuthenticator

@app.on_event("startup")
async def startup():
    # Do not connect to db elsewhere
    # await db.connect()
    cluster = Cluster(
        "couchbase://localhost",
        authenticator=PasswordAuthenticator("username", "password"),
    )

However, I get following error:

ERROR: Future exception was never retrieved
future: <Future finished exception=ObjectDestroyedException(‘Connection object was garbage collected’)>
couchbase.exceptions.ObjectDestroyedException:

2

@shivshankar_dayal I see a couple of minor issues may be that helps

from acouchbase.cluster import Cluster, Bucket

remove the a and it should be from couchbase.cluster import Cluster, Bucket

secondly,

cluster = Cluster(
“couchbase://localhost”,
authenticator=PasswordAuthenticator(“username”, “password”),
)

Should be

cluster = Cluster(‘couchbase://localhost’, ClusterOptions(PasswordAuthenticator(‘username’, ‘password’)))

See if that helps !

3

I have already tried that and it works. But then my operations will become synchronous blocking FastAPI event loop. Also, I see that acouchbase.cluster.Cluster fundamentally inherits from couchbase.clucter.Cluster

4

@ellis.breen can you please help ?

5

Hi, ‘cluster’ is being destroyed some time between returning from the Cluster constructor call, and this error is therefore appearing. This appears to be because ‘cluster’ is not stored anywhere - so the solution would be to do this (e.g. self.cluster=Cluster(...). You will need the Cluster object later to perform any operations anyway, so maybe I am misunderstanding what you are doing with cluster?

6

I got it working in the sense that object destruction exception is no longer there. However, it refuses to perform any operation. It raises the exception LCB_NO_CONFIGGURATION.

So in file db.py I have following:

from couchbase.cluster import Cluster, Bucket
from couchbase.cluster import ClusterOptions
from couchbase.cluster import PasswordAuthenticator

cluster = None

if cluster == None:
    print("Connecting to cluster")
    cluster = Cluster.connect(
        "couchbase://localhost",
        authenticator=PasswordAuthenticator("shiv", "yagyavalkya"),
    )

I use this in app.py as:

from db import cluster
@app.on_event("startup")
async def startup():
     assert cluster is not None
7

I got it working though I do not like how it works because now in every endpoint I have to call a function to get database handle. If anyone is interested I can share my code.

8

Hi @shivshankar_dayal - glad you got it working - of course you cannot operate on the cluster until the on_connect callback has fired (although most cluster operations will automatically requeue themselves should this be the case, particularly since the latest PYCBC/LCB update) - so this is probably why you were seeing the LCB_NO_CONFIGURATION errors.

Normally one would store the ‘cluster’ variable somewhere convenient like a member variable of a class - this is what I would suggest you do - but feel free to share your code if this would be hard to implement.

9

@ellis.breen My app is working, however, when I tried to run test client I again ran into LCB_ERR_NO_CONFIGURATION issue. I see that cluster does not connect immediately nor I can await for it as connect is a plain sync function. I do not fully understand the Python code because lots of it is based on libcouchbase which I am unfamiliar with. I do not understand what or when on_connect callback is called. How can I ensure that cluster is connected to before proceeding forward. I see that connected is false. For example, following code does not work:

import asyncio
from acouchbase.cluster import Cluster
from couchbase.cluster import ClusterOptions
from couchbase.cluster import PasswordAuthenticator

async def do_crud_op():
    cb = Cluster.connect("couchbase://localhost", options=ClusterOptions(PasswordAuthenticator("user", "password")))
    cb = cb.bucket('customers')
    await cb.upsert('id', {'some': 'value'})
    return await cb.get('id')

loop = asyncio.get_event_loop()
rv = loop.run_until_complete(do_crud_op())
print(rv.value)
10

Hi, the ‘on_connect’ callback fires when the connection has been completed to a bucket/cluster. Calling cb.on_connect() will return an asyncio.Future which can be await ed:

from acouchbase.cluster import Cluster
from couchbase.cluster import ClusterOptions
from couchbase.cluster import PasswordAuthenticator

async def do_crud_op():
    cb = Cluster.connect("couchbase://localhost", options=ClusterOptions(PasswordAuthenticator("user", "password")))
    cb = cb.bucket('customers')
    await cb.on_connect()
    await cb.upsert('id', {'some': 'value'})
    return await cb.get('id')

loop = asyncio.get_event_loop()
rv = loop.run_until_complete(do_crud_op())
print(rv.value)

I did think we had added code to automatically chain an ‘await on_connect()’ to most KV operations but will recheck this.

There were also some issues with operation retries prior to bootstrap completing that might cause you to see the LCB_NO_CONFIGURATION error which were fixed in the recent LCB 3.0.2 release (included with PYCBC 3.0.1) - please ensure you have this installed.

Hope that helps,

Ellis

11

Again I do not understand this. The function on_connect is a sync function as defined on line no. 68 of acouhbase/cluster.py. However, it makes the code work. As far as I understand it has to be an async function before one can await it.

12

Hi, one can in fact await any awaitable object, which includes the Future objects that any async function returns (this is the magic of the async syntactic sugar), and which the cluster/bucket.on_connect() function (which happens to be synchronous) returns. I may adapt the docs to mention this, as the awaitability of Future objects isn’t necessarily obvious. Glad this has made it work . As stated, I thought we’d added a requeue command to KV operations at least to prevent the need for this, but will recheck that.