Connection Pooling Node.JS

I have been working with the .Net SDK and the Node.js SDK, and in the .Net SDK @jmorris added a nice helper class to help with connection pooling. I was writing another tool using Node.js and I saw my connection counts balloon, and I realized the same thing need to be done for this SDK. I have written a hacky connection pool, which I will post later, but am I missing something in documentation?

Hey @envitraux,

In Node.js and I believe .NET, you should only be opening a very small number of connections across your whole application (usually 1). Why are you pooling your connections?

Cheers, Brett

Because I was having errors occur with too many connections. I think I was hitting tens of thousands of connections, and then I was getting an error. This was an ETL like operation, where I was running through a lot of docs and updating them

Hey @envitraux,
Why do you have more than a single connection? There is a significant overhead associated with opening new connections. You should only have a handful (usually 1) of Couchbase Server connections at any time.
Cheers, Brett

This is why I was writing code to share a connection among many event driven operations. I will show you my solution soon as I clean it up.

Hey @envitraux,
You should be able to implement a connection manager as a singleton which holds a single couchbase connection. Then simply have all operations use this singleton to access that single connection.

That’s what I did in .Net, then @jmorris rolled a similar thing into the SDK, and I now use that feature. I just thought it would be a nice feature in Node.js

Hey,
My apologies. I assumed you meant maintaining multiple Couchbase connections when you spoke about ‘connection pooling’. In terms of the Node.js SDK, simply created and export your connection from a single .js file, and require it wherever you need to use Couchbase.
Cheers, Brett

I am new to Node.JS but here is the code I use

var cbHelper = { clusterName: cbServer, cluster: null, buckets: [] };

cbHelper.getBucket = function (bucketName, password, callback) {
    
    // make sure we have a reference
    if (cbHelper.cluster == null)
        cbHelper.cluster = new couchbase.Cluster(cbHelper.clusterName);
    
    var bucket = _.find(cbHelper.buckets, function (b) { return b.bucketName == bucketName });

    if (bucket != undefined) {
        bucket.connectionCount++;
        callback(bucket.bucket);
    }
    else {
        bucket = { bucketName: bucketName, connectionCount: 0 };
        bucket.bucket = cbHelper.cluster.openBucket(bucketName, password, function (errOpen) {
            if (errOpen) {
                // Failed to make a connection to the Couchbase cluster.
                winston.log('error', errOpen);
                callback(null);
            }
            else {
                bucket.connectionCount++;
                cbHelper.buckets.push(bucket);
                callback(bucket.bucket);
            }
        });
    }
};

cbHelper.closeBucket = function (bucketName) {
    
    var bucket = _.find(cbHelper.buckets, function (b) { return b.bucketName == bucketName });

    if (bucket != null) {
        bucket.connectionCount--;
        if (bucket.connectionCount <= 0) {
            bucket.connectionCount= 0;
        }
    }

}

Hey @envitraux,

That looks reasonable, however we usually suggest that you instead expose specific buckets rather than being generic as in your case. For instance:

db.js:

var cluster = new couchbase.Cluster(...);
exports.bucket1 = cluster.openBucket('bucket1');
exports.bucket2 = cluster.openBucket('bucket2');

myfile.js:

var db = require('./db');
db.get('key', ...);

Cheers, Brett

How do we use this connection when we use ottoman?

var EmployeeMdl = ottoman.model(‘Employee’, {
employeeID: { type: ‘string’, auto: ‘uuid’ },
name: {
first: ‘string’,
last: ‘string’
},
email: ‘string’
}

Also I don’t see real code for closing the connection and it is just updating the connection count…