open files/sockets

i'm quite new in node js/couchbase.
in my project i try to make a small web app to query data from a couchbase cluster. after some test i've realised (because of some mysterious errors) the app keeps open connections (files and or sockets) and the os limit (linux - 1024) is reached quite soon.
(i could higher the limit, but it seems it will increase all the time!)

i tried to debug but no success. i tried to close the couchbase connection but it was even worst.

here is my (test) code:

var http = require('http');
var couchbase = require('couchbase');
var config = {
    "host" : "10.21.11.45:9999",
    "bucket" : "tBucket",
    "password" : "xxx",
    "operationTimeout" : 20000,
    "connectionTimeout" : 20000
  };
 
 
var view = function(docname, viewname, viewopt, callback){
   var o = this;
   var cl = new couchbase.Connection(config, function(err) {
     if (err){
        if (cl)
          //cl.shutdown();
        callback (result);
     } else {
       var q = cl.view(docname, viewname, viewopt);
       q.query(function(err,result) {
         //cl.shutdown();
         callback(result);
       });
     }
     return;
   });
}
 
http.createServer(function (req, response) {
 
  var docname = 'spamdoc';
  var viewname = 'targets';
  var viewopt = {};
 
  view(docname, viewname, viewopt, function(res){
    if(res.err) {
      response.writeHead(200, {'Content-Type': 'text/plain'});
      response.end('ERROR\n');
    } else {
      try {
        console.log('couchbase view ok');
        response.writeHead(200, {'Content-Type': 'text/plain'});
        response.end('View:\n'+res);
      } catch (e) {
        response.writeHead(200, {'Content-Type': 'text/plain'});
        response.end('ERROR\n');
      }
    }
  });
}).listen(1337);
console.log('Server running at http://127.0.0.1:1337/');

the console output:

>>node t.js &
Server running at http://127.0.0.1:1337/
>>lsof | grep -w node |wc -l
298
>> couchbase view ok
>>lsof | grep -w node |wc -l
301
>> couchbase view ok
>>lsof | grep -w node |wc -l
304

2 Answers

« Back to question.

Hey fmarci,

Each time you execute your view function, it is opening a new connection the server which involves quite a large number of socket connections. Additionally, due to a flaw in the implementation of the Node.js driver, connections can take a great deal of time to be garbage collected which is contributing to your issue. Please try instantiating one connection in a `global` scope and using that throughout the lifetime of your application.

Cheers, Brett

« Back to question.

Hello,

Couchbase clients (so Node.js/libcouchbase) open sockets between your application and all the nodes of your cluster.

Can you give us more information about:
- your cluster
- code of your application
- type of request...

I am surprised when you say that the number of sockets is growing all the time..

It is important that you keep your connection opened in your application since the "connection process" is a long one:
- get the topology using REST calls
- open sockets to all the nodes...

Regards
Tug
@tgrall