Why a simple query is slow on nodejs?

Node.js SDK
1

Hello,

I have a simple query and it takes 240ms - 280ms to execute.

In my shell on my capella dashboard it takes 3ms.

config:

import {
  Bucket,
  Cluster,
  Collection,
  connect,
  GetResult,
  Scope,
} from 'couchbase';
import * as Sentry from '@sentry/node';

export const cbdb = async () => {
  try {
    const cluster: Cluster = await connect('couchbases://cb.xxx.cloud.couchbase.com', {
      username: 'xxxx',
      password: 'xxxxx!',
		  // Use the pre-configured profile below to avoid latency issues with your connection.
		  configProfile: "wanDevelopment",
    });

    const bucket: Bucket = cluster.bucket('bucket');

    const scope: Scope = bucket.scope('scope');

    console.log('COUCHBASE CONNECTED');

    return scope;
  } catch(e) {
    console.log('CBB')
    console.log(e);
    Sentry.captureException(`${e}`);
    return null;
  }
};

server.ts

import express from 'express';
import { cbdb } from './config/db';

const app = express();

const db = await cbdb();

app.get('/', async(req, res) => {
  try {
    console.time("concatenation");
    const d = await db?.query('SELECT click FROM products WHERE META().id = $1', {
      parameters: ['products1']
    });
    console.timeEnd("concatenation");

    res.send('hello');
  } catch(e) {
    console.log(e);
    res.send(e);
  }
})

const port = process.env.PORT || 3000;
const server = app.listen(port, async () => {
  console.log(`server started on port: ${port}`);
});
2

Some (all?) of the SDKs don’t connect to the cluster until they need to for the purpose of executing an operation, so the first execution will also include the connection time. Measure the second execution of the query. Alternatively, call WaitUntilReady (which also connects to the cluster) before starting the timer.

3

All requests take 240-280ms, also the second or the third execution.

I start the server and connect the cluster and when the cluster is ready then I execute the query after 1min I execute also the query more then one time it everytime takes 240-280ms.

€: Where I can set waitUntilReady ß

€2: I use a trial capella service, I am located in germany and the cloud is in ireland. When I select this in shell in the capella UI it takes 4ms

4

Can you post your code that does this, just to be sure we are talking about the same thing?

5

its the code above I dont know where I set waitUntilReady, I mean I do not create a new connection, when I start express I await cbdb() so I call it once. How should I rewrite this with your meaning ?

6

[Edit: my mistake, there is no WaitUntilReady in nodejs SDK]

I have a simple query and it takes 240ms - 280ms to execute.

In my shell on my capella dashboard it takes 3ms.

Maybe your client is 240ms away from your server?

7

This is my second Account on my phone because I Forgot my Password

When I make other calls with fetch or other dB calls from my other dB its fast and it Takes 10-50ms

The kv Operation takes 50ms

And I cant See the waitUntilReady Class in nodejs

8

[Edit: My apologies. There is no WaitUntilReady in the nodejs sdk. No matter - if you have other queries executing in a few ms, then there is no need to wait long for initialization]

SELECT click FROM products WHERE META().id = $1’

That’s really something that should be done with the kv api.

9

with kv it takes 50-80ms so its also slow,

with ottomann it takes 2-4ms

when I use ottoman it goes very fast,so anything with my code is wrong

10

Ottoman uses either the kv api or the query api. So it doesn’t seem possible that Ottoman can be faster than the kv api and the query api.

11

With ottoman and I used the query statement not KV. (2ms)
ss

With node sdk (240ms)
ss2

12

Why don’t you print out some of the profile timings from the QueryResult?

13

I figure it out, he creates always a new connection idk why…

when I do this

let cached = global.couchbase

if (!cached) {
  cached = global.couchbase = { conn: null }
}

async function createCouchbaseCluster() {
  if (cached.conn) {
    return cached.conn
  }

    cached.conn = await couchbase.connect('xxx', {
      username: 'xxx',
      password: 'xxxxx',
    })
  

  return cached.conn
}

the first takes 240ms
the second and all other takes 20ms.
but I think this is not a good solution or do I get any disadvantages when I use it as global cache in nodejs ?

14

Huh.
‐-----------‐-----

15

Sorry I did not understand it correctly :frowning:

but how can I solve this issue now ? is there a good solution ?

16

Couchbase Node.js travel-sample Application REST Backend - GitHub GitHub - couchbaselabs/try-cb-nodejs

17

Tried and again takes 240-280ms every execution.

with KV it takes 40ms its fast:

app.get("/profile/:pid", async (req, res) => {
  const { profileCollection, scope, cluster } = await connectToDatabase();

    console.time("concatenation");
    
    await profileCollection.get(req.params.pid)
      .then((result) => res.send(result.value))
      .catch((error) => res.status(500).send({
        "message": `KV Operation Failed: ${error.message}`
      }))
      
    console.timeEnd("concatenation");
})

with query
its slow takes 290ms

app.get("/profile/:pid", async (req, res) => {
  const { profileCollection, scope, cluster } = await connectToDatabase();

    console.time("concatenation");
    const query = `
      SELECT p.*
      FROM ${process.env.CB_BUCKET}._default.profile
    `
    await cluster.query(query)
      .then((result) => res.send(result.rows))
      .catch((error) => res.status(500).send({
        "message": `Query failed: ${error.message}`
      }))
    console.timeEnd("concatenation");

})

conn.js

import * as couchbase from 'couchbase'

const CB_USER = process.env.CB_USER
const CB_PASS = process.env.CB_PASS
const CB_URL = process.env.CB_URL
const CB_BUCKET = process.env.CB_BUCKET
const IS_CAPELLA = process.env.IS_CAPELLA

if (!CB_USER) {
  throw new Error(
      'Please define the CB_USER environment variable inside dev.env'
  )
}

if (!CB_PASS) {
  throw new Error(
      'Please define the CB_PASS environment variable inside dev.env'
  )
}

if (!CB_URL) {
  throw new Error(
      'Please define the CB_URL environment variable inside dev.env'
  )
}

if (!CB_BUCKET) {
  throw new Error(
      'Please define the CB_BUCKET environment variable inside dev.env'
  )
}

if (!IS_CAPELLA) {
  throw new Error(
      'Please define the IS_CAPELLA environment variable inside dev.env. \nSet to \`true\` if you are connecting to a Capella cluster, and \`false\` otherwise.\n'
  )
}

/**
 * Global is used here to maintain a cached connection across hot reloads
 * in development. This prevents connections growing exponentially
 * during API Route usage.
 */
let cached = global.couchbase

if (!cached) {
  cached = global.couchbase = { conn: null }
}

async function createCouchbaseCluster() {
  if (cached.conn) {
    return cached.conn
  }

  if (IS_CAPELLA === 'true') {
    // Capella requires TLS connection string but we'll skip certificate verification with `tls_verify=none`
    cached.conn = await couchbase.connect('couchbases://' + CB_URL + '?tls_verify=none', {
      username: CB_USER,
      password: CB_PASS,
    })
  } else {
    // no TLS needed, use traditional connection string
    cached.conn = await couchbase.connect('couchbase://' + CB_URL, {
      username: CB_USER,
      password: CB_PASS,
    })
  }

  return cached.conn
}

export async function connectToDatabase() {
  const cluster = await createCouchbaseCluster()
  const bucket = cluster.bucket(CB_BUCKET);
  const scope = bucket.scope('_default');
  const collection = bucket.defaultCollection();
  const profileCollection = bucket.collection('profile');

  let dbConnection = {
    cluster,
    bucket,
    scope,
    collection,
    profileCollection,
  }

  return dbConnection;
}

I installed/use this: Developer Portal | Couchbase

the first exectuion is fast on query: 56ms all other exeuction takes 290ms

18

Query discards/recreates http connections that have not been used in the last second.

19

But why ? What I am doing when I use fts where I need query, my response time should not be taken 290ms, thats too slow for searching or for other tasks. How can I prevent this that it not recreate a connection ?

20

Put your client closer to the server. Or your server closer to the client.
You can also set the idle timeout to a larger value.
When you deploy an application serving hundreds of requests per second, the connection will never be idle for one second.