Slow view performance with large dataset

Couchbase Server
1

Hi,

I’m getting pretty slow (?) performance from a view query.

We have about 67952374 documents with 8 data servers.

I have the following view to count # of followers of users.

function (doc, meta) {

    if (
        doc &&
        doc.entityType === 'activity' &&
        doc.clientId &&
        doc.target_id &&
        doc.target_form === 'user' &&
        doc.action === 'follow'
    )  {
        emit([doc.clientId,doc.target_id], null);
    }
}

When I give it 10 keys, it’s pretty okay. When I give it 200 keys, avg. response time is between 150~200ms.

Is there a way to improve it?

2

could you share the exact view query API call?
for key based lookups, I’d recommend N1QL - the indexing topologies between views and GSI make a big difference even on 8 nodes.
more details here on the post that describe global and local indexes: http://blog.couchbase.com/2016/may/faster-indexing-and-query-with-memory-optimized-global-secondary-indexes-gsi

thanks
-cihan

3

Hello,

We had a meeting with couchbase to discuss Enterprise license yesterday. I’ve confirmed that 150~200ms is a little bit higher side, but it’s not out of ordinary. We get around 50ms when we pass just 50~100 keys.

Setting limit(200) also helped a little. We get 110~130ms now.

I’m experimenting with 4.5 beta to see if we can get better performance with n1ql.

Thank you!
Moon