Performance of big amount of indexes in Cluster configuration

Hi people!

We’re playing around with cluster on Couchbase 4.6, trying to tune performance. At the moment we’ve got 59 indexes, with average 3 fields in predicate. Our cluster has 2 index nodes, and Cluster Index RAM quota = 4Gb. The size of bucket is 100K documents. We replicated each index into two nodes, thus obtaining in total 118 indexes divided on 2 nodes.

We observe the following: average query execution delay around 2-3 second. This shows up even on non-loaded cluster, i.e. just with a single query.
Our concern was that we may be stumbled with the Index RAM quota. To estimate how much disk space our index(es) could take we used Sizing Guidelines ( which states - for 20M documents index on 1 field will take 6Gb of disk. The rest being equal, we calculate it for our case:
~ 100K * (100 indexes)*3 = 30M documents. So the corresponding scale out would be 9Gb.

Now, do the Cluster Index RAM quote is the sum of node’s index RAM quota(s) - i.e. if we set Cluster index RAM quota = 9Gb does it mean each node will get 4,5 Gb? However increasing Cluster Index RAM Quota = 9Gb doesn’t help. We still get same delays. What are we doing wrong?

And one more question, as our test experience query delay~ 2-3 seconds, the Elapsed and Execution times in response show ~200ms. I wonder how could they mismatch, as the execution time is basically time between query arrival and results appearance, right? It would take full query processing (including index) and thus would also be 2-3 seconds?