We have a use-case where we are using N1QL to read data from a cluster and write the incremental data to ElasticSearch. The script was designed to be executed every few minutes - this allows us to have “live” dashboards within Kibana.
The problem we have right now is that because of the lack of support of proper indexing in N1QL (covering Indexes, etc) the queries take a long time and the load on CB is very heavy. The script currently takes about a minute to run and the database averages 25-30k iops during the period. I wouldn’t be too worried just by pure iops usage but we see avg CPU usage between 80-95% during the execution.
A solution that we thought off was to possibly have a small (maybe even single node) cluster that is being fed via XDCR from our live cluster and to use this secondary cluster as just a N1QL cluster. Would that be a recommended solution or could someone propose an alternate architecture?