I’m working on a project where we got high insert/update rate (10K/s average) and I’d like to use the Elasticsearch transport plugin to benefit from the rich queying capabilities of Elastcisearch.
The issue is that, most of our workload is CAS updates on counters which are fields inside Couchbase documents. Reads are infrequent relative to writes and we could happily cope with few hours of delay between the update in Couchbase and Elasticsearch. So, what I’d wish to do is to only sync between couchbase and elasticsearch once in a few hours, and only have the last version of a document send to elasticsearch (instead of all updates) in order to reduce the load on the ES cluster.
Is it possible to do this with the elasticsearch transport ?