Couchbase Plug-in for Elasticsearch

The Couchbase plug-in for Elasticsearch integrates Couchbase Server and Elasticsearch, a full-text search engine, by streaming data in real-time from Couchbase to Elasticsearch. The combined solution enables you to build richer and more powerful apps with full-text search, indexing and querying and real-time analytics for variety of use cases such as a content store or aggregation of data from different data sources.

Couchbase Server provides easy scalability, low-latency document access, indexing and querying of JSON documents and real-time analytics with incremental map reduce. Elasticsearch provides real-time distributed full-text search for JSON documents with a rich query domain-specific language (DSL) for simple or advanced queries. It also provides advanced relevance-based query features like boosting and faceted filtering.

High-Level Architecture

As shown in the figure below, the application uses the Couchbase Server SDKs to query Couchbase Server and the Elasticsearch query DSL APIs to query Elasticsearch.

The plug-in is installed on the Elasticsearch cluster and uses Couchbase’s cross datacenter replication (XDCR) to propagate document updates made in Couchbase to Elasticsearch. The search index on the Elasticsearch cluster is always kept up to date with the data in Couchbase.

To search across all your documents, you can use the Elasticsearch query DSL. You get back document IDs for relevant documents. You can directly access documents in Couchbase to retrieve document content.

Based on the requirements of your application for Couchbase and Elasticsearch, you can scale the Couchbase and Elasticsearch clusters independently – the clusters can both have different number of servers.

Getting Started

The installation process for the Couchbase plug-in for Elasticsearch is fairly straightforward. To get started with Couchbase Server and Elasticsearch:

  • Download the required prerequisites
  • Install the plug-ins using the following steps:
    • Go to the install directory for Elasticsearch
      cd elasticsearch-1.0.1
    • Install the Couchbase plug-in
      bin/plugin -install transport-couchbase -url http://packages.couchbase.com.s3.amazonaws.com/releases/elastic-search-adapter/1.3.0/elasticsearch-transport-couchbase-1.3.0.zip

      After a successful install, the plugin installer returns:
      DONE Installed transport-couchbase
    • Set the Couchbase credentials for the plug-in
      echo "couchbase.password: password" >> config/elasticsearch.yml
      echo "couchbase.username: Administrator" >> config/elasticsearch.yml
    • Install the head Elasticsearch plug-in which provides a simple user interface so that you can interact with Elasticsearch
      bin/plugin -install mobz/elasticsearch-head
    • Start the Elasticsearch process. The Elasticsearch process will run in the background.
      bin/elasticsearch
  • Configure the Couchbase plug-in for Elasticsearch
  • Setup cross-datacenter replication from Couchbase to Elasticsearch

Learn More

Here are some more resources to help you get started: