The Couchbase Flink connector is updated to version 0.5.0.
Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink is designed to run in all common cluster environments, and perform computations at in-memory speed and at any scale.
With our recently updated Couchbase connector, Flink projects can use Couchbase clusters not only to store and retrieve data but also to subscribe and react to changes in data as they are applied on the cluster. This feature is implemented by directly reading the Data Change Protocol stream from a Couchbase cluster.
There are several noticeable changes in this release of the Flink connector:
- Flink 2.1.1 is now supported by the connector
- Couchbase java SDK is updated to the latest version
- The DCP library dependency is updated to the latest version
Besides introducing support for Flink 2.1.1, this new version updates Couchbase dependencies and removes previously deprecated APIs.
The connector can be installed from a Maven repository:
Or it can be built from sources that are located at the following GitHub repository: https://github.com/couchbaselabs/flink-connector-couchbase.
The connector provides Flink classes like:
CouchbaseDCPSource– subscribes to the Couchbase DCP stream for real-time mutation processingCouchbaseQuerySource– Executes SQL++ queries and streams results into FlinkCouchbaseCollectionSink– Stores documents into Couchbase collections.
Small example Java projects that demonstrate how to use the connector can be found in the exemplos folder in the repository: https://github.com/couchbaselabs/flink-connector-couchbase/tree/master/examples
The connector library is currently experimental and in alpha stage. We welcome feedback and contributions from the community. If you run into any problems when using the connector, or if you have any feedback, please open a GitHub issue in the project repository: https://github.com/couchbaselabs/flink-connector-couchbase/issues
