{"id":5155,"date":"2018-05-21T06:45:33","date_gmt":"2018-05-21T13:45:33","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=5155"},"modified":"2025-06-13T19:28:11","modified_gmt":"2025-06-14T02:28:11","slug":"nifi-processing-flow-couchbase-server","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/","title":{"rendered":"Getting Started with NiFi and Couchbase Server"},"content":{"rendered":"<div class=\"paragraph\">\n<div class=\"paragraph\">\n<p>I am just getting started with NiFi, a tool to automate the flow of data. It\u2019s a tool for migration, synchronization, and other types of data processing. It was introduced to me by one of Couchbase\u2019s newest customers: the Cincinnati Reds.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>In this post, I\u2019ll describe what the Reds are using NiFi for and I\u2019ll show you how to get up and running with a very basic data flow from SQL Server to Couchbase Server.<\/p>\n<\/div>\n<div class=\"sect1\">\n<h2 id=\"_nifi_and_the_reds\">NiFi and the Reds<\/h2>\n<div class=\"sectionbody\">\n<div class=\"paragraph\">\n<p>The Reds want to create some visualizations of tickets being scanned on game day at Great American Ball Park.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>The data team has access to a SQL Server database which is used to store live data about a game. Whenever a ticket is scanned at the gate, data is put into this database. (This database also tracks concessions and other data).<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>The Reds could potentially query data directly from the SQL Server, but a real-time visualization during the heavy load of game time would result in a slow visualization or too much load for the database, or both. Instead, they\u2019d like to copy that data into a Couchbase cluster, and use the cluster as the backend for the visualization.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>There are a number of ways to move data from to Couchbase, but the Reds are already using open-source Apache NiFi with SQL Server, and it would be ideal if they could use that same combination for this project. Fortunately, NiFi already supports Couchbase, so it\u2019s pretty easy to do.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"sect1\">\n<h2 id=\"_getting_started_with_nifi_and_couchbase\">Getting Started with NiFi and Couchbase<\/h2>\n<div class=\"sectionbody\">\n<div class=\"paragraph\">\n<p>To start experimenting with NiFi locally, I decided to use Docker. Within the Docker host, I can easily spin up an instance each of:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li><a href=\"https:\/\/hub.docker.com\/_\/couchbase\/\">Couchbase Server<\/a> (of course)<\/li>\n<li>Apache Nifi (<a href=\"https:\/\/hub.docker.com\/r\/apache\/nifi\">Docker hub link<\/a>)<\/li>\n<li><a href=\"https:\/\/hub.docker.com\/r\/microsoft\/mssql-server-linux\/\">Microsoft SQL Server<\/a> (for Linux\u2014\u200bI don\u2019t think the Reds are using SQL for Linux, but close enough)<\/li>\n<\/ul>\n<\/div>\n<div class=\"paragraph\">\n<p>You don\u2019t need to use Docker, but it made it very easy for me to get up and running and productive right away.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>Here are the commands I used to run the Docker images:<\/p>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"content\">\n<pre class=\"highlight decode:true\"><code>docker run -d --name db55beta -p 8091-8094:8091-8094 -p 11210:11210 couchbase:5.5.0-beta\r\n\r\ndocker run -d --name NiFi -p 8080:8080 apache\/nifi:latest\r\n\r\ndocker run -d -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=&lt;use a strong password here&gt;' -p 1433:1433 microsoft\/mssql-server-linux:2017-latest<\/code><\/pre>\n<\/div>\n<\/div>\n<div class=\"paragraph\">\n<p><em>Note that the password you supply to SA_PASSWORD <strong>must<\/strong> meet the strong password requirements of SQL Server. Otherwise, you will be unable to use SQL Server and get a little frustrated and confused for about 20 minutes.<\/em><\/p>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_starting_with_sql_server\">Starting with SQL Server<\/h3>\n<div class=\"paragraph\">\n<p>I used SQL Server Management Studio to connect to the SQL Server instance in Docker (localhost, port 1433). I don\u2019t (yet) have access to the Reds actual server, so I came up with my own schema to approximate:<\/p>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"content\">\n<pre class=\"highlight decode:true\"><code class=\"language-SQL\">CREATE TABLE [dbo].[TicketCheck](\r\n\t[Id] [uniqueidentifier] NOT NULL,\r\n\t[FullName] [varchar](100) NOT NULL,\r\n\t[Section] [varchar](10) NOT NULL,\r\n\t[Row] [varchar](10) NOT NULL,\r\n\t[Seat] [varchar](10) NOT NULL,\r\n\t[GameDay] [datetime] NOT NULL,\r\n CONSTRAINT [PK_TicketCheck] PRIMARY KEY CLUSTERED\r\n(\r\n\t[Id] ASC\r\n)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]\r\n) ON [PRIMARY]\r\n\r\nGO\r\n\r\nALTER TABLE [dbo].[TicketCheck] ADD  CONSTRAINT [DF_TicketCheck_Id]  DEFAULT (newid()) FOR [Id]\r\nGO<\/code><\/pre>\n<\/div>\n<\/div>\n<div class=\"paragraph\">\n<p>Later, I\u2019ll be populating it with an <code>INSERT<\/code> statement like so:<\/p>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"content\">\n<pre class=\"highlight decode:true\"><code class=\"language-SQL\">INSERT INTO TicketCheck (FullName, Section, [Row], Seat, GameDay) VALUES (\r\n\t'Joey Votto',\r\n\t'429',\r\n\t'C',\r\n\t'11',\r\n\tGETDATE()\r\n)<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_setting_up_couchbase\">Setting up Couchbase<\/h3>\n<div class=\"paragraph\">\n<p>Once I logged in to Couchbase for the first time (localhost:8091) and created a cluster, I did two things:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li>I created a bucket called &#8220;tickets&#8221;. This is where I want the data from SQL Server to go.<\/li>\n<li>I created a user <em>also<\/em> called &#8220;tickets&#8221;, with adequate permission for the bucket. It\u2019s important that the user has the same name as the bucket.<\/li>\n<\/ul>\n<\/div>\n<div class=\"paragraph\">\n<p><em>The reason that you need to create a user with the same name is because the NiFi Couchbase processor is a little bit out of date, so this is a workaround. NiFi hasn\u2019t been updated to handle the new RBAC capabilities of Couchbase, yet. See Apache Nifi <a href=\"https:\/\/issues.apache.org\/jira\/browse\/NIFI-5054\">issue 5054<\/a> for more information.<\/em><\/p>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_setting_up_nifi\">Setting up NiFi<\/h3>\n<div class=\"paragraph\">\n<p>NiFi is a web-based, visual data flow tool. I\u2019m a developer, I\u2019m used to code and command lines, but I definitely appreciate a nice visual interface when applicable.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>If you used Docker, just visit <strong>localhost:8080\/NiFi<\/strong>. You\u2019ll see what looks like a big sheet of graph paper with a few toolbars\/windows on top.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>I\u2019m going to skip a head a bit and show the complete data flow that I built:<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p><span class=\"image\"><img decoding=\"async\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2018\/05\/10701-nifi-end-state.png\" alt=\"NiFi complete data flow\" \/><\/span><\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>I\u2019ll go through this step-by-step, but keep in mind that I\u2019m not a NiFi expert.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>At a high level, each of these boxes are &#8220;processors&#8221;. Each of them get data from somewhere, process the data in some way, and write the data somewhere else. This &#8220;flow&#8221; of data may come from an external source, a NiFi queue, or be written to an external source. Each processor can be &#8220;Started&#8221; and &#8220;Stopped&#8221;.<\/p>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_database_connection_pooling_service\">Database Connection Pooling Service<\/h3>\n<div class=\"paragraph\">\n<p>Before we create a processor, let\u2019s tell NiFi about the databases we\u2019ll be using.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>There\u2019s an &#8220;Operate&#8221; window that\u2019s floating over the graph paper. Click the settings icon to bring up the NiFi Flow Configuration window.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p><span class=\"image\"><img decoding=\"async\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2018\/05\/10703-nifi-configuration.png\" alt=\"NiFi Configuration\" \/><\/span><\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>Look at the Controller Services tab. I think of this as a collection of external data sources that the processors can connect to. We\u2019re going to add two controller services: one for SQL Server and one for Couchbase.<\/p>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_dbcpconnectionpool\">DBCPConnectionPool<\/h3>\n<div class=\"paragraph\">\n<p>Click the &#8220;+&#8221; button to add. Let\u2019s start with SQL Server: find DBCPConnectionPool and click &#8220;Add&#8221;. It should appear in the list. Click the gear icon and navigate to the properties tab:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li><strong>Database Connection URL<\/strong> &#8211; Enter a value like <code>jdbc:sqlserver:\/\/172.17.0.4<\/code>.<\/li>\n<li><strong>Database Driver Class Name<\/strong> &#8211; If you\u2019re using SQL Server, it\u2019s <code>com.microsoft.sqlserver.jdbc.SQLServerDriver<\/code><\/li>\n<li><strong>Database Driver Location(s)<\/strong> &#8211; Enter <code>file:\/\/\/usr\/share\/java\/mssql-jdbc-6.4.0.jre8.jar<\/code>. Note that NiFi does not come with this driver out of the box (at least not the Docker image). <a href=\"https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=56615\">Download this driver from Microsoft<\/a> and put it in the \/usr\/share\/java folder on your NiFi server (you can use <code><a href=\"https:\/\/docs.docker.com\/engine\/reference\/commandline\/cp\/\">docker cp<\/a><\/code> if you\u2019re using Docker like me).<\/li>\n<li><strong>Database User<\/strong> and <strong>Password<\/strong> &#8211; The SQL Server credentials you need to connect.<\/li>\n<\/ul>\n<\/div>\n<div class=\"paragraph\">\n<p>Once you\u2019ve added it, you\u2019ll need to &#8220;enable&#8221; it (click the lightning icon) in order to use it. If you need to make changes in the future, you\u2019ll need to disable it first.<\/p>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_couchbaseclusterservice\">CouchbaseClusterService<\/h3>\n<div class=\"paragraph\">\n<p>Next, let\u2019s tell NiFi about Couchbase. Again, click the &#8220;plus&#8221; button to add. Look for CouchbaseClusterService. Again, navigate to the properties tab. There should be one property called <strong>Connection String<\/strong>. Enter something like <code>couchbase:\/\/172.17.0.3<\/code>. Next, click the &#8220;plus&#8221; button on this tab and create a new property called &#8220;Bucket Password for tickets&#8221;. Note that the property name must be of the form <strong>&#8220;Bucket Password for &lt;your bucket name&gt;&#8221;<\/strong>. The value of this property should be the password of the Couchbase user you created earlier.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>Now, NiFi knows about SQL Server and Couchbase. Let\u2019s put it to work.<\/p>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_querydatabasetable\">QueryDatabaseTable<\/h3>\n<div class=\"paragraph\">\n<p>I\u2019ll start with the source of the data: a SQL Server. More specifically, a table in SQL Server. And even more specifically, only new rows of data in that table (more on how to define that later).<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>First, drag the &#8220;processor&#8221; icon from the top left onto the graph paper. Then, find the QueryDatabaseTable processor and hit &#8220;add&#8221;. At this point, you\u2019ll have a processor on the board with a warning icon that indicates you need to do some configuration.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p><span class=\"image\"><img decoding=\"async\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2018\/05\/10702-nifi-processor.gif\" alt=\"Adding a processor to NiFi\" \/><\/span><\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>You can double-click on this processor to bring up its details. I am primarily interested in the &#8220;Properties&#8221; tab. On this tab, I\u2019m going to tell this processor what database to connect to and how to query data from it:<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>The properties of interest:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li><strong>Database Connection Pooling Service<\/strong> &#8211; Select the DBCPConnectionPool that was created earlier.<\/li>\n<li><strong>Database Type<\/strong> &#8211; I selected MS SQL 2008, which seems to work fine with MS SQL for Linux, but there are also options for MS SQL 2012+ and &#8220;Generic&#8221;.<\/li>\n<li><strong>Table Name<\/strong> &#8211; Enter the name of the table to query. <code>TicketCheck<\/code> is the one I used.<\/li>\n<li><strong>Maximum-value Columns<\/strong> &#8211; I entered <code>GameDay<\/code>. This is the column that NiFi will check against to find new\/updated data in the table. You might want to use an auto-incremented field, or a timestamp, or some other combination. The NiFi processor will store the latest value in its &#8220;state&#8221; as it goes along.<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_putcouchbasekey\">PutCouchbaseKey<\/h3>\n<div class=\"paragraph\">\n<p>Let\u2019s skip ahead a bit and create another processor. This time it will be a PutCouchbaseKey processor. All this processor does it take a piece of data that\u2019s flowing into it and create\/update a Couchbase document with that data.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>To configure it, set these properties:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li><strong>Couchbase Cluster Controller Service<\/strong> &#8211; select the CouchbaseClusterService that was created earlier.<\/li>\n<li><strong>Bucket Name<\/strong> &#8211; tickets<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_getting_from_point_a_to_point_b\">Getting from point A to point B<\/h3>\n<div class=\"paragraph\">\n<p>At this point, NiFi is able to pull data from SQL Server and put documents into Couchbase. To finish it off, they need to be connected. But there\u2019s still a bit of work to do. the QueryDatabaseTable processor outputs &#8220;Avro&#8221; data, which is designed for Hadoop, but is also used by Spark and, of course, Nifi. We could feed this directly into Couchbase, but it would be stored as binary data, and not JSON. So, there are a couple of intermediate steps to get it into pure JSON form.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>I added a SplitAvro processor and a ConvertAvroToJSON processor to the graph paper.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>The SplitAvro processor will split the (potentially large) Avro data file into smaller files. This may not strictly be necessary, but it\u2019s a good precaution to take, and it helps split up the data for easier viewing and debugging. The default properties of this processor are fine.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>The ConvertAvroToJSON processor does exactly what it says. This will prepare the Avro data for Couchbase. I changed the <strong>JSON container options<\/strong> property from <code>array<\/code> to <code>none<\/code>. I just want a plain JSON document, and not an array containing a single document.<\/p>\n<\/div>\n<\/div>\n<div class=\"sect2\">\n<h3 id=\"_connecting_it_all_up\">Connecting it all up<\/h3>\n<div class=\"paragraph\">\n<p>Now that you have these four pieces in place, you need to connect them.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>First, hover over the QueryDatabaseTable until you see an arrow icon appear. Click and drag this arrow to the SplitAvro processor. A queue will appear between them named &#8220;success&#8221;.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>Repeat this with the other processors. A processor may have a variety of termination points that define the relationship. For instance, when you drag a connection between SplitAvro and ConvertAvroToJSON, you will be presented with three choices: failure, original, and split. This will vary from processor to processor, but here\u2019s the idea:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li><strong>failure<\/strong> &#8211; SplitAvro failed at the conversion, then it will send data to &#8220;failure&#8221;<\/li>\n<li><strong>original<\/strong> &#8211; SplitAvro can pipe the original data out this way<\/li>\n<li><strong>split<\/strong> &#8211; The actual split-up data goes this way. This is what you should feed in to ConvertAvroToJSON.<\/li>\n<\/ul>\n<\/div>\n<div class=\"paragraph\">\n<p>With the other connections, you could pipe the data back into the process to retry, or perhaps pipe it into some notification or debugging processor.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"sect1\">\n<h2 id=\"_turn_on_the_nifi_flow\">Turn on the NiFi flow<\/h2>\n<div class=\"sectionbody\">\n<div class=\"paragraph\">\n<p>To start a processor, click on it and then click the &#8220;start&#8221; button in the Operate window (it looks like the play button on a VCR). You might want to experiment with just one processor at a time, and watch the data start to stack up in the queues. Ultimately, when you start inserting rows into the SQL Server table, they should end up as new documents in Couchbase Server.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p><span class=\"image\"><img decoding=\"async\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2018\/05\/10704-sql-to-nifi-to-couchbase.gif\" alt=\"SQL Server to NiFi to Couchbase Server\" \/><\/span><\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>In the above animation, I\u2019m inserting two new rows to a table in SQL Server. NiFi (not pictured) is processing them and putting them into Couchbase.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"sect1\">\n<h2 id=\"_summary\">Summary<\/h2>\n<div class=\"sectionbody\">\n<div class=\"paragraph\">\n<p>This blog post provides background for how to get started with NiFi. There is plenty you can do. If you are at an enterprise with a variety of data sources, NiFi is a great tool to orchestrate all those data flows. Couchbase Server is a great fit as well:<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li>The flexibility of JSON allows you to ingest data from just about any source<\/li>\n<li>The memory-first architecture helps maximize the performance of your data flow<\/li>\n<li>The scaling capabilities of Couchbase allows you to increase your capacity without having to take your flow offline<\/li>\n<\/ul>\n<\/div>\n<div class=\"paragraph\">\n<p>I\u2019m just learning NiFi for the first time, and I\u2019m already loving the graphical interface and the simplicity of getting started. I still have a lot to learn, but hopefully this post will help you out with using the Couchbase processor in NiFi.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>If you are using NiFi and couchbase, I want to hear from you. The Couchbase connector could use updated (<a href=\"https:\/\/issues.apache.org\/jira\/browse\/NIFI-5054\">see issue 5054<\/a>), and the more of you that I hear from, the easier it is to justify the time spent working on it.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>If you have questions about Couchbase, please check out the <a href=\"https:\/\/www.couchbase.com\/forums\/c\/couchbase-server\/\">Couchbase Server forums<\/a>. If you have questions about NiFi, check out the Apache Nifi <a href=\"https:\/\/nifi.apache.org\/\">project website<\/a>.<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>I\u2019m also happy to talk with you about all of the above. You can leave a comment below, or find me on <a href=\"https:\/\/twitter.com\/mgroves\">Twitter @mgroves<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>I am just getting started with NiFi, a tool to automate the flow of data. It\u2019s a tool for migration, synchronization, and other types of data processing. It was introduced to me by one of Couchbase\u2019s newest customers: the Cincinnati [&hellip;]<\/p>\n","protected":false},"author":71,"featured_media":5162,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1816],"tags":[1458,1481],"ppma_author":[8937],"class_list":["post-5155","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-couchbase-server","tag-migration","tag-nifi"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.7.1 (Yoast SEO v25.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Getting Started with NiFi and Couchbase Server - The Couchbase Blog<\/title>\n<meta name=\"description\" content=\"NiFi automates data flow, providing migration, synchronization, and more. Learn how the Cincinnati Reds baseball club uses NiFi to help with ticketing.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Getting Started with NiFi and Couchbase Server\" \/>\n<meta property=\"og:description\" content=\"NiFi automates data flow, providing migration, synchronization, and more. Learn how the Cincinnati Reds baseball club uses NiFi to help with ticketing.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\" \/>\n<meta property=\"og:site_name\" content=\"The Couchbase Blog\" \/>\n<meta property=\"article:published_time\" content=\"2018-05-21T13:45:33+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-14T02:28:11+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"823\" \/>\n\t<meta property=\"og:image:height\" content=\"404\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Matthew Groves\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@mgroves\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Matthew Groves\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\"},\"author\":{\"name\":\"Matthew Groves\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/3929663e372020321b0152dc4fa65a58\"},\"headline\":\"Getting Started with NiFi and Couchbase Server\",\"datePublished\":\"2018-05-21T13:45:33+00:00\",\"dateModified\":\"2025-06-14T02:28:11+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\"},\"wordCount\":2052,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg\",\"keywords\":[\"migration\",\"NiFi\"],\"articleSection\":[\"Couchbase Server\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\",\"name\":\"Getting Started with NiFi and Couchbase Server - The Couchbase Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg\",\"datePublished\":\"2018-05-21T13:45:33+00:00\",\"dateModified\":\"2025-06-14T02:28:11+00:00\",\"description\":\"NiFi automates data flow, providing migration, synchronization, and more. Learn how the Cincinnati Reds baseball club uses NiFi to help with ticketing.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg\",\"width\":823,\"height\":404,\"caption\":\"Mr. Redlegs and Couchbase together again\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.couchbase.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Getting Started with NiFi and Couchbase Server\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"name\":\"The Couchbase Blog\",\"description\":\"Couchbase, the NoSQL Database\",\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\",\"name\":\"The Couchbase Blog\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"width\":218,\"height\":34,\"caption\":\"The Couchbase Blog\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/3929663e372020321b0152dc4fa65a58\",\"name\":\"Matthew Groves\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/ba51e6aacc53995c323a634e4502ef54\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/70feb1b28a099ad0112b8d21fe1e81e1a4524beed3e20b7f107d5370e85a07ab?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/70feb1b28a099ad0112b8d21fe1e81e1a4524beed3e20b7f107d5370e85a07ab?s=96&d=mm&r=g\",\"caption\":\"Matthew Groves\"},\"description\":\"Matthew D. Groves is a guy who loves to code. It doesn't matter if it's C#, jQuery, or PHP: he'll submit pull requests for anything. He has been coding professionally ever since he wrote a QuickBASIC point-of-sale app for his parent's pizza shop back in the 90s. He currently works as a Senior Product Marketing Manager for Couchbase. His free time is spent with his family, watching the Reds, and getting involved in the developer community. He is the author of AOP in .NET, Pro Microservices in .NET, a Pluralsight author, and a Microsoft MVP.\",\"sameAs\":[\"https:\/\/crosscuttingconcerns.com\",\"https:\/\/x.com\/mgroves\"],\"url\":\"https:\/\/www.couchbase.com\/blog\/author\/matthew-groves\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Getting Started with NiFi and Couchbase Server - The Couchbase Blog","description":"NiFi automates data flow, providing migration, synchronization, and more. Learn how the Cincinnati Reds baseball club uses NiFi to help with ticketing.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/","og_locale":"en_US","og_type":"article","og_title":"Getting Started with NiFi and Couchbase Server","og_description":"NiFi automates data flow, providing migration, synchronization, and more. Learn how the Cincinnati Reds baseball club uses NiFi to help with ticketing.","og_url":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/","og_site_name":"The Couchbase Blog","article_published_time":"2018-05-21T13:45:33+00:00","article_modified_time":"2025-06-14T02:28:11+00:00","og_image":[{"width":823,"height":404,"url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg","type":"image\/jpeg"}],"author":"Matthew Groves","twitter_card":"summary_large_image","twitter_creator":"@mgroves","twitter_misc":{"Written by":"Matthew Groves","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#article","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/"},"author":{"name":"Matthew Groves","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/3929663e372020321b0152dc4fa65a58"},"headline":"Getting Started with NiFi and Couchbase Server","datePublished":"2018-05-21T13:45:33+00:00","dateModified":"2025-06-14T02:28:11+00:00","mainEntityOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/"},"wordCount":2052,"commentCount":0,"publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg","keywords":["migration","NiFi"],"articleSection":["Couchbase Server"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/","url":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/","name":"Getting Started with NiFi and Couchbase Server - The Couchbase Blog","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg","datePublished":"2018-05-21T13:45:33+00:00","dateModified":"2025-06-14T02:28:11+00:00","description":"NiFi automates data flow, providing migration, synchronization, and more. Learn how the Cincinnati Reds baseball club uses NiFi to help with ticketing.","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#primaryimage","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2018\/05\/107-hero-couchbase-team-and-mr-redlegs.jpg","width":823,"height":404,"caption":"Mr. Redlegs and Couchbase together again"},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/blog\/nifi-processing-flow-couchbase-server\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Getting Started with NiFi and Couchbase Server"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/blog\/#website","url":"https:\/\/www.couchbase.com\/blog\/","name":"The Couchbase Blog","description":"Couchbase, the NoSQL Database","publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/blog\/#organization","name":"The Couchbase Blog","url":"https:\/\/www.couchbase.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","width":218,"height":34,"caption":"The Couchbase Blog"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/3929663e372020321b0152dc4fa65a58","name":"Matthew Groves","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/ba51e6aacc53995c323a634e4502ef54","url":"https:\/\/secure.gravatar.com\/avatar\/70feb1b28a099ad0112b8d21fe1e81e1a4524beed3e20b7f107d5370e85a07ab?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/70feb1b28a099ad0112b8d21fe1e81e1a4524beed3e20b7f107d5370e85a07ab?s=96&d=mm&r=g","caption":"Matthew Groves"},"description":"Matthew D. Groves is a guy who loves to code. It doesn't matter if it's C#, jQuery, or PHP: he'll submit pull requests for anything. He has been coding professionally ever since he wrote a QuickBASIC point-of-sale app for his parent's pizza shop back in the 90s. He currently works as a Senior Product Marketing Manager for Couchbase. His free time is spent with his family, watching the Reds, and getting involved in the developer community. He is the author of AOP in .NET, Pro Microservices in .NET, a Pluralsight author, and a Microsoft MVP.","sameAs":["https:\/\/crosscuttingconcerns.com","https:\/\/x.com\/mgroves"],"url":"https:\/\/www.couchbase.com\/blog\/author\/matthew-groves\/"}]}},"authors":[{"term_id":8937,"user_id":71,"is_guest":0,"slug":"matthew-groves","display_name":"Matthew Groves","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/70feb1b28a099ad0112b8d21fe1e81e1a4524beed3e20b7f107d5370e85a07ab?s=96&d=mm&r=g","author_category":"","last_name":"Groves","first_name":"Matthew","job_title":"","user_url":"https:\/\/crosscuttingconcerns.com","description":"Matthew D. Groves is a guy who loves to code.  It doesn't matter if it's C#, jQuery, or PHP: he'll submit pull requests for anything.  He has been coding professionally ever since he wrote a QuickBASIC point-of-sale app for his parent's pizza shop back in the 90s.  He currently works as a Senior Product Marketing Manager for Couchbase. His free time is spent with his family, watching the Reds, and getting involved in the developer community.  He is the author of AOP in .NET, Pro Microservices in .NET, a Pluralsight author, and a Microsoft MVP."}],"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/5155","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/users\/71"}],"replies":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/comments?post=5155"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/5155\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media\/5162"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media?parent=5155"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/categories?post=5155"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/tags?post=5155"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=5155"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}