{"id":5195,"date":"2018-05-24T13:45:21","date_gmt":"2018-05-24T20:45:21","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=5195"},"modified":"2023-06-13T02:28:44","modified_gmt":"2023-06-13T09:28:44","slug":"compression-and-ipv6-in-kafka-connector-3-3","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/","title":{"rendered":"Compression and IPv6 in Kafka Connector 3.3"},"content":{"rendered":"

In recent months the Kafka connector has seen several updates, adding features like log redaction in version 3.2.3, configurable durability requirements in 3.2.2, and a new SourceHandler<\/code> extension point for custom processing in 3.2.0.<\/p>\n

Now, with the release of Couchbase Server 5.5 right around the corner, we’ve updated the Kafka connector again to take advantage of some exciting new features available in the Enterprise Edition.<\/p>\n

Add IPv6 to your bag of tricks (or, “Like two IPs in a quad”)<\/h3>\n

The industry-wide migration to IPv6 is in full swing. As Perry Krug wrote in March<\/a>, Couchbase will join the party with support for IPv6 in the upcoming 5.5 release.<\/p>\n

Version 3.2.3 of the Kafka connector could be made to work with IPv6 by setting a Java system property. This was inconvenient, so in version 3.3 we’ve enabled IPv6 support by default.<\/p>\n

In most cases, the connector will continue to work with IPv4 without requiring any config tweaks. However, if you find yourself in an dual-stack environment where you need to force hostname resolution to use IPv4, you can get the desired behavior by setting the couchbase.forceIPv4<\/code> config property to true<\/code>.<\/p>\n

Data compression for enhanced resource utilization (or, “Just Squeeze Me”)<\/h3>\n

Chaitra Ramarao wrote a great introduction to the data compression features coming in Couchbase Server 5.5<\/a>. Performance testing revealed that enabling compression can significantly improve throughput, so we’ve enabled compression by default in Kafka connector 3.3.<\/p>\n

If you want to tune the compression settings, you remain in full control. The couchbase.compression<\/code> config property lets you specify the compression mode to use. There are two modes to choose from (three if you count “disabled”). Let’s go over them in detail.<\/p>\n

The new default mode is ENABLED<\/code>. In this mode, you’re giving Couchbase Server permission to send data to the Kafka connector in compressed form. It might still send an uncompressed document if compressing it would require the server to do more work. However, most of the time the server will already have easy access to the compressed form, and the document will be sent compressed. We made this the default mode after our performance tests showed an overall improvement when compression is used at the cluster\u2019s discretion.<\/p>\n

The second compression mode is FORCED<\/code>. In this mode, you’re telling Couchbase Server that network bandwidth is at a premium, and you want the server to make every effort to send documents in compressed form. As you might expect, this mode requires the server to do more work, so you’ll be using some additional CPU at the cluster side in exchange for reduced bandwidth usage.<\/p>\n

To opt out of compression, use the DISABLED<\/code> mode. Be aware that disabling compression might change CPU usage or throughput\/latency, since the cluster must decompress each document before sending it to the connector.<\/p>\n

Regardless of the compression mode you select, documents are published to the Kafka topic in uncompressed form.<\/p>\n

That’s all for now. To download the Kafka connector and kick the tires, check out the Quickstart Guide<\/a>. If you have any questions, come visit the Kafka connector category<\/a> on the Couchbase Forums!<\/p>\n","protected":false},"excerpt":{"rendered":"

In recent months the Kafka connector has seen several updates, adding features like log redaction in version 3.2.3, configurable durability requirements in 3.2.2, and a new SourceHandler extension point for custom processing in 3.2.0. Now, with the release of Couchbase […]<\/p>\n","protected":false},"author":14341,"featured_media":13873,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[2242],"tags":[2183,2186,1487],"ppma_author":[9070],"class_list":["post-5195","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-connectors","tag-compression","tag-ipv6","tag-kafka"],"acf":[],"yoast_head":"\nCompression and IPv6 in Kafka Connector 3.3 - The Couchbase Blog<\/title>\n<meta name=\"description\" content=\"With the release of Couchbase Server 5.5 right around the corner, we've updated the Kafka connector to take advantage of some exciting new features available in the Enterprise Edition.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Compression and IPv6 in Kafka Connector 3.3\" \/>\n<meta property=\"og:description\" content=\"With the release of Couchbase Server 5.5 right around the corner, we've updated the Kafka connector to take advantage of some exciting new features available in the Enterprise Edition.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\" \/>\n<meta property=\"og:site_name\" content=\"The Couchbase Blog\" \/>\n<meta property=\"article:author\" content=\"https:\/\/duckduckgo.com\/?q=delete+facebook\" \/>\n<meta property=\"article:published_time\" content=\"2018-05-24T20:45:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-13T09:28:44+00:00\" \/>\n<meta name=\"author\" content=\"David Nault\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@orzobat\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"David Nault\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\"},\"author\":{\"name\":\"David Nault\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ec7e18bcee3337b77367560e6aeb75ef\"},\"headline\":\"Compression and IPv6 in Kafka Connector 3.3\",\"datePublished\":\"2018-05-24T20:45:21+00:00\",\"dateModified\":\"2023-06-13T09:28:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\"},\"wordCount\":513,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"keywords\":[\"compression\",\"IPv6\",\"kafka\"],\"articleSection\":[\"Connectors\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\",\"name\":\"Compression and IPv6 in Kafka Connector 3.3 - The Couchbase Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"datePublished\":\"2018-05-24T20:45:21+00:00\",\"dateModified\":\"2023-06-13T09:28:44+00:00\",\"description\":\"With the release of Couchbase Server 5.5 right around the corner, we've updated the Kafka connector to take advantage of some exciting new features available in the Enterprise Edition.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"width\":1800,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.couchbase.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Compression and IPv6 in Kafka Connector 3.3\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"name\":\"The Couchbase Blog\",\"description\":\"Couchbase, the NoSQL Database\",\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\",\"name\":\"The Couchbase Blog\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"width\":218,\"height\":34,\"caption\":\"The Couchbase Blog\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ec7e18bcee3337b77367560e6aeb75ef\",\"name\":\"David Nault\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/fbceda752a7ba278f83b24641d607212\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9be1cd3e3c55df4d11b191172d382965de29da3a98d341167b33a8ad00546057?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9be1cd3e3c55df4d11b191172d382965de29da3a98d341167b33a8ad00546057?s=96&d=mm&r=g\",\"caption\":\"David Nault\"},\"description\":\"David Nault is a Database Connector Developer for Couchbase. He spent the greater part of two decades writing back end web services in Java, and thinks JSON-RPC is really cool. He lives in California.\",\"sameAs\":[\"https:\/\/github.com\/dnault\",\"https:\/\/duckduckgo.com\/?q=delete+facebook\",\"https:\/\/x.com\/orzobat\"],\"url\":\"https:\/\/www.couchbase.com\/blog\/author\/dnault\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Compression and IPv6 in Kafka Connector 3.3 - The Couchbase Blog","description":"With the release of Couchbase Server 5.5 right around the corner, we've updated the Kafka connector to take advantage of some exciting new features available in the Enterprise Edition.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/","og_locale":"en_US","og_type":"article","og_title":"Compression and IPv6 in Kafka Connector 3.3","og_description":"With the release of Couchbase Server 5.5 right around the corner, we've updated the Kafka connector to take advantage of some exciting new features available in the Enterprise Edition.","og_url":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/","og_site_name":"The Couchbase Blog","article_author":"https:\/\/duckduckgo.com\/?q=delete+facebook","article_published_time":"2018-05-24T20:45:21+00:00","article_modified_time":"2023-06-13T09:28:44+00:00","author":"David Nault","twitter_card":"summary_large_image","twitter_creator":"@orzobat","twitter_misc":{"Written by":"David Nault","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#article","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/"},"author":{"name":"David Nault","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ec7e18bcee3337b77367560e6aeb75ef"},"headline":"Compression and IPv6 in Kafka Connector 3.3","datePublished":"2018-05-24T20:45:21+00:00","dateModified":"2023-06-13T09:28:44+00:00","mainEntityOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/"},"wordCount":513,"commentCount":0,"publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","keywords":["compression","IPv6","kafka"],"articleSection":["Connectors"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/","url":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/","name":"Compression and IPv6 in Kafka Connector 3.3 - The Couchbase Blog","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","datePublished":"2018-05-24T20:45:21+00:00","dateModified":"2023-06-13T09:28:44+00:00","description":"With the release of Couchbase Server 5.5 right around the corner, we've updated the Kafka connector to take advantage of some exciting new features available in the Enterprise Edition.","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#primaryimage","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","width":1800,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/blog\/compression-and-ipv6-in-kafka-connector-3-3\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Compression and IPv6 in Kafka Connector 3.3"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/blog\/#website","url":"https:\/\/www.couchbase.com\/blog\/","name":"The Couchbase Blog","description":"Couchbase, the NoSQL Database","publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/blog\/#organization","name":"The Couchbase Blog","url":"https:\/\/www.couchbase.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","width":218,"height":34,"caption":"The Couchbase Blog"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ec7e18bcee3337b77367560e6aeb75ef","name":"David Nault","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/fbceda752a7ba278f83b24641d607212","url":"https:\/\/secure.gravatar.com\/avatar\/9be1cd3e3c55df4d11b191172d382965de29da3a98d341167b33a8ad00546057?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9be1cd3e3c55df4d11b191172d382965de29da3a98d341167b33a8ad00546057?s=96&d=mm&r=g","caption":"David Nault"},"description":"David Nault is a Database Connector Developer for Couchbase. He spent the greater part of two decades writing back end web services in Java, and thinks JSON-RPC is really cool. He lives in California.","sameAs":["https:\/\/github.com\/dnault","https:\/\/duckduckgo.com\/?q=delete+facebook","https:\/\/x.com\/orzobat"],"url":"https:\/\/www.couchbase.com\/blog\/author\/dnault\/"}]}},"authors":[{"term_id":9070,"user_id":14341,"is_guest":0,"slug":"dnault","display_name":"David Nault","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/9be1cd3e3c55df4d11b191172d382965de29da3a98d341167b33a8ad00546057?s=96&d=mm&r=g","author_category":"","last_name":"Nault","first_name":"David","job_title":"","user_url":"https:\/\/github.com\/dnault","description":"David Nault writes code at Couchbase, where he works on the SDK & Connectors team."}],"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/5195","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/users\/14341"}],"replies":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/comments?post=5195"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/5195\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media\/13873"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media?parent=5195"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/categories?post=5195"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/tags?post=5195"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=5195"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}