{"id":5432,"date":"2018-06-29T01:23:36","date_gmt":"2018-06-29T08:23:36","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=5432"},"modified":"2018-06-29T01:23:36","modified_gmt":"2018-06-29T08:23:36","slug":"inside-the-java-sdk-connection-management","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/inside-the-java-sdk-connection-management\/","title":{"rendered":"Inside the Java SDK: Connection Management"},"content":{"rendered":"

In this second instalment of “Inside the Java SDK” we are going to take an in-depth look at how the SDK manages and pools sockets to the various nodes and services. While not ultimately necessary to follow, I recommend you check out the first post on bootstrapping\u00a0<\/a>as well.<\/p>\n

Note that this post was written with the Java SDK 2.5.9 \/ 2.6.0 releases in mind. Things might change over time, but the overall approach should stay mostly the same.<\/p>\n

In the spirit of the OSI and TCP models, I propose a three layer model that represents the SDKs connection stack:<\/p>\n

+-----------------+\r\n| Service Layer   |\r\n+-----------------+\r\n| Endpoint Layer  |\r\n+-----------------+\r\n| Channel Layer   |\r\n+-----------------+<\/pre>\n
Higher levels build on top of lower levels so we’ll start with the Channel layer and work our way up the stack.<\/p>\n
The Channel Layer<\/h2>\n
The channel layer is the lowest level the SDK deals with networking and is built on top of the excellent, fully asynchronous IO library called Netty<\/a>\u00a0We’ve been extensive Netty users for years and also contributed patches as well as the memcache codec<\/a> back to the project.<\/p>\n
Every Netty Channel<\/a> corresponds to a socket and is multiplexed on top of event loops. We’ll cover the threading model in a later blog post, but for now it’s important to know that instead of the “one thread per socket” model of traditional blocking IO, Netty takes all open sockets and distributes them across a handful of event loops. It does this in a very efficient way, so it’s no wonder that Netty is used all over the industry<\/a>\u00a0for high performance and low latency networking components.<\/p>\n
Since a channel is only concerned with bytes going in and out, we need a way to encode and decode application level requests (like a N1QL query or a Key\/Value get request) into their proper binary representation. In Netty this is done by adding handlers<\/a> to the channel pipeline<\/a>. All network write operations work their way down the pipeline and server responses come back up the pipeline (also called inbound and outbound in Netty terminology).<\/p>\n
Some handlers are added independent of the service used (like logging or encryption) and others depend on the service type (for example for a N1QL response we have JSON streaming parsers customized to the response structure).<\/p>\n
If you ever wondered how to get packet-level logging output during development or debugging (for production use tcpdump, wireshark or similar), all you need to do is enable the TRACE log level in your favourite log library and you’ll see output like this:<\/p>\n
[cb-io-1-1] 2018-06-28 14:03:34 TRACE LoggingHandler:94 - [id: 0x41407638, L:\/127.0.0.1:60923 - R:localhost\/127.0.0.1:11210] WRITE: 243B\r\n+-------------------------------------------------+\r\n| 0 1 2 3 4 5 6 7 8 9 a b c d e f |\r\n+--------+-------------------------------------------------+----------------+\r\n|00000000| 80 1f 00 db 00 00 00 00 00 00 00 e5 00 00 00 00 |................|\r\n|00000010| 00 00 00 00 00 00 00 00 7b 22 61 22 3a 22 63 6f |........{\"a\":\"co|\r\n|00000020| 75 63 68 62 61 73 65 2d 6a 61 76 61 2d 63 6c 69 |uchbase-java-cli|\r\n|00000030| 65 6e 74 2f 32 2e 36 2e 30 2d 53 4e 41 50 53 48 |ent\/2.6.0-SNAPSH|\r\n|00000040| 4f 54 20 28 67 69 74 3a 20 32 2e 36 2e 30 2d 62 |OT (git: 2.6.0-b|\r\n|00000050| 65 74 61 2d 31 36 2d 67 35 63 65 30 38 62 30 2c |eta-16-g5ce08b0,|\r\n|00000060| 20 63 6f 72 65 3a 20 31 2e 36 2e 30 2d 62 65 74 | core: 1.6.0-bet|\r\n|00000070| 61 2d 33 33 2d 67 31 62 33 65 36 66 62 29 20 28 |a-33-g1b3e6fb) (|\r\n|00000080| 4d 61 63 20 4f 53 20 58 2f 31 30 2e 31 33 2e 34 |Mac OS X\/10.13.4|\r\n|00000090| 20 78 38 36 5f 36 34 3b 20 4a 61 76 61 20 48 6f | x86_64; Java Ho|\r\n|000000a0| 74 53 70 6f 74 28 54 4d 29 20 36 34 2d 42 69 74 |tSpot(TM) 64-Bit|\r\n|000000b0| 20 53 65 72 76 65 72 20 56 4d 20 31 2e 38 2e 30 | Server VM 1.8.0|\r\n|000000c0| 5f 31 30 31 2d 62 31 33 29 22 2c 22 69 22 3a 22 |_101-b13)\",\"i\":\"|\r\n|000000d0| 30 43 34 37 35 41 43 41 35 46 33 38 30 41 32 31 |0C475ACA5F380A21|\r\n|000000e0| 2f 30 30 30 30 30 30 30 30 34 31 34 30 37 36 33 |\/000000004140763|\r\n|000000f0| 38 22 7d |8\"} |\r\n+--------+-------------------------------------------------+----------------+<\/pre>\n
Note the little LoggingHandler<\/span>\u00a0 up there? This is because we only add the logging handler if tracing is enabled to the pipeline so you are not paying the overhead if you are not using it (which is most of the time):<\/p>\n
bootstrap = new BootstrapAdapter(new Bootstrap()\r\n  \/\/ *snip*\r\n  .option(ChannelOption.ALLOCATOR, allocator)\r\n  .option(ChannelOption.TCP_NODELAY, tcpNodelay)\r\n  .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, env.socketConnectTimeout())\r\n  .handler(new ChannelInitializer<Channel>() {\r\n    @Override\r\n    protected void initChannel(Channel channel) throws Exception {\r\n      ChannelPipeline pipeline = channel.pipeline();\r\n      if (env.sslEnabled()) {\r\n        pipeline.addLast(new SslHandler(sslEngineFactory.get()));\r\n      }\r\n      if (LOGGER.isTraceEnabled()) {\r\n        pipeline.addLast(LOGGING_HANDLER_INSTANCE);\r\n      }\r\n    customEndpointHandlers(pipeline);\r\n  }\r\n}));<\/pre>\n
You can also see that depending on the environment configuration we make other adjustments like adding a SSL\/TLS handler to the pipeline or configuring TCP nodelay and the socket timeouts.<\/p>\n
The customEndpointHandlers<\/span>\u00a0 method is overridden for each service, here is the pipeline for the KV layer (slightly simplified):<\/p>\n
if (environment().keepAliveInterval() > 0) {\r\n    pipeline.addLast(new IdleStateHandler(environment().keepAliveInterval(), 0, 0, TimeUnit.MILLISECONDS));\r\n}\r\n\r\npipeline\r\n    .addLast(new BinaryMemcacheClientCodec())\r\n    .addLast(new BinaryMemcacheObjectAggregator(Integer.MAX_VALUE));\r\n\r\npipeline\r\n    .addLast(new KeyValueFeatureHandler(context()))\r\n    .addLast(new KeyValueErrorMapHandler());\r\n\r\nif (!environment().certAuthEnabled()) {\r\n    pipeline.addLast(new KeyValueAuthHandler(username(), password(), environment().forceSaslPlain()));\r\n}\r\n\r\npipeline\r\n    .addLast(new KeyValueSelectBucketHandler(bucket()))\r\n    .addLast(new KeyValueHandler(this, responseBuffer(), false, true));<\/pre>\n
Lots going on here! Let’s go through it one by one:<\/p>\n