Migrating from Moxi to Couchbase SDK – Handling Data

[MickaelM]

Hello,

I am currently working on migrating an application from Moxi to the latest Couchbase SDK. However, I am facing an issue where data written by Moxi, particularly larger values, are compressed and I am not being able to read correctly by the new SDK.

Is there a configuration or setting in the Couchbase SDK that enables automatic compression and decompression (like memcached), allowing seamless reading and writing of data while maintaining its original format?

I appreciate any guidance you can provide.

Example:

[mreiche]

Hi @MickaelM - I think the best solution would be to get the data from Moxi in uncompressed form and let the Couchbase SDK compress it. By default, the Couchbase SDKs will compress documents.
Mike

[david.nault]

Hi @MickaelM,

If your application used the SpyMemcached client to talk to Moxi, the Sdk2CompatibleLegacyTranscoder included in the modern Couchbase Java SDK might do what you want.

Here’s some code you could use to experiment:

String connectionString = "couchbase://127.0.0.1";
String username = "Administrator";
String password = "password";
String bucketName = "default";

try (Cluster cluster = Cluster.connect(connectionString, username, password)) {
  Bucket bucket = cluster.bucket(bucketName);
  Collection collection = bucket.defaultCollection();

  GetResult result = collection.get(
    "e738bad9-test_key8",
    GetOptions.getOptions()
      .transcoder(Sdk2CompatibleLegacyTranscoder.INSTANCE)
  );
  System.out.println(result);

  // If you know it's a JSON Object...
  JsonObject jsonObject = JsonObject.fromJson(result.contentAs(String.class));

  String jsonToUpsert = JsonObject.create()
    .put("foo", "bar")
    .toString();
  collection.upsert(
    "some-other-key",
    jsonToUpsert,
    UpsertOptions.upsertOptions()
      .transcoder(Sdk2CompatibleLegacyTranscoder.INSTANCE)
  );
}

Note that Sdk2CompatibleLegacyTranscoder behaves differently than the standard (default) JsonTranscoder. To upsert a JSON document using the legacy transcoder, you must build the JSON string yourself, then upsert the string. Contrast this with the standard JSON transcoder, which uses a JsonSerializer to automatically convert the object you upsert into JSON.

Thanks,
David

EDIT: Oops, I just realized you posted this in the PHP SDK category, so Java sample code isn’t of much use. Sorry about that.

[david.nault]

This was all way before my time, but I think the Moxi clients may have used plain old GZIP compression, and marked a document as compressed by setting the flag 0x02.

Let me see if we can rustle up a PHP expert who might be able to show how to do that with the modern Couchbase PHP SDK.

[MickaelM]

Does the compress available in Community Edition? Other question, is it enabled by default?

[mreiche]

Compression is enabled by default in the SDKs except for .NET which needs … Compression | Couchbase Docs