{"id":1844,"date":"2014-12-16T17:28:59","date_gmt":"2014-12-16T17:28:58","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=1844"},"modified":"2014-12-16T17:28:59","modified_gmt":"2014-12-16T17:28:58","slug":"writing-your-own-storage-engine-memcached-part-3","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/","title":{"rendered":"Writing your own storage engine for Memcached, part 3"},"content":{"rendered":"

Right now we've got an engine capable of running get and set load, but it is doing synchrounus filesystem IO. We can't serve our client faster than we can read the item from disk, but we might serve other<\/b> connections while we're reading the item off disk.<\/p>\n

In this entry we're going to fix our get<\/font><\/code> and store<\/font><\/code> method so that they don't block the engine API. As I've said earlier, the intention of this tutorial is to focus on the engine API<\/em>. That means I'm not going to try to make an effective design, because that could distract the focus from what I'm trying to explain. If people are interested in how we could optimize this, I could add a second part of the tutorial in the future… Just let me know.<\/p>\n

In order to implement asynchronous operations in our engine, we need to make use of the API the server makes available to us in create_instance<\/font><\/code>. Let's extend our engine\u00a0structure to keep track of the server API:<\/p>\n

\u00a0<\/p>\n

\n
struct fs_engine {
\u00a0 \u00a0ENGINE_HANDLE_V1 engine;
\u00a0 \u00a0SERVER_HANDLE_V1 sapi;<\/b>
};<\/p>\n

…<\/p>\n

MEMCACHED_PUBLIC_API
ENGINE_ERROR_CODE create_instance(uint64_t interface,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 GET_SERVER_API get_server_api,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_HANDLE **handle)
{<\/p>\n

…<\/p>\n

\u00a0 \u00a0h->engine.item_set_cas = fs_item_set_cas;
\u00a0 \u00a0h->engine.get_item_info = fs_get_item_info;
\u00a0 \u00a0h->sapi = *get_server_api();<\/b><\/p>\n

…<\/p>\n

\u00a0 \u00a0 \u00a0 <\/p><\/div>\n<\/div>\n

To implement an\u00a0asynchronous\u00a0function in the engine interface, the engine needs to dispatch the request to another thread before it return ENGINE_EWOULDBLOCK<\/font><\/code> from the engine function. When the backend is done processing the result, it notifies the memcached core by using the notify_io_complete<\/font><\/code> function in the server interface. If an error\u00a0occurred\u00a0while processing the request, the memcached core will return the error message to the client. If your engine called notify_io_complete<\/font><\/code> with ENGINE_SUCCESS<\/font><\/code>, the memcached core will call the engine interface function once more with the same argument as the
first time.<\/p>\n

If you look at the server api, you'll see that the it got an interface for storing an engine-specific pointer. This will make our life easier when we want to implement async io. So let's go ahead and update our fs_get<\/font><\/code> method:<\/p>\n

\u00a0<\/p>\n

\n
static ENGINE_ERROR_CODE fs_get(ENGINE_HANDLE* handle,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const void* cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 item** item,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const void* key,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const int nkey,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 uint16_t vbucket)
{
\u00a0 \u00a0struct fs_engine *engine = (struct fs_engine *)engine;
\u00a0 \u00a0void *res = engine->sapi.cookie->get_engine_specific(cookie);
\u00a0 \u00a0if (res != NULL) {
\u00a0 \u00a0 \u00a0 *item = res;
\u00a0 \u00a0 \u00a0 engine->sapi.cookie->store_engine_specific(cookie, NULL);
\u00a0 \u00a0 \u00a0 return ENGINE_SUCCESS;<\/p>\n

\u00a0 \u00a0}<\/p>\n

…
\u00a0 \u00a0 \u00a0 <\/div>\n<\/div>\n

The next thing we need to do is to create a function that runs asynchronously and stores the result in the engine_specific setting for the cookie. Since we're going to use async tasks for\u00a0all of the engine methods, let's go ahead and create a function to run tasks in another thread:<\/p>\n

\u00a0<\/p>\n

\n
static ENGINE_ERROR_CODE execute(struct task *task)
{
\u00a0 \u00a0pthread_attr_t attr;
\u00a0 \u00a0pthread_t tid;<\/p>\n

\u00a0 \u00a0if (pthread_attr_init(&attr) != 0 ||
\u00a0 \u00a0 \u00a0 \u00a0pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED) != 0 ||
\u00a0 \u00a0 \u00a0 \u00a0pthread_create(&tid, &attr, task->callback, task) != 0) {
\u00a0 \u00a0 \u00a0 return ENGINE_FAILED;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0return ENGINE_EWOULDBLOCK;
}
\u00a0 \u00a0 \u00a0 <\/div>\n<\/div>\n

As you can see from the code I'm going to create a new thread to execute each operation. This isn't very efficient, because creating a new thread got a substansial overhead. In your\u00a0design you would probably want a pool of threads to run your tasks.<\/p>\n

The newly created thread would run the specialized callback with a pointer to the task as it's only argument. So what does this task structure look like?<\/p>\n

\u00a0<\/p>\n

\n
struct task {
\u00a0 \u00a0struct fs_engine *engine; \/* Pointer to the engine *\/
\u00a0 \u00a0const void *cookie; \/* The cookie requesting the operation *\/
\u00a0 \u00a0void *(*callback)(void *arg);
\u00a0 \u00a0union {
\u00a0 \u00a0 \u00a0 struct {
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0char key[PATH_MAX];
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0size_t nkey;
\u00a0 \u00a0 \u00a0 } get; \/* Data used by the get operation *\/
\u00a0 \u00a0 \u00a0 struct {
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0item *item;
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0ENGINE_STORE_OPERATION operation;
\u00a0 \u00a0 \u00a0 } store; \/* Data used by the store operation *\/
\u00a0 \u00a0} data;
};
\u00a0 \u00a0 \u00a0 <\/div>\n<\/div>\n

So let's finish up fs_get<\/font><\/code>:<\/p>\n

\u00a0<\/p>\n

\n
static ENGINE_ERROR_CODE fs_get(ENGINE_HANDLE* handle,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const void* cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 item** item,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const void* key,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const int nkey,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 uint16_t vbucket)
{
\u00a0 \u00a0struct fs_engine *engine = (struct fs_engine *)handle;
\u00a0 \u00a0\/* Check to see if this is the callback from an earlier ewouldblock *\/
\u00a0 \u00a0void *res = engine->sapi.cookie->get_engine_specific(cookie);
\u00a0 \u00a0if (res != NULL) {
\u00a0 \u00a0 \u00a0 *item = res;
\u00a0 \u00a0 \u00a0 engine->sapi.cookie->store_engine_specific(cookie, NULL);
\u00a0 \u00a0 \u00a0 return ENGINE_SUCCESS;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0\/* We don’t support keys longer than PATH_MAX *\/
\u00a0 \u00a0if (nkey >= PATH_MAX) {
\u00a0 \u00a0 \u00a0 return ENGINE_FAILED;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0\/* Set up the callback struct *\/
\u00a0 \u00a0struct task *task = calloc(1, sizeof(*task));
\u00a0 \u00a0if (task == NULL) {
\u00a0 \u00a0 \u00a0 return ENGINE_ENOMEM;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0task->engine = (struct fs_engine *)handle;
\u00a0 \u00a0task->cookie = cookie;
\u00a0 \u00a0task->callback = fs_get_callback;
\u00a0 \u00a0memcpy(task->data.get.key, key, nkey);
\u00a0 \u00a0task->data.get.nkey = nkey;<\/p>\n

\u00a0 \u00a0ENGINE_ERROR_CODE ret = execute(task);
\u00a0 \u00a0if (ret != ENGINE_EWOULDBLOCK) {
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0}
\u00a0 \u00a0return ret;
}
\u00a0 \u00a0 \u00a0 <\/div>\n<\/div>\n

If you look at the code above, you'll see that we specify gs_get_callback<\/font><\/code> as the function to execute. So let's go ahead and implement the callback:<\/p>\n

\u00a0<\/p>\n

\n
static void *fs_get_callback(void *arg)
{
\u00a0 \u00a0struct task *task = arg;
\u00a0 \u00a0char *fname = task->data.get.key;
\u00a0 \u00a0task->data.get.key[task->data.get.nkey] = ‘\u0000’;<\/p>\n

\u00a0 \u00a0struct stat st;
\u00a0 \u00a0if (stat(fname, &st) == -1) {
\u00a0 \u00a0 \u00a0 task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_KEY_ENOENT);
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0 \u00a0 return NULL;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0struct fs_item* it = NULL;
\u00a0 \u00a0ENGINE_ERROR_CODE ret = fs_allocate((ENGINE_HANDLE*)task->engine,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0task->cookie, (void**)&it,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0task->data.get.key,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0task->data.get.nkey,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0st.st_size, 0, 0);
\u00a0 \u00a0if (ret != ENGINE_SUCCESS) {
\u00a0 \u00a0 \u00a0 task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_ENOMEM);
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0 \u00a0 return NULL;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0FILE *fp = fopen(fname, “r”);
\u00a0 \u00a0if (fp == NULL) {
\u00a0 \u00a0 \u00a0 fs_release((ENGINE_HANDLE*)task->engine, task->cookie, it);
\u00a0 \u00a0 \u00a0 task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_FAILED);
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0 \u00a0 return NULL;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0size_t nr = fread(it->data, 1, it->ndata, fp);
\u00a0 \u00a0fclose(fp);
\u00a0 \u00a0if (nr != it->ndata) {
\u00a0 \u00a0 \u00a0 fs_release((ENGINE_HANDLE*)task->engine, task->cookie, it);
\u00a0 \u00a0 \u00a0 task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_FAILED);
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0 \u00a0 return NULL;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0task->engine->sapi.cookie->store_engine_specific(task->cookie, it);
\u00a0 \u00a0task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0ENGINE_SUCCESS);
\u00a0 \u00a0return NULL;
}
\u00a0 \u00a0 \u00a0 <\/div>\n<\/div>\n

As you see it's quite easy to add asynchronous support for the engine functions. Let's go ahead and do the same for fs_store<\/font><\/code>:<\/p>\n

\u00a0<\/p>\n

\n
static ENGINE_ERROR_CODE fs_store(ENGINE_HANDLE* handle,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 const void *cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 item* item,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 uint64_t *cas,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_STORE_OPERATION operation,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 uint16_t vbucket)
{
\u00a0 \u00a0struct fs_engine *engine = (struct fs_engine *)handle;
\u00a0 \u00a0\/* Check to see if this is the callback from an earlier ewouldblock *\/
\u00a0 \u00a0void *res = engine->sapi.cookie->get_engine_specific(cookie);
\u00a0 \u00a0if (res != NULL) {
\u00a0 \u00a0 \u00a0 *cas = 0;
\u00a0 \u00a0 \u00a0 engine->sapi.cookie->store_engine_specific(cookie, NULL);
\u00a0 \u00a0 \u00a0 return ENGINE_SUCCESS;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0\/* Set up the callback struct *\/
\u00a0 \u00a0struct task *task = calloc(1, sizeof(*task));
\u00a0 \u00a0if (task == NULL) {
\u00a0 \u00a0 \u00a0 return ENGINE_ENOMEM;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0task->engine = (struct fs_engine *)handle;
\u00a0 \u00a0task->cookie = cookie;
\u00a0 \u00a0task->callback = fs_store_callback;
\u00a0 \u00a0task->data.store.item = item;
\u00a0 \u00a0task->data.store.operation = operation;<\/p>\n

\u00a0 \u00a0ENGINE_ERROR_CODE ret = execute(task);
\u00a0 \u00a0if (ret != ENGINE_EWOULDBLOCK) {
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0}
\u00a0 \u00a0return ret;
}<\/div>\n<\/div>\n

And fs_store_callback<\/font><\/code> looks like the following:<\/p>\n

\u00a0<\/p>\n

\n
static void *fs_store_callback(void *arg)
{
\u00a0 \u00a0struct task *task = arg;
\u00a0 \u00a0struct fs_item* it = task->data.store.item;
\u00a0 \u00a0char fname[it->nkey + 1];
\u00a0 \u00a0memcpy(fname, it->key, it->nkey);
\u00a0 \u00a0fname[it->nkey] = ‘\u0000’;<\/p>\n

\u00a0 \u00a0FILE *fp = fopen(fname, “w”);
\u00a0 \u00a0if (fp == NULL) {
\u00a0 \u00a0 \u00a0 task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_NOT_STORED);
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0 \u00a0 return NULL;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0size_t nw = fwrite(it->data, 1, it->ndata, fp);
\u00a0 \u00a0fclose(fp);
\u00a0 \u00a0if (nw != it->ndata) {
\u00a0 \u00a0 \u00a0 remove(fname);
\u00a0 \u00a0 \u00a0 task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ENGINE_NOT_STORED);
\u00a0 \u00a0 \u00a0 free(task);
\u00a0 \u00a0 \u00a0 return NULL;
\u00a0 \u00a0}<\/p>\n

\u00a0 \u00a0task->engine->sapi.cookie->store_engine_specific(task->cookie, it);
\u00a0 \u00a0task->engine->sapi.cookie->notify_io_complete(task->cookie,
\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0ENGINE_SUCCESS);
\u00a0 \u00a0return NULL;
}<\/div>\n<\/div>\n

If you look closely at the code above you'll see that we still don't differentiate between add<\/font><\/code>\/set<\/font><\/code>\/replace<\/font><\/code>, but we'll fix that in the next session.<\/p>\n","protected":false},"excerpt":{"rendered":"

Right now we've got an engine capable of running get and set load, but it is doing synchrounus filesystem IO. We can't serve our client faster than we can read the item from disk, but we might serve other connections […]<\/p>\n","protected":false},"author":14,"featured_media":13873,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"ppma_author":[8981],"class_list":["post-1844","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"acf":[],"yoast_head":"\nWriting your own storage engine for Memcached, part 3 - The Couchbase Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Writing your own storage engine for Memcached, part 3\" \/>\n<meta property=\"og:description\" content=\"Right now we've got an engine capable of running get and set load, but it is doing synchrounus filesystem IO. We can't serve our client faster than we can read the item from disk, but we might serve other connections […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\" \/>\n<meta property=\"og:site_name\" content=\"The Couchbase Blog\" \/>\n<meta property=\"article:published_time\" content=\"2014-12-16T17:28:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2022\/11\/couchbase-nosql-dbaas.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1800\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Trond Norbye, Senior Developer, Couchbase\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Trond Norbye, Senior Developer, Couchbase\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\"},\"author\":{\"name\":\"Trond Norbye, Senior Developer, Couchbase\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ef0f9ff42d878c2fc0ab3a685e96f36d\"},\"headline\":\"Writing your own storage engine for Memcached, part 3\",\"datePublished\":\"2014-12-16T17:28:58+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\"},\"wordCount\":1305,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\",\"name\":\"Writing your own storage engine for Memcached, part 3 - The Couchbase Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"datePublished\":\"2014-12-16T17:28:58+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"width\":1800,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.couchbase.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Writing your own storage engine for Memcached, part 3\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"name\":\"The Couchbase Blog\",\"description\":\"Couchbase, the NoSQL Database\",\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\",\"name\":\"The Couchbase Blog\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"width\":218,\"height\":34,\"caption\":\"The Couchbase Blog\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ef0f9ff42d878c2fc0ab3a685e96f36d\",\"name\":\"Trond Norbye, Senior Developer, Couchbase\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/03d13b4ab5eaa14c91cab7658f04df07\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/bf3c00ca228efa25f7bfc168f566d6389279b44d4bbba4683c260a8bf33da03d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/bf3c00ca228efa25f7bfc168f566d6389279b44d4bbba4683c260a8bf33da03d?s=96&d=mm&r=g\",\"caption\":\"Trond Norbye, Senior Developer, Couchbase\"},\"description\":\"Trond Norbye is a Software Architect at Couchbase. Core contributor to Couchbase & Memcached projects. Created the C\/C++ & node.js Couchbase client libraries.\",\"url\":\"https:\/\/www.couchbase.com\/blog\/author\/trond-norbye\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Writing your own storage engine for Memcached, part 3 - The Couchbase Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/","og_locale":"en_US","og_type":"article","og_title":"Writing your own storage engine for Memcached, part 3","og_description":"Right now we've got an engine capable of running get and set load, but it is doing synchrounus filesystem IO. We can't serve our client faster than we can read the item from disk, but we might serve other connections […]","og_url":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/","og_site_name":"The Couchbase Blog","article_published_time":"2014-12-16T17:28:58+00:00","og_image":[{"width":1800,"height":630,"url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2022\/11\/couchbase-nosql-dbaas.png","type":"image\/png"}],"author":"Trond Norbye, Senior Developer, Couchbase","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Trond Norbye, Senior Developer, Couchbase","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#article","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/"},"author":{"name":"Trond Norbye, Senior Developer, Couchbase","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ef0f9ff42d878c2fc0ab3a685e96f36d"},"headline":"Writing your own storage engine for Memcached, part 3","datePublished":"2014-12-16T17:28:58+00:00","mainEntityOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/"},"wordCount":1305,"commentCount":0,"publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/","url":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/","name":"Writing your own storage engine for Memcached, part 3 - The Couchbase Blog","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","datePublished":"2014-12-16T17:28:58+00:00","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#primaryimage","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","width":1800,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/blog\/writing-your-own-storage-engine-memcached-part-3\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Writing your own storage engine for Memcached, part 3"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/blog\/#website","url":"https:\/\/www.couchbase.com\/blog\/","name":"The Couchbase Blog","description":"Couchbase, the NoSQL Database","publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/blog\/#organization","name":"The Couchbase Blog","url":"https:\/\/www.couchbase.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","width":218,"height":34,"caption":"The Couchbase Blog"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/ef0f9ff42d878c2fc0ab3a685e96f36d","name":"Trond Norbye, Senior Developer, Couchbase","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/03d13b4ab5eaa14c91cab7658f04df07","url":"https:\/\/secure.gravatar.com\/avatar\/bf3c00ca228efa25f7bfc168f566d6389279b44d4bbba4683c260a8bf33da03d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/bf3c00ca228efa25f7bfc168f566d6389279b44d4bbba4683c260a8bf33da03d?s=96&d=mm&r=g","caption":"Trond Norbye, Senior Developer, Couchbase"},"description":"Trond Norbye is a Software Architect at Couchbase. Core contributor to Couchbase & Memcached projects. Created the C\/C++ & node.js Couchbase client libraries.","url":"https:\/\/www.couchbase.com\/blog\/author\/trond-norbye\/"}]}},"authors":[{"term_id":8981,"user_id":14,"is_guest":0,"slug":"trond-norbye","display_name":"Trond Norbye, Senior Developer, Couchbase","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/bf3c00ca228efa25f7bfc168f566d6389279b44d4bbba4683c260a8bf33da03d?s=96&d=mm&r=g","author_category":"","last_name":"Norbye","first_name":"Trond","job_title":"","user_url":"","description":"Trond Norbye is a Software Architect at Couchbase. Core contributor to Couchbase & Memcached projects. Created the C\/C++ & node.js Couchbase client libraries."}],"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/1844","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/comments?post=1844"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/1844\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media\/13873"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media?parent=1844"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/categories?post=1844"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/tags?post=1844"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1844"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}