Common Utilities for C C++ SDK functionality

Skip to end of metadata
Go to start of metadata
You are viewing an old version of this page. View the current version. Compare with Current  |   View Page History

This page will discuss some of the work required to unify and consolidate common tasks which are required in order for libcouchbase to function effectively in user environments.

As it stands, libcouchbase is primarily the protocol layer (both memcached and HTTP) for talking to the Couchbase server; however there are some higher level operations which may need further common handling:

Functional Requirements/Specifications

Durability API

This section will discuss the durability API which may be provided in a future utility library. The API should be neutral enough to be integrated into all libcouchbase-based SDKs.

It should provide an API to allow for the verification of a specific durability criteria across one or many nodes, and should be event-based:

API Inputs

Should follow the general format of an lcb_* command structure.

  • Key
  • Expected CAS
  • Number of servers to which the key should be persisted (or deleted) to
  • Number of servers to which the key should be replicated (or deleted) to
  • Poll interval (for retries between cycles)
  • Timeout interval (after which a response will be received indicating the durability requirements have not been satisfied)
  • Sense. This means whether the request is for a negative verification (as in the case of deletes), or positive verification (as is the case of mutations)

API Outputs

API Should follow the form of a lcb_*_resp_t. The callback should be invoked for the requested key (in the input) after either the durability requirement has been satisfied, or when it has been determined that the requirements may never be satisfied (e.g. different CAS on master), or when the timeout interval has expired.

  • Key
  • CAS (as reported by server)
  • Number of servers key was persisted to
  • Number of servers key was replicated to
  • Error code (this will be an lcb_error_t, either indicating success, or a reason for which the durability could not be satisfied)

Usage Model

This operation will present itself as a normal libcouchbase operation and will occupy a refcount in the event loop (that is, lcb_wait should not return until all pending "durability" operations have been completed.

The user should be able to schedule this operation either from a successful set/delete callback, OR as an individual wait sequence after lcb_wait. In either event the handling should be fairly opaque to the user.

Example Code

This code is from a WIP changeset in gerrit, but contains an outline and example of how it may be used:

View Parameter API

This API is predominantly a string library. It allows a consistent and symbolic way of handling and creating a Couchbase view query string. Its primary purpose is to provide a common interface and behavior for converting between "classical" and equivalent semantics to strict parameters as expected by the View REST API.

API Inputs

  • View Name
  • Design Document Name
  • List of query parameters. Each query parameter may be created using the following inputs:
    • String parameter (i.e. "group_level") OR a symbolic constant, e.g. LCB_VOPT_GROUPLEVEL
    • Value Type. This may be either an int, string, boolean, etc. - dependent on the parameter name. Each parameter may have a callback to verify and convert to the appropriate value
    • Value Conversion Options; these include:
      • Whether to validate the value
      • Whether to URI-escape the value
      • Whether the value pointer should be copied as an internal string
      • Whether to use "ad-hoc" mode (i.e. both parameter and value are user-supplied. Useful for supporting new options in the server without requiring an update to the SDK code)

API Outputs

As this is a string library, the output will always be a simple string. This string may be optionally URI-encoded but should be configurable (as it may be the case that the user is already passing a URI-encoded string). The string shall be allocated and free-able by using free(3)

Usage Model

This library feature is intended to be used in conjunction with other SDK-native features and functions. Specifically it intends to eliminate the option processing code which is fairly duplicate across all SDKs.

There may still be need to actually use a native JSON library to encode JSON parameters. How this is handled is SDK specific.

Example Code

As it exists in php-ext-couchbase

And a slightly-modified version, with some fixes:

View Row API

This API is intended to help with common pre-processing and buffering of view responses. This is particularly useful for large payloads. It also provides an option to incrementally fetch the views.

API Inputs

The view API inputs consist of feeding string data into an opaque parser context. The parser context will also be set up with callbacks which will be invoked upon receipt of each parsed view row, or on a parse error.

  • Opaque parse context (API function to initialize/create this)
  • Error callback
  • Row callback

API Output

  • Row Callback
    • Will contain a single row only
  • Error Callback
    • Will contain an error code, as well as the raw text for the response
  • Parse context
    • Will contain non-row meta-data returned in the view result, as well as any HTTP metadata (such as the return code)

Usage Model

The usage model for the incremental row parsing is slightly more involved as it requires setting up two different callbacks. Generally speaking however, the receipt of each view row is to follow the semantics of responses for something like a multi-get, in that a new callback is invoked for each row.

One is required to create a "view row parser context" (which may be re-used) which allocates the necessary handles for the parser object. Then incoming data is fed into the buffer (a convenience HTTP callback may be provided to allow for this boilerplate to be performed transparently). When sufficient data has been fed and at least one row has been internally delimited by the parser, the row callback is invoked. If an error occurs during parsing, the error callback is invoked.

Example Code

Common Flag/Datatype handling

 This may be a simple set of macros to set/unset flags

C++ Wrappers

It might be prudent to also create proper C++ wrappers around libcouchbase, so that one may be able to do something like:

LCB::Result& result = lcb->get("foo");

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.