With append and
prepend methods, you can add
information to the start or end of a binary data that already
exists in the data store. Both of these methods, along with the
incrementing and decrementing methods, are considered 'binary'
methods since they operate on binary data such as string or
integers, not JSON documents. These methods can add raw
serialized data to existing data for a key. The Couchbase Server
treats an existing value as a binary stream and concatenates the
new content to either beginning or end.
Both append and
prepend are atomic operations; this
means that multiple threads can be appending or pre-pending the
same key without accidentally overwriting changes from another
append/prepend request. Note however
that the order in which Couchbase Server appends or prepends
data is not guaranteed for concurrent
append/prepend requests.
Both append and
prepend originated from the request
that Couchbase Server supports 'lists' or sets. Developers
wanted to maintain documents representing the latest 100 RSS
feeds, or the latest 100 tweets about a certain topic, or
hash-tag. At this point, you can use it to maintain lists, but
be aware that the content needs to be a binary form, such as
strings or numeric information.
In the chapter on more advanced development topics, we provide
an example on managing a data set using
append; we provide the sample as a
Python script. Please refer to
Section 7.5.3, “Using the Fastest Methods”. You can also view the
entire blog post about the topic from Dustin Sallings at the
Couchbase blog,
Maintaining
a Set.
For purposes of this introduction to pre-pending and appending with Couchbase SDKs, we offer these illustrations to show how the two methods work. Imagine we are running an intergalactic empire, and we decide to knight/dame numerous users. We have also improved our document-keeping system and we want to enable users to have suffixes:
Notice that we provide the appropriate spacing and since the two
methods are separate Couchbase Server requests, Couchbase Server
updates the CAS value two times. Be aware that in some SDKs you
may provide prepend and
append with a CAS value as a parameter
in order to perform the operation; however no SDK requires it.
The next example demonstrates use of
append in Ruby. Note in this case,
providing a CAS value is optional. If provided however, it will
raise an error if the given CAS value does not match the CAS
value of the stored document:
#simple append operation and get c.set("foo", "aaa") c.append("foo", "bbb") c.get("foo") #=> "aaabbb" #append using CAS option ver = c.set("foo", "aaa") c.append("foo", "bbb", :cas => ver) #simple prepend c.set("foo", "aaa") c.prepend("foo", "bbb") c.get("foo") #=> "bbbaaa"
The following examples demonstrates
append and
prepend in Java. In this case the
get-with-cas operation in Java is gets:
/* get cas value for sample key and then append string */ CASValue<Object> casv = client.gets("samplekey"); client.append(casv.getCas(),"samplekey", "appendedstring"); /* handling possible errors using return value of append */ OperationFuture<Boolean> appendOp = client.append(casv.getCas(),"notsamplekey", "appendedstring"); try { if (appendOp.get().booleanValue()) { System.out.printf("Append succeeded\n"); } else { System.out.printf("Append failed\n"); } } catch (Exception e) { ... } /* prepend a string to an existing value */ CASValue<Object> casv = client.gets("samplekey"); client.prepend(casv.getCas(),"samplekey", "prependedstring");
The most significant error developers can make with
prepend and
append is that they repeatedly use the
method and create a value that is too large for Couchbase
Server. When append and
prepend add content to a document, they
do not remove the equivalent amount of content, such as removing
the oldest item from a list when new list content is added.
Therefore you can quickly reach the limit of data allowed for a
document if you do not keep track of it as you prepend or
append. The limit for values is 20MB so if you repeatedly use
these two methods, you may receive an error from the server as
well as inconsistent results. When you start getting these
errors you need to go back to your application logic, determine
how often you are actually triggering an
append and
prepend for the document. The three
possible approaches, which can be used simultaneously are:
Change Methods Used: Instead of using
append/prepend use a
set or add
and additional programming logic to add the new content, but
also remove old content. This will maintain a more constant
document size and reduce oversized documents.
Reduce Use: Reduce the number times you are appending and pre-pending, or reduce the amount of information you add to the document.
Split Documents: To avoid documents that are too large, logically separate the documents. For instance, instead of one document for set of tweets on Etsy products, break it up into several documents on different types of Etsy products, or tweets occuring during different time periods.
Compaction: In this case we explicitly remove data from a document if it makes sense to remove the data. This helps us avoid documents that are too large. For more information and an example implementation, view the entire blog post about the topic from Dustin Sallings at the Couchbase blog, Maintaining a Set.
Be aware that for Couchbase SDKs, if you try to
append or
prepend a different data-type to an
existing key, an SDK may perform no data cast, but rather
overwrite the entire value with the new value. For instance this
Ruby example shows an overwrite and cast:
c.set('karen', 'karen') #returns cas value for 'karen' c.get('karen').class #returns String c.prepend('karen', 2) #returns new cas value c.get('karen').class #Fixnum
The equivalent memcached protocol calls are
append and
prepend. These are the methods for
appending and prepending; for more information see,
memcached
protocol.
If you encounter a data type or generally data you did not expect, refer back to the methods that create your keys, as well as prepend or append them. Confirm that you provide the data as a consistent data-type.
If a key is missing, you will get a 'key does not exist' error in response. If you did not expect this result, you should check any application logic that creates that type of key, or any logic that deletes it may inadvertently cause this result. Another reason why you might get this result is that the item expired and Couchbase Server returns a 'key not found' type error. So you will want to check any logic that sets an explicit expiration set for that key.
The other error that can occur when you are prepending or appending is if you provide a bad value, such as a newline, or invalid character. Check the value you want to use with either of these methods so that it is valid.
The types of errors that can occur during this operation include 1) inability to connect to a node, or 2) some error exists while attempting to format a value being stored. If you have a connection-level error you may need to reattempt connection, and possibly check the status of the server. If you have an error with the size of your value or formatting, you need to check the value itself, and how it is encoded and see if there are any issues that make the document incompatible with Couchbase Server.
For more information about connections and connection-level settings, see Section 7.5.4, “Optimizing Client Instances” and Section 7.7.1, “Client-Side Timeouts”