Home | Forums | Couchbase | Couchbase Server 2.0

Bulk upsert with couchbase

1 reply [Last post]
Wed, 10/19/2011 - 13:04
josephgp
Offline
Joined: 10/19/2011
Groups: None

Hi,

I am considering CouchBase as part of our enterprise solution and had two questions:

1. We have a 20 million row table in RDBMS which we'd like to migrate to a Couchbase vBucket. The key challenge is that we need frequent bulk upserts of 500K+ or so to be made on recurring basis, often times from multiple clients.
Does CouchBase support bulk upserts at these volume levels? If so how? And are there any constraints?

2. Does Couchbase support replication or bulk export, so we can replicate data to external systems?

Thanks,
JGP

Top
Tue, 11/08/2011 - 11:00
ingenthr
Offline
Joined: 03/16/2010
Groups:

There should be no problem implementing an upsert from your client application. A cluster can certainly support that rate, depending on the configuration.

Logic will be something like:

GET key

if NOT_FOUND
SET key
else
while (CAS_FAIL)
// modify record
CAS key

Have a look at CAS operations for the client you plan to use. For example:
http://docs.couchbase.org/couchbase-sdk-java-getting-started/index.html
http://docs.couchbase.org/couchbase-sdk-java-api-reference/couchbase-sdk...

Couchbase (and Membase) have a generic protocol named TAP, which can be used for both replication and bulk export. As a matter of fact, Couchbase's internal replication is based on TAP. There are examples in the software distribution and several client libraries support TAP.

Top