Couchbase
  • Why NoSQL?
  • Couchbase Server
  • Download
  • Resources
  • Careers
Home | Forums | Couchbase | Couchbase Server 2.0

100 millions records?

1 reply [Last post]
  • Login or register to post comments
Tue, 11/29/2011 - 11:44
avyaktha
Offline
Joined: 11/11/2011
Groups: None

I am trying to load couchbase with 100 million records for a test

I have cluster with two servers (8 core / 8 GB) and have allocated 6 GB of ram per each server. I am using a protocol buffer object as value about 450 bytes and key is around 20 bytes. I am already using the 2.8-preview3 client. I am using a singleton client from a simple test program running a simple loop that does both the read an write , so its just one single thread that does the write.
np101 and np103 are my cluster machines and I am running the client from np102 (0.021 ms ping)

I tried pushing in 100 million records , I tried the synchronous write function as I wasn't concerned about time it took I just wanted the the DB to be loaded up

I have a setup as described in my earlier messages.About 80-90% of writes seem to be timing out. I think it uses the DEFAULT_OPERATION_TIMEOUT (2500) which seems to be reasonably high. I am writing to it in batches of 1000. 15 million is the amount of records I could push with about 10 attempts.

I understand that we are limited with speed of disk I/O here still 15 million seems to be too low a number to get so many timeouts. What is the recommend server configuration for a setup like this?

Please find below the source code

public CouchBasePerfTestClient() throws IOException, URISyntaxException {
List uris = new ArrayList();
uris.add(URI.create("http://np101.xxx.com:8091/pools"));
uris.add(URI.create("http://np103.xxx.com:8091/pools"));
couchbaseClient = new CouchbaseClient(uris, "default", "");
}

public void callCouchBase(int arg) throws IOException, ExecutionException, InterruptedException {

long ttime = 0;
List profiles = new ArrayList();
for (int i = 0; i < arg; i++) {
final ProfileBuffers.ProfilePayload.Builder builder = ProfileBuffers.ProfilePayload.newBuilder();
buildValue(builder, i);
profiles.add(builder.build());
if (i % BATCH_SIZE == 0) {
ttime += batchWriteToCouch(i, profiles);
}
}
System.out.println(ttime / NANO_DIVIDER + " secs for " + arg + " writes");
}

private long batchWriteToCouch(int i, List builder) {
long ptime = 0;
List> booleans = new ArrayList>();
for (ProfileBuffers.ProfilePayload next : builder) {
final long putStart = System.nanoTime();
final String key = "xxxxx.net@1828383838" + putStart + i;
try {
couchbaseClient.set(key, 0, next.toByteArray()).get();
} catch (InterruptedException e) {
} catch (Exception e) {
System.out.println("error "+e.getMessage());
}
ptime += (System.nanoTime() - putStart);
}
final int failedCount = countFailed(booleans);
System.out.println(" batch write took " + ptime / NANO_DIVIDER);
System.out.println(failedCount + "writes in batch failed");

return ptime;
}

Top
  • Login or register to post comments
Mon, 01/02/2012 - 23:49
ingenthr
Offline
Joined: 03/16/2010
Groups:

I think the main problem there is that you don't check the OperationFuture being returned. What's happening is that at some point, after you fill memory on the server, it will start returning TMP_OOM errors, which means you've filled memory and now you have to back off while the server frees up more memory and writes to disk.

You should consider replacing...
couchbaseClient.set(key, 0, next.toByteArray()).get();

...with a do-while loop that tries to set it, and does an incremental backoff if the set failed due to TMP_OOM will probably give you really high throughput in doing a data load on the server. I've done this myself a couple of times before and helped others be successful with the same appraoch.

The way your code will work it's just creating operations as fast as it can, and that means heavy memory allocation as fast as you can on the JVM. Chances are, the timeouts you're running into correlate to the full GCs you're hitting because you're generating a lot of garbage very, very quickly.

Top
  • Login or register to post comments
  • Login or register to post comments
  • Login
  • Register

Company

  • About Us
  • Leadership
  • Customers
  • Partners
  • Contact Us

Product

  • Couchbase Server
  • Couchbase SDKs
  • Use Cases
  • Documentation
  • Forums

Open Source

  • Couchbase Project
  • Couchbase vs. CouchDB

Commercial

  • Subscriptions & Support
  • Training & Services

News

  • Blog
  • Newsletter
  • Press Releases
  • Buzz

Follow Us

    
  • Customer Login
  • Terms of Service
  • Privacy Policy
  • Trademark Policy
  • Site Map

© 2013 COUCHBASE All rights reserved.

Sign in to Couchbase Community

close
  • Create new account
  • Request new password
You are logging into the Forums, Wiki and Issue Tracker