Running Out of Memory in an application

Java SDK
1

Hi,

I am running out of memory in an application, Exception getting logged is :

java.lang.RuntimeException:
java.util.concurrent.TimeoutException
at com.couchbase.client.java.util.Blocking.blockForSingle(Blocking.java:93)
at com.couchbase.client.java.CouchbaseBucket.upsert(CouchbaseBucket.java:267)
at com.couchbase.client.java.CouchbaseBucket.upsert(CouchbaseBucket.java:262)
at
com.mmt.cache.dao.impl.UserQueryCouchbaseDaoImpl.storeToCouchbase(UserQueryCouchbaseDaoImpl.java:51)
at com.mmt.cache.dao.impl.AbstractCouchBaseDaoImpl.singlePutToCouchbase(AbstractCouchBaseDaoImpl.java:59)
at
com.mmt.cache.transformation.delegator.impl.CouchBaseServiceDelegatorImpl.putUntransformedPacket(CouchBaseServiceDelegatorImpl.java:381)
at com.mmt.cache.transformation.delegator.impl.CouchBaseServiceDelegatorImpl.put(CouchBaseServiceDelegatorImpl.java:155)
at
com.mmt.cache.core.AbstractCacheService.putUsingCouchBaseDelegator(AbstractCacheService.java:123)
at com.mmt.cache.core.impl.SimpleCacheService.put(SimpleCacheService.java:55)
at
com.mmt.cache.batching.service.impl.MultiPutBatchServiceImpl$1.run(MultiPutBatchServiceImpl.java:95)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

I am using operations as below :

Single get operation :

LegacyDocument document = bucket.get(key, LegacyDocument.class,1000,TimeUnit.MILLISECONDS);
if(document != null){
value = (String)document .content();
}

Single Put Operation:

protected void storeToCouchbase(String key, int ttl,String value){
LegacyDocument document = LegacyDocument.create(key, ttl, value);
client.upsert(document);
}

Multi Get Operation :
protected Map asyncBulkGet(final Collection ids, final Bucket bucket) {
Map responseMap = null;
try{
responseMap = Observable
.from(ids)
.flatMap(new Func1>() {
@Override
public Observable call(String id) {
return bucket.async().get(id, LegacyDocument.class);
}
})
.toMap(new Func1() {
@Override
public String call(LegacyDocument jsonDoc) {
if(jsonDoc != null){
return jsonDoc.id();
}
return null;
}
},new Func1() {
@Override
public Object call(LegacyDocument jsonDoc) {
if(jsonDoc != null){
return jsonDoc.content();
}
return null;
}
})
.toBlocking()
.single();

}catch(Exception e){

}
return responseMap;
}

I am using java-client 2.1.0 and couchbase server is 3.0.2

Need help here to understand what exactly I am doing wrong.

Thanks,
Bharat

2

Hi,

Will really appreciate if anyone can help here.

Thanks,
Bharat

3

A couple of things to check. Is your client app running close to the server? What can you tell us about the environment and setup.

Do you still see this if you use the default 2500ms timeout value? Note that value was selected because the OS is supposed to wait 1s before TCP retransmit. Some do, some don’t. If there was a retransmit, many items would time out in this case.

Are the timeouts always on store operations? Does the web console show any background fetches? Those would tie up the connection during the disk IO fetch.

Finally, look at cbstats timings. What does the cluster report for timing of handling of operations? That measure ignores network IO though.

4

Hi Matt,

We have Couch server and Application server in same DC

Yes we see this exception even if we keep default timeout and it is coming for both GET and PUT operations.

Important thing that we have noticed is : Same application is running fine with Couch 1.4.4 SDK problem we are facing is with Couch 2.1.0 SDK

Memory consumption is very high with Couch 2.1.0 .

Thanks,
Bharat

5

Hmmm. Current release is 2.1.2. I don’t know of a bug that affects this, but if it’s simple to do so you may want to grab the latest fixes.

Otherwise, I’d recommend upping the log level and filing an issue with the logs and cbstats timings.

6

Hi Matt,

Thanks, We Switched to Java SDK 2.1.2 and it seems to be working fine now.

7

Great to hear that it works, even if I’m not 100% sure why. Anyways, if you hit the issue again can you please open a ticket here: http://www.couchbase.com/issues/browse/JCBC

edit, from the related ticket:

@bkachhal hmmm that is actually not the best thing, I mean if it works its okay but its not how it should be. Can you raise an issue here: http://www.couchbase.com/issues/browse/JCBC, maybe its a bug in the LegacyDocument. Would it be possible for you to simulate with a JsonDocument and see if you have the same issues?