[JCBC-189] Views having odd timeout issues on some clusters Created: 18/Dec/12  Updated: 23/Jul/13  Resolved: 23/Jul/13

Status: Closed
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File failover-debug.args     Text File fo-dbg.log.txt     File vm-4nodes-2.ini    
Issue Links:
Dependency
depends on MB-7661 [Done, RN 2.1.0] View query on Rebala... Resolved
blocks JCBC-176 Using BucketTool helper - error in co... Resolved

 Description   
We're seeing really strange timeout issues which seem to affect only this specific cluster and only in terms of views. We've re-installed this cluster time and time again, and we've had similar configurations run successfully as well.

More details in comment..

 Comments   
Comment by Mark Nunberg [ 18/Dec/12 ]
Just try to do this with a simple java script (i.e. connect and query the view)
Comment by Deepti Dawar [ 20/Dec/12 ]
This issue persists when the view retrieval is ran from standalone java program as well.
I tried this on both the VMs - 10.3.3.203, 10.3.3.209.
Comment by Deepti Dawar [ 20/Dec/12 ]
Its working fine for server deployed on localhost.
Comment by Matt Ingenthron [ 02/Jan/13 ]
Michael: found out today that this is a large issue for SDKQE. Can you have a quick look at this in the next day? You may find the underlying issue.
Comment by Michael Nitschinger [ 03/Jan/13 ]
Please pass me the script as commented and then (or if you can't) please assign it back to me! Thanks
Comment by Michael Nitschinger [ 18/Jan/13 ]
According to the posted logs, it looks like debugging was not turned on. Can you please run this again with debugging turned on? To get the full logs to STDOUT, use this before initializing the CouchbaseClient in The App:

      // Tell spy to use the SunLogger
        Properties systemProperties = System.getProperties();
        systemProperties.put("net.spy.log.LoggerImpl", "net.spy.memcached.compat.log.SunLogger");
        System.setProperties(systemProperties);

        Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST);

        //get the top Logger
        Logger topLogger = java.util.logging.Logger.getLogger("");

        // Handler for console (reuse it if it already exists)
        Handler consoleHandler = null;
        //see if there is already a console handler
        for (Handler handler : topLogger.getHandlers()) {
            if (handler instanceof ConsoleHandler) {
                //found the console handler
                consoleHandler = handler;
                break;
            }
        }

        if (consoleHandler == null) {
            //there was no console handler found, create a new one
            consoleHandler = new ConsoleHandler();
            topLogger.addHandler(consoleHandler);
        }

        //set the console handler to fine:
        consoleHandler.setLevel(java.util.logging.Level.FINEST);


Would be great if we can get all the output so we can investigate where the timeouts come from. I'm sure with a debug log it will be much easier. Thanks!
Comment by Mark Nunberg [ 23/Jan/13 ]
So this bug isn't such an "unknown" anymore, and has exposed itself in 1.1.0 as well as 1.1.1 and isn't limited to particular clusters - (it just seems that some clusters are more likely than others to trigger this bug).

However this still needs a lot of care and analysis
Comment by Michael Nitschinger [ 01/Feb/13 ]
This is "kinda" blocker.
Comment by Deepti Dawar [ 23/Jul/13 ]
JCBC-189 was primarily related to the timeouts and environmental issues. Now, after some progress in this, its figured out that these env related issues have been ironed out with the latest set up of the client and server VMs.
Also, the issue that remains now is very specific to the cluster and the client which can be tracked as part of a new JCBC issue - JCBC-333
Generated at Tue Sep 23 05:47:34 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.