[NCBC-615] [ssl] data load stuck with 'Socket Exception was unhandled' Created: 29/Aug/14  Updated: 29/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File NCBC-615_1.png     PNG File NCBC-615_2.png     PNG File NCBC-615_3.png     Zip Archive NCBC-615.zip     File test.logs    

 Description   
Hi Jeff,

I don't see the issue without ssl but see it in maby tests with ssl.
I'm not 100% sure that it is the client problem, but I use the same sdkd code and there is no such problem.

in short, I see that not all the data has been downloaded, and tests hang. when I stop the command line to debug, I see the following exception. Could you take a look at the problem? my research have so far failed. I am ready to provide the environment or provide more detailed information if required.

 Comments   
Comment by Jeff Morris [ 29/Aug/14 ]
Can you post the stacktrace? Click on the "View Detail" in the exception dialog and navigate through. If an inner exception exists, make sure it's included.





[NCBC-614] Review and reduce verbosity of logging in INFO mode Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency




[NCBC-613] Refactor operation result classes so that they have a common inheritence chain Created: 28/Aug/14  Updated: 28/Aug/14

Status: In Progress
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently the "operation result" from each of the API's supported by the SDK (Views, N1QL, Memcached, Document) all have different "operation result" classes, which is confusing and inconsistent. This refactoring will make a common inheritance chain for all derived "operation result" classes.




[NCBC-612] Provide configurable timeouts for View and N1QL queries Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
These should default to certain values at the ClientConfiguration level and be overridable at the BucketConfiguration and the query levels themselves.




[NCBC-611] CouchbaseAspNet 1.2.1 Nuget package was released built in debug mode Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Stephen Phillips Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
v1.2.1 of "Couchbase ASP.NET SessionState and OutputCache Providers" on nuget the dll included in the package was built in debug mode and was raised to us when we sent a dump file to microsoft for anaylsis. This has been suggested as a potential performance issue. An updated release build would be appreciated.

 Comments   
Comment by Stephen Phillips [ 28/Aug/14 ]
Apologies if this is the wrong project I did not know where else to create the issue.
Comment by Jeff Morris [ 28/Aug/14 ]
Thanks Stephen, I'll look into it. It's fine to create issues for that project here :)




[NCBC-610] [ssl] data load is very slow: 1 ops/sec(regression) Created: 28/Aug/14  Updated: 28/Aug/14  Resolved: 28/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive log_If457e837f.zip     Text File log.txt     PNG File without_ssl.png     PNG File with_ssl_If457e837fdda.png     PNG File with_ssl.png    

 Description   
Hi Jeff,

I see a significant drop in performance with ssl

please see screenshots




 Comments   
Comment by Jeff Morris [ 28/Aug/14 ]
Running against my local cluster (VMs) the performance is nearly identical. From the logs you provided, there is something not right with your config:

2014-08-27 23:35:14,921 [4] DEBUG Couchbase.Core.ClusterManager - Trying to boostrap with Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.
2014-08-27 23:35:14,939 [4] DEBUG Couchbase.Configuration.Server.Providers.ConfigProviderBase - Getting config for bucket default
2014-08-27 23:35:14,950 [4] DEBUG Couchbase.Core.ClusterManager - Creating DefaultIOStrategy
2014-08-27 23:35:14,951 [4] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Creating DefaultIOStrategy 642ecc71-964b-42bb-aff6-4d691605c598
2014-08-27 23:35:14,960 [4] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-27 23:35:14,998 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire new: 2fee4eef-d170-4af2-b92f-4856dfc8e359 | 10.3.121.134:11207 | [0, 0] - 154ff9f2-fc7c-452a-98f7-614407abd6dc - Disposed: False
2014-08-27 23:35:17,517 [4] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-27 23:35:17,529 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire new: 25f51e8c-8149-4415-82a3-16c8bd183139 | 10.3.121.134:11207 | [0, 1] - 154ff9f2-fc7c-452a-98f7-614407abd6dc - Disposed: False
2014-08-27 23:35:20,035 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: 25f51e8c-8149-4415-82a3-16c8bd183139 on 10.3.121.134:11207 - 154ff9f2-fc7c-452a-98f7-614407abd6dc
2014-08-27 23:35:20,038 [4] WARN Couchbase.Core.ClusterManager - Couchbase.Configuration.ConfigException: Could not retrieve configuration for default. Reason: ??
   at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 67
   at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159

After trying CCCP, the client defaults to HTTP streaming and configuration continues. But every op fails:

2014-08-27 23:35:33,050 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: f268e1c1-1cbf-41bb-ba70-3c88ae9f4e14 on 10.3.121.135:11207 - 6b53a72a-c726-4641-b34c-f30f71ba14bd
2014-08-27 23:35:33,052 [4] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation doesn't support retries for key ViewFillerSeedrep0rep0rep0
2014-08-27 23:35:33,052 [4] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation for key ViewFillerSeedrep0rep0rep0 failed after 0 retries. Reason: ?
2014-08-27 23:35:33,065 [4] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#429 on thread 4
2014-08-27 23:35:33,065 [4] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 58 for key ViewFillerSeedrep1rep1rep1 - rev0
2014-08-27 23:35:33,065 [4] DEBUG Couchbase.Core.Server - Sending ViewFillerSeedrep1rep1rep1 using server 10.3.121.134:11207
2014-08-27 23:35:33,066 [4] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-27 23:35:33,079 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire new: 471d6ca5-cd63-48da-8d99-369897e727b2 | 10.3.121.134:11207 | [0, 1] - 9884859d-0bcd-4732-

Try checking your configuration and see if their is some error there.

-Jeff
Comment by Andrei Baranouski [ 28/Aug/14 ]
Hi Jeff,

I I rolled back couchbase-net-client to version

$ git reset --hard HEAD^
HEAD is now at 0b95eb0 NCBC-604: NullReferenceException in OperationBase`1.HandleClientError

Administrator@WIN-335SPEPOCGT /c/jenkins/workspace/.net-sdkdclient-ng2-andrei/couchbase-net-client (master)
$ git log
WARNING: terminal is not fully functional
commit 0b95eb0e888c394c00048b66f15fe7fb0fbf5146
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Tue Aug 26 14:28:57 2014 -0700

    NCBC-604: NullReferenceException in OperationBase`1.HandleClientError

    Change-Id: If457e837fdda073b51b370a588863200910a29be
    Reviewed-on: http://review.couchbase.org/40938
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Brett Lawson <brett19@gmail.com>

and loading looks fine( there are no changes in sdkd frameworks)

see attached screenshot and log with this revision



Comment by Jeff Morris [ 28/Aug/14 ]
Thanks Andrei, I'll look deeper into it this afternoon, as you mentioned it's probably a regression.
Comment by Jeff Morris [ 28/Aug/14 ]
http://review.couchbase.org/#/c/41064/




[NCBC-609] Number base not specified for error codes Created: 27/Aug/14  Updated: 28/Aug/14  Resolved: 28/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: docs
Affects Version/s: 1.3.8
Fix Version/s: 1.3.8, 1.3.9

Type: Bug Priority: Major
Reporter: Amy Kurtzman Assignee: Amy Kurtzman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A customer reported the following:

Hi,
I’m not sure who is in charge of documentation or how to reach said person so I sent this mail to info, I hope this reaches the right person.
 
I wanted to point out a small mistake in the .NET SDK documentation (1.3).
The error codes you have listed on the site are in hexadecimal base but without any mention that these values are in hexadecimal base.
See: http://docs.couchbase.com/couchbase-sdk-net-1.3/#checking-error-codes
For example: OperationTimeout is stated to be 94, it’s actually 148which in hexadecimal is 0x94.
I think this should be changed to base10 or at least clarified.
 
Cheers, Yosef Orbach.

 Comments   
Comment by Amy Kurtzman [ 27/Aug/14 ]
@Jeff, Can you verify that the numbers are indeed in hex? If so, I'll add a note and prefix the numbers with 0x. Thanks.
Comment by Jeff Morris [ 27/Aug/14 ]
Yes, the values are indeed in hex.




[NCBC-608] Implement Common Flags Created: 27/Aug/14  Updated: 27/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Brett Lawson Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Implement the Common Flags formatting as defined by the specification.

 Comments   
Comment by Jeff Morris [ 27/Aug/14 ]
Brett, can you link to the specification?
Comment by Brett Lawson [ 27/Aug/14 ]
https://docs.google.com/document/d/1V653a6FF6DOqdT4d-fKIjGkHabDaNGZsvbtsUKJyeLc




[NCBC-607] [ssl] NullReferenceException at Couchbase.IO.Strategies.SslConnection.Send ( first CRUD operation) Created: 27/Aug/14  Updated: 27/Aug/14  Resolved: 27/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File log.txt     PNG File NCBC-607.png     File test.logs    

 Description   
call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
ig.ini -I sdkd.args --variants HYBRID -d all:trace
[0.32 INFO] (BRun run:436) Initializing S3
[1.19 INFO] (BRun run:441) Initializing history database
[1.82 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools

============================================================
Running ReAdd2-HYBRID: 2014-08-26 22:52:11
    workload=HybridWorkloadGroup
    failover/next_action=FO_READD_REBALANCE
    failover/count=2
    failover/next_delay=45
    testcase=FailoverScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0211
31\68eb149dd819e0fcf974500c9b7554ef

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --workload HybridWorkloadGroup --failover-next_action
 \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45
 \ --testcase FailoverScenario -C share\rexec --rexec_path
 \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675
 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
 \ --cluster_ssh-username=root --cluster_ssh-password=couchbase
        --cluster_useSSL=True
    ---------------------------------------->8


[57.94 INFO] (Workload setupDesign:63) Creating design test_design
[57.95 TRACE] (CouchbaseAdmin getResponseJson:167) PUT http://10.3.121.134:8092/default/_design/test_design
[58.19 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8092/default/_design/test_design/_view/test_view?limit=
1
[58.23 INFO] (Workload setupDesign:80) Design creation done
[58.23 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.24 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=true, Username=default,
Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[58.24 TRACE] (Handle sendMessageAsync:185) > {"Command":"NEWHANDLE","Handle":101,"ReqID":1,"CommandData":{"Port":8091,"Bucket":"d
efault","Options":{"SSL":true,"Username":"default","Password":"","OtherNodes":[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.
3.206",8091]]},"Hostname":"10.3.121.134"}}
[58.25 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[58.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[58.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToke
n=af08829b84f0328e
[58.30 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[58.30 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d42611
5821
[58.58 INFO] (SDKD log:137)
[58.59 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an obje
ct.
[198.47 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[198.48 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.2
[198.48 INFO] (SDKD log:137) at Couchbase.IO.Strategies.SslConnection.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.
net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\SslConnection.cs:line 53
   at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation, IConnection connection) in c:\jenkins\workspace
\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 35
   at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-and
rei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 45
   at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String p
assword) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\Carr
ierPublication\CarrierPublicationProvider.cs:line 41
   at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-a
ndrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159
   at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei
\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 84
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 134
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[198.49 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":101,"ReqID":2,"CommandData":{}}
[198.50 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]
[199.41 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re
Add2-HYBRID/08-26-14/021131/68eb149dd819e0fcf974500c9b7554ef.zip
[199.42 ERROR] (BRun main:513) Caught exception
com.couchbase.sdkdclient.context.HarnessException: java.net.SocketException: Connection reset
        at com.couchbase.sdkdclient.context.HarnessException.create(HarnessException.java:56) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.scenario.PhasedScenario.run(PhasedScenario.java:80) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:169) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:165) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
        at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.implRead(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.read(Unknown Source) ~[na:1.6.0_27]


seems like it's a regression for ssl

 Comments   
Comment by Jeff Morris [ 27/Aug/14 ]
Indeed a regression bug. http://review.couchbase.org/#/c/40982/




[NCBC-606] Couchbase.ServerUnavailableException at Couchbase.Configuration.ConfigContextBase.GetServer() when some nodes failed over Created: 27/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-606.zip     File test.logs    
Issue Links:
Duplicate
is duplicated by NCBC-598 ServerUnavailableException after reba... Open

 Description   
call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
ig.ini -I sdkd.args --variants HYBRID -d all:trace
[0.30 INFO] (BRun run:436) Initializing S3
[1.12 INFO] (BRun run:441) Initializing history database
[1.75 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools

============================================================
Running ReAdd2-HYBRID: 2014-08-26 22:31:28
    workload=HybridWorkloadGroup
    failover/next_action=FO_READD_REBALANCE
    failover/count=2
    failover/next_delay=45
    testcase=FailoverScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0198
88\0dcd5a5bc61e625f06a757ab9dfa7db5

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --workload HybridWorkloadGroup --failover-next_action
 \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45
 \ --testcase FailoverScenario -C share\rexec --rexec_path
 \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675
 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
        --cluster_ssh-username=root --cluster_ssh-password=couchbase

.....


[123.71 INFO] (Scenario run:86) Starting CHANGE phase
[123.71 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[123.72 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[124.10 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[124.13 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[124.14 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[124.40 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds
[125.61 INFO] (SDKD log:137)
[125.63 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavail
ableException' was thrown.
[169.41 INFO] (FailoverAction doNextAction:68) Readding nodes after failover
[169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode
[169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode
[169.42 INFO] (FailoverAction doNextAction:85) Rebalancing nodes after failover
[169.42 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/nodes
[169.43 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/rebalance
[169.45 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[169.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[170.46 TRACE] (CouchbaseAdmin getResponseJson:167) GET
....
[196.67 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[196.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37%
[197.59 INFO] (SDKD log:137) [Sdkd.ViewQuery|Error] ... System.NullReferenceException Object reference not set to an instance of a
n object. at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Views\ViewComman
d.cs:line 268
[197.61 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[197.61 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace\.net-sdkdclient-n
g2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 210
   at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace\.net
-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194
   at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, ISer
ver server) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:
line 179
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c
lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 175
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[197.64 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
        at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
[197.66 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73
[197.66 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":103,"ReqID":73,"CommandData":{}}
[197.66 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]
[197.68 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[197.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39%
[198.80 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re
Add2-HYBRID/08-26-14/019888/0dcd5a5bc61e625f06a757ab9dfa7db5.zip
[198.83 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress


 Comments   
Comment by Jeff Morris [ 27/Aug/14 ]
These are same issue. Still trying to come up with a plan of attack for this one. A likely one will put the client back into bootstrap mode or possibly to force another config update.




[NCBC-605] [ssl] Seems like SsslConnection.SendCallback/ReceiveCallback should handle exceptions "Cannot access a disposed object" as EapConnection does Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Hi Jeff,

unfortunately I do not have any logs to confirm it now, but in a single run I watched this exception in https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/IO/Strategies/SslConnection.cs#L76

(or https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/IO/Strategies/SslConnection.cs#L105)


while EapConnection handle all exceptions:
https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/IO/Strategies/EapConnection.cs#L67

Jeff, do you think my guesses are correct?



 Comments   
Comment by Jeff Morris [ 26/Aug/14 ]
Andrei -

Yeah, it looks like the SslConnection class needs to be updated to what EapConnection is like.

Here is a patch: http://review.couchbase.org/#/c/40944/

-Jeff
Comment by Jeff Morris [ 26/Aug/14 ]
http://review.couchbase.org/#/c/40944/




[NCBC-604] NullReferenceException in OperationBase`1.HandleClientError Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-604.zip     PNG File NullReferenceException.png     File test.logs    

 Description   
c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
[0.32 INFO] (BRun run:441) Initializing history database
[1.31 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools

============================================================
Running ReAdd2-HYBRID: 2014-08-26 13:43:19
    workload=HybridWorkloadGroup
    failover/next_action=FO_READD_REBALANCE
    failover/count=2
    failover/next_delay=45
    testcase=FailoverScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0745
99\cb48df8b14222ae1e9fe55dbb23ec693

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --workload HybridWorkloadGroup --failover-next_action
 \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45
 \ --testcase FailoverScenario -C share\rexec --rexec_path
 \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675
 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
        --cluster_ssh-username=root --cluster_ssh-password=couchbase


....


[123.36 INFO] (Scenario run:86) Starting CHANGE phase
[123.36 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[123.37 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[123.82 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[123.87 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[123.87 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[124.15 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds
[124.18 INFO] (SDKD log:137)
[124.18 INFO] (SDKD log:137) Unhandled Exception:
[124.18 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an obj
ect.
[124.19 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1
[124.19 INFO] (SDKD log:137) at Couchbase.IO.Operations.OperationBase`1.HandleClientError(String message) in c:\jenkins\workspa
ce\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Operations\OperationBase'.cs:line 85
   at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-clien
t\Src\Couchbase\Core\Server.cs:line 93
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 98
   at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\cou
chbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 312
   at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\SetCommand.cs:line 77
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 175
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()



 Comments   
Comment by Jeff Morris [ 26/Aug/14 ]
http://review.couchbase.org/#/c/40938/




[NCBC-603] Nuget is including wrong Enyim.Caching version for CouchbaseLog4Net Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
{"The value of the property 'factory' cannot be parsed. The error is: Could not load file or assembly 'Enyim.Caching, Version=1.3.5001.0, Culture=neutral, PublicKeyToken=05e9c6b5a9ec94c2' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) (c:\\users\\jmorris\\documents\\visual studio 2012\\Projects\\Couchbase.NET.JsonExamples\\StoreJsonPoco\\bin\\Debug\\StoreJsonPoco.vshost.exe.config line 12)"}

 Comments   
Comment by Jeff Morris [ 26/Aug/14 ]
Nuget was using my local packages, so it was only an issue locally for me.




[NCBC-602] [ssl] ArgumentOutOfRangeException in VBucket.LocatePrimary (ReAdd2-HYBRID test) Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File ArgumentOutOfRangeException.png     File test.logs    

 Description   
Hi Jeff,

Now I'm running sdk situational tests with ssl feature.
currently several tests pass, some hang (using the same code). I'm trying to figure out what was wrong. appears on the client side there is some problem with the ssl that will require your help

I finally didn't figure out why tests hang, and so I hope for your help :)


test steps: 4 nodes, falover 2 of them, add back and rebalance


call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
ig.ini -I sdkd.args --variants HYBRID -d all:trace

[229.23 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96%
[230.23 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[230.26 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 98%
[231.26 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[231.26 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[231.26 INFO] (Scenario run:97) CHANGE phase done
[231.26 INFO] (Scenario run:103) Starting REBOUND for 90 seconds
[316.55 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[316.55 INFO] (SDKD log:137) at System.ThrowHelper.ThrowArgumentOutOfRangeException()
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at Couchbase.Core.VBucket.LocatePrimary() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase
\Core\VBucket.cs:line 20
   at Couchbase.Core.Buckets.CouchbaseBucket.GetServer(String key, IVBucket& vBucket) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 86
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 96
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c
lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 293
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 167
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[316.56 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed


please see test's and client's logs in the attached files



 Comments   
Comment by Andrei Baranouski [ 26/Aug/14 ]
duplicate of NCBC-601




[NCBC-601] ThrowArgumentOutOfRangeException in Couchbase.Core.VBucket.LocatePrimary() after failover nodes Created: 24/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/66/console


Running ReAdd2-HYBRID: 2014-08-24 03:25:00

.....
[592.87 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@204.281
[592.87 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@204.282 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[592.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY 
[592.88 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds
[592.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[592.88 INFO] (SDKD log:137)  "DSType": "DSTYPE_SEEDED",
[592.88 INFO] (SDKD log:137)  "DS": {},
[592.88 INFO] (SDKD log:137)  "ViewParameters": {
[592.88 INFO] (SDKD log:137)  "limit": 0
[592.88 INFO] (SDKD log:137)  },
[592.88 INFO] (SDKD log:137)  "DesignName": "test_design",
[592.88 INFO] (SDKD log:137)  "Options": {
[592.88 INFO] (SDKD log:137)  "DelayMax": 10,
[592.88 INFO] (SDKD log:137)  "DelayMin": 1,
[592.89 INFO] (SDKD log:137)  "IterWait": 1,
[592.89 INFO] (SDKD log:137)  "ViewQueryCount": -1,
[592.89 INFO] (SDKD log:137)  "ViewQueryDelay": 0,
[592.89 INFO] (SDKD log:137)  "TimeRes": 1
[592.89 INFO] (SDKD log:137)  },
[592.89 INFO] (SDKD log:137)  "ViewName": "test_view"
[592.89 INFO] (SDKD log:137) } 
[622.88 INFO] (Scenario run:86) Starting CHANGE phase
[622.88 INFO] (CBCluster failoverNodes:582) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[623.18 INFO] (CBCluster failoverNodes:582) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[623.38 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds
[623.50 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 
[623.60 INFO] (SDKD log:137) 
[623.62 INFO] (SDKD log:137) Unhandled Exception: 
[623.62 INFO] (SDKD log:137) Unhandled Exception: 
[623.62 INFO] (SDKD log:137) Unhandled Exception: System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
[623.62 INFO] (SDKD log:137) Parameter name: index
[623.63 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1
[623.63 INFO] (SDKD log:137)  at System.ThrowHelper.ThrowArgumentOutOfRangeException()
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at Couchbase.Core.VBucket.LocatePrimary() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\VBucket.cs:line 20
   at Couchbase.Core.Buckets.CouchbaseBucket.GetServer(String key, IVBucket& vBucket) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 86
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 96
   at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 312
   at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()





 Comments   
Comment by Andrei Baranouski [ 24/Aug/14 ]
https://s3.amazonaws.com/bugdb/jira/NCBC-601/NCBC-601_log.zip
Comment by Jeff Morris [ 24/Aug/14 ]
Tentative patch: http://review.couchbase.org/#/c/40871/1

Discussed this with Mark Nunberg sometime back: What to do when the server returns -1 for a primary in it's VBucket map? I believe we agreed upon trying a replica (really any other available server) and if a NMV is encountered, the config will be refreshed. Let's see how this works.

We might need to handle the case of a null IServer by gracefully returning an error back to the application, instead of letting the null reference end up as a NRE.
Comment by Andrei Baranouski [ 26/Aug/14 ]
When I run with this commit I get

[270.31 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[270.32 INFO] (SDKD log:137) at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
at Couchbase.Core.VBucket.LocatePrimary() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase
\Core\VBucket.cs:line 29
at Couchbase.Core.Buckets.CouchbaseBucket.GetServer(String key, IVBucket& vBucket) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 86
at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 96
at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c
lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\GetCommand.cs:line 18
at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295
at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 175
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
[270.34 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
Comment by Jeff Morris [ 26/Aug/14 ]
Another patch:

git fetch ssh://jmorris@review.couchbase.org:29418/couchbase-net-client refs/changes/71/40871/2 && git checkout FETCH_HEAD




Generated at Fri Aug 29 14:53:40 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.