[NCBC-615] [ssl] data load stuck with 'Socket Exception was unhandled' Created: 29/Aug/14  Updated: 29/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File NCBC-615_1.png     PNG File NCBC-615_2.png     PNG File NCBC-615_3.png     Zip Archive NCBC-615.zip     File test.logs    

 Description   
Hi Jeff,

I don't see the issue without ssl but see it in maby tests with ssl.
I'm not 100% sure that it is the client problem, but I use the same sdkd code and there is no such problem.

in short, I see that not all the data has been downloaded, and tests hang. when I stop the command line to debug, I see the following exception. Could you take a look at the problem? my research have so far failed. I am ready to provide the environment or provide more detailed information if required.

 Comments   
Comment by Jeff Morris [ 29/Aug/14 ]
Can you post the stacktrace? Click on the "View Detail" in the exception dialog and navigate through. If an inner exception exists, make sure it's included.





[NCBC-614] Review and reduce verbosity of logging in INFO mode Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency




[NCBC-613] Refactor operation result classes so that they have a common inheritence chain Created: 28/Aug/14  Updated: 28/Aug/14

Status: In Progress
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently the "operation result" from each of the API's supported by the SDK (Views, N1QL, Memcached, Document) all have different "operation result" classes, which is confusing and inconsistent. This refactoring will make a common inheritance chain for all derived "operation result" classes.




[NCBC-612] Provide configurable timeouts for View and N1QL queries Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
These should default to certain values at the ClientConfiguration level and be overridable at the BucketConfiguration and the query levels themselves.




[NCBC-611] CouchbaseAspNet 1.2.1 Nuget package was released built in debug mode Created: 28/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Stephen Phillips Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
v1.2.1 of "Couchbase ASP.NET SessionState and OutputCache Providers" on nuget the dll included in the package was built in debug mode and was raised to us when we sent a dump file to microsoft for anaylsis. This has been suggested as a potential performance issue. An updated release build would be appreciated.

 Comments   
Comment by Stephen Phillips [ 28/Aug/14 ]
Apologies if this is the wrong project I did not know where else to create the issue.
Comment by Jeff Morris [ 28/Aug/14 ]
Thanks Stephen, I'll look into it. It's fine to create issues for that project here :)




[NCBC-610] [ssl] data load is very slow: 1 ops/sec(regression) Created: 28/Aug/14  Updated: 28/Aug/14  Resolved: 28/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive log_If457e837f.zip     Text File log.txt     PNG File without_ssl.png     PNG File with_ssl_If457e837fdda.png     PNG File with_ssl.png    

 Description   
Hi Jeff,

I see a significant drop in performance with ssl

please see screenshots




 Comments   
Comment by Jeff Morris [ 28/Aug/14 ]
Running against my local cluster (VMs) the performance is nearly identical. From the logs you provided, there is something not right with your config:

2014-08-27 23:35:14,921 [4] DEBUG Couchbase.Core.ClusterManager - Trying to boostrap with Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.
2014-08-27 23:35:14,939 [4] DEBUG Couchbase.Configuration.Server.Providers.ConfigProviderBase - Getting config for bucket default
2014-08-27 23:35:14,950 [4] DEBUG Couchbase.Core.ClusterManager - Creating DefaultIOStrategy
2014-08-27 23:35:14,951 [4] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Creating DefaultIOStrategy 642ecc71-964b-42bb-aff6-4d691605c598
2014-08-27 23:35:14,960 [4] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-27 23:35:14,998 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire new: 2fee4eef-d170-4af2-b92f-4856dfc8e359 | 10.3.121.134:11207 | [0, 0] - 154ff9f2-fc7c-452a-98f7-614407abd6dc - Disposed: False
2014-08-27 23:35:17,517 [4] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-27 23:35:17,529 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire new: 25f51e8c-8149-4415-82a3-16c8bd183139 | 10.3.121.134:11207 | [0, 1] - 154ff9f2-fc7c-452a-98f7-614407abd6dc - Disposed: False
2014-08-27 23:35:20,035 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: 25f51e8c-8149-4415-82a3-16c8bd183139 on 10.3.121.134:11207 - 154ff9f2-fc7c-452a-98f7-614407abd6dc
2014-08-27 23:35:20,038 [4] WARN Couchbase.Core.ClusterManager - Couchbase.Configuration.ConfigException: Could not retrieve configuration for default. Reason: ??
   at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 67
   at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159

After trying CCCP, the client defaults to HTTP streaming and configuration continues. But every op fails:

2014-08-27 23:35:33,050 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: f268e1c1-1cbf-41bb-ba70-3c88ae9f4e14 on 10.3.121.135:11207 - 6b53a72a-c726-4641-b34c-f30f71ba14bd
2014-08-27 23:35:33,052 [4] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation doesn't support retries for key ViewFillerSeedrep0rep0rep0
2014-08-27 23:35:33,052 [4] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation for key ViewFillerSeedrep0rep0rep0 failed after 0 retries. Reason: ?
2014-08-27 23:35:33,065 [4] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#429 on thread 4
2014-08-27 23:35:33,065 [4] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 58 for key ViewFillerSeedrep1rep1rep1 - rev0
2014-08-27 23:35:33,065 [4] DEBUG Couchbase.Core.Server - Sending ViewFillerSeedrep1rep1rep1 using server 10.3.121.134:11207
2014-08-27 23:35:33,066 [4] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-27 23:35:33,079 [4] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire new: 471d6ca5-cd63-48da-8d99-369897e727b2 | 10.3.121.134:11207 | [0, 1] - 9884859d-0bcd-4732-

Try checking your configuration and see if their is some error there.

-Jeff
Comment by Andrei Baranouski [ 28/Aug/14 ]
Hi Jeff,

I I rolled back couchbase-net-client to version

$ git reset --hard HEAD^
HEAD is now at 0b95eb0 NCBC-604: NullReferenceException in OperationBase`1.HandleClientError

Administrator@WIN-335SPEPOCGT /c/jenkins/workspace/.net-sdkdclient-ng2-andrei/couchbase-net-client (master)
$ git log
WARNING: terminal is not fully functional
commit 0b95eb0e888c394c00048b66f15fe7fb0fbf5146
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Tue Aug 26 14:28:57 2014 -0700

    NCBC-604: NullReferenceException in OperationBase`1.HandleClientError

    Change-Id: If457e837fdda073b51b370a588863200910a29be
    Reviewed-on: http://review.couchbase.org/40938
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Brett Lawson <brett19@gmail.com>

and loading looks fine( there are no changes in sdkd frameworks)

see attached screenshot and log with this revision



Comment by Jeff Morris [ 28/Aug/14 ]
Thanks Andrei, I'll look deeper into it this afternoon, as you mentioned it's probably a regression.
Comment by Jeff Morris [ 28/Aug/14 ]
http://review.couchbase.org/#/c/41064/




[NCBC-609] Number base not specified for error codes Created: 27/Aug/14  Updated: 28/Aug/14  Resolved: 28/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: docs
Affects Version/s: 1.3.8
Fix Version/s: 1.3.8, 1.3.9

Type: Bug Priority: Major
Reporter: Amy Kurtzman Assignee: Amy Kurtzman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A customer reported the following:

Hi,
I’m not sure who is in charge of documentation or how to reach said person so I sent this mail to info, I hope this reaches the right person.
 
I wanted to point out a small mistake in the .NET SDK documentation (1.3).
The error codes you have listed on the site are in hexadecimal base but without any mention that these values are in hexadecimal base.
See: http://docs.couchbase.com/couchbase-sdk-net-1.3/#checking-error-codes
For example: OperationTimeout is stated to be 94, it’s actually 148which in hexadecimal is 0x94.
I think this should be changed to base10 or at least clarified.
 
Cheers, Yosef Orbach.

 Comments   
Comment by Amy Kurtzman [ 27/Aug/14 ]
@Jeff, Can you verify that the numbers are indeed in hex? If so, I'll add a note and prefix the numbers with 0x. Thanks.
Comment by Jeff Morris [ 27/Aug/14 ]
Yes, the values are indeed in hex.




[NCBC-608] Implement Common Flags Created: 27/Aug/14  Updated: 27/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Brett Lawson Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Implement the Common Flags formatting as defined by the specification.

 Comments   
Comment by Jeff Morris [ 27/Aug/14 ]
Brett, can you link to the specification?
Comment by Brett Lawson [ 27/Aug/14 ]
https://docs.google.com/document/d/1V653a6FF6DOqdT4d-fKIjGkHabDaNGZsvbtsUKJyeLc




[NCBC-607] [ssl] NullReferenceException at Couchbase.IO.Strategies.SslConnection.Send ( first CRUD operation) Created: 27/Aug/14  Updated: 27/Aug/14  Resolved: 27/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File log.txt     PNG File NCBC-607.png     File test.logs    

 Description   
call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
ig.ini -I sdkd.args --variants HYBRID -d all:trace
[0.32 INFO] (BRun run:436) Initializing S3
[1.19 INFO] (BRun run:441) Initializing history database
[1.82 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools

============================================================
Running ReAdd2-HYBRID: 2014-08-26 22:52:11
    workload=HybridWorkloadGroup
    failover/next_action=FO_READD_REBALANCE
    failover/count=2
    failover/next_delay=45
    testcase=FailoverScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0211
31\68eb149dd819e0fcf974500c9b7554ef

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --workload HybridWorkloadGroup --failover-next_action
 \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45
 \ --testcase FailoverScenario -C share\rexec --rexec_path
 \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675
 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
 \ --cluster_ssh-username=root --cluster_ssh-password=couchbase
        --cluster_useSSL=True
    ---------------------------------------->8


[57.94 INFO] (Workload setupDesign:63) Creating design test_design
[57.95 TRACE] (CouchbaseAdmin getResponseJson:167) PUT http://10.3.121.134:8092/default/_design/test_design
[58.19 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8092/default/_design/test_design/_view/test_view?limit=
1
[58.23 INFO] (Workload setupDesign:80) Design creation done
[58.23 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.24 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=true, Username=default,
Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[58.24 TRACE] (Handle sendMessageAsync:185) > {"Command":"NEWHANDLE","Handle":101,"ReqID":1,"CommandData":{"Port":8091,"Bucket":"d
efault","Options":{"SSL":true,"Username":"default","Password":"","OtherNodes":[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.
3.206",8091]]},"Hostname":"10.3.121.134"}}
[58.25 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[58.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[58.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToke
n=af08829b84f0328e
[58.30 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[58.30 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d42611
5821
[58.58 INFO] (SDKD log:137)
[58.59 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an obje
ct.
[198.47 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[198.48 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.2
[198.48 INFO] (SDKD log:137) at Couchbase.IO.Strategies.SslConnection.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.
net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\SslConnection.cs:line 53
   at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation, IConnection connection) in c:\jenkins\workspace
\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 35
   at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-and
rei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 45
   at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String p
assword) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\Carr
ierPublication\CarrierPublicationProvider.cs:line 41
   at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-a
ndrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159
   at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei
\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 84
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 134
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[198.49 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":101,"ReqID":2,"CommandData":{}}
[198.50 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]
[199.41 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re
Add2-HYBRID/08-26-14/021131/68eb149dd819e0fcf974500c9b7554ef.zip
[199.42 ERROR] (BRun main:513) Caught exception
com.couchbase.sdkdclient.context.HarnessException: java.net.SocketException: Connection reset
        at com.couchbase.sdkdclient.context.HarnessException.create(HarnessException.java:56) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.scenario.PhasedScenario.run(PhasedScenario.java:80) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:169) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:165) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
        at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.implRead(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.read(Unknown Source) ~[na:1.6.0_27]


seems like it's a regression for ssl

 Comments   
Comment by Jeff Morris [ 27/Aug/14 ]
Indeed a regression bug. http://review.couchbase.org/#/c/40982/




[NCBC-606] Couchbase.ServerUnavailableException at Couchbase.Configuration.ConfigContextBase.GetServer() when some nodes failed over Created: 27/Aug/14  Updated: 28/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-606.zip     File test.logs    
Issue Links:
Duplicate
is duplicated by NCBC-598 ServerUnavailableException after reba... Open

 Description   
call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
ig.ini -I sdkd.args --variants HYBRID -d all:trace
[0.30 INFO] (BRun run:436) Initializing S3
[1.12 INFO] (BRun run:441) Initializing history database
[1.75 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools

============================================================
Running ReAdd2-HYBRID: 2014-08-26 22:31:28
    workload=HybridWorkloadGroup
    failover/next_action=FO_READD_REBALANCE
    failover/count=2
    failover/next_delay=45
    testcase=FailoverScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0198
88\0dcd5a5bc61e625f06a757ab9dfa7db5

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --workload HybridWorkloadGroup --failover-next_action
 \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45
 \ --testcase FailoverScenario -C share\rexec --rexec_path
 \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675
 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
        --cluster_ssh-username=root --cluster_ssh-password=couchbase

.....


[123.71 INFO] (Scenario run:86) Starting CHANGE phase
[123.71 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[123.72 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[124.10 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[124.13 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[124.14 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[124.40 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds
[125.61 INFO] (SDKD log:137)
[125.63 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavail
ableException' was thrown.
[169.41 INFO] (FailoverAction doNextAction:68) Readding nodes after failover
[169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode
[169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode
[169.42 INFO] (FailoverAction doNextAction:85) Rebalancing nodes after failover
[169.42 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/nodes
[169.43 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/rebalance
[169.45 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[169.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[170.46 TRACE] (CouchbaseAdmin getResponseJson:167) GET
....
[196.67 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[196.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37%
[197.59 INFO] (SDKD log:137) [Sdkd.ViewQuery|Error] ... System.NullReferenceException Object reference not set to an instance of a
n object. at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Views\ViewComman
d.cs:line 268
[197.61 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[197.61 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace\.net-sdkdclient-n
g2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 210
   at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace\.net
-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194
   at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, ISer
ver server) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:
line 179
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c
lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 175
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[197.64 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
        at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
[197.66 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73
[197.66 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":103,"ReqID":73,"CommandData":{}}
[197.66 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]
[197.68 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[197.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39%
[198.80 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re
Add2-HYBRID/08-26-14/019888/0dcd5a5bc61e625f06a757ab9dfa7db5.zip
[198.83 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress


 Comments   
Comment by Jeff Morris [ 27/Aug/14 ]
These are same issue. Still trying to come up with a plan of attack for this one. A likely one will put the client back into bootstrap mode or possibly to force another config update.




[NCBC-605] [ssl] Seems like SsslConnection.SendCallback/ReceiveCallback should handle exceptions "Cannot access a disposed object" as EapConnection does Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Hi Jeff,

unfortunately I do not have any logs to confirm it now, but in a single run I watched this exception in https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/IO/Strategies/SslConnection.cs#L76

(or https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/IO/Strategies/SslConnection.cs#L105)


while EapConnection handle all exceptions:
https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/IO/Strategies/EapConnection.cs#L67

Jeff, do you think my guesses are correct?



 Comments   
Comment by Jeff Morris [ 26/Aug/14 ]
Andrei -

Yeah, it looks like the SslConnection class needs to be updated to what EapConnection is like.

Here is a patch: http://review.couchbase.org/#/c/40944/

-Jeff
Comment by Jeff Morris [ 26/Aug/14 ]
http://review.couchbase.org/#/c/40944/




[NCBC-604] NullReferenceException in OperationBase`1.HandleClientError Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-604.zip     PNG File NullReferenceException.png     File test.logs    

 Description   
c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
[0.32 INFO] (BRun run:441) Initializing history database
[1.31 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools

============================================================
Running ReAdd2-HYBRID: 2014-08-26 13:43:19
    workload=HybridWorkloadGroup
    failover/next_action=FO_READD_REBALANCE
    failover/count=2
    failover/next_delay=45
    testcase=FailoverScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0745
99\cb48df8b14222ae1e9fe55dbb23ec693

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --workload HybridWorkloadGroup --failover-next_action
 \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45
 \ --testcase FailoverScenario -C share\rexec --rexec_path
 \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675
 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
        --cluster_ssh-username=root --cluster_ssh-password=couchbase


....


[123.36 INFO] (Scenario run:86) Starting CHANGE phase
[123.36 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[123.37 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[123.82 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[123.87 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[123.87 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver
[124.15 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds
[124.18 INFO] (SDKD log:137)
[124.18 INFO] (SDKD log:137) Unhandled Exception:
[124.18 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an obj
ect.
[124.19 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1
[124.19 INFO] (SDKD log:137) at Couchbase.IO.Operations.OperationBase`1.HandleClientError(String message) in c:\jenkins\workspa
ce\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Operations\OperationBase'.cs:line 85
   at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-clien
t\Src\Couchbase\Core\Server.cs:line 93
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 98
   at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\cou
chbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 312
   at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\SetCommand.cs:line 77
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 175
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()



 Comments   
Comment by Jeff Morris [ 26/Aug/14 ]
http://review.couchbase.org/#/c/40938/




[NCBC-603] Nuget is including wrong Enyim.Caching version for CouchbaseLog4Net Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
{"The value of the property 'factory' cannot be parsed. The error is: Could not load file or assembly 'Enyim.Caching, Version=1.3.5001.0, Culture=neutral, PublicKeyToken=05e9c6b5a9ec94c2' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) (c:\\users\\jmorris\\documents\\visual studio 2012\\Projects\\Couchbase.NET.JsonExamples\\StoreJsonPoco\\bin\\Debug\\StoreJsonPoco.vshost.exe.config line 12)"}

 Comments   
Comment by Jeff Morris [ 26/Aug/14 ]
Nuget was using my local packages, so it was only an issue locally for me.




[NCBC-602] [ssl] ArgumentOutOfRangeException in VBucket.LocatePrimary (ReAdd2-HYBRID test) Created: 26/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File ArgumentOutOfRangeException.png     File test.logs    

 Description   
Hi Jeff,

Now I'm running sdk situational tests with ssl feature.
currently several tests pass, some hang (using the same code). I'm trying to figure out what was wrong. appears on the client side there is some problem with the ssl that will require your help

I finally didn't figure out why tests hang, and so I hope for your help :)


test steps: 4 nodes, falover 2 of them, add back and rebalance


call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf
ig.ini -I sdkd.args --variants HYBRID -d all:trace

[229.23 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96%
[230.23 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[230.26 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 98%
[231.26 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress
[231.26 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[231.26 INFO] (Scenario run:97) CHANGE phase done
[231.26 INFO] (Scenario run:103) Starting REBOUND for 90 seconds
[316.55 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[316.55 INFO] (SDKD log:137) at System.ThrowHelper.ThrowArgumentOutOfRangeException()
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at Couchbase.Core.VBucket.LocatePrimary() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase
\Core\VBucket.cs:line 20
   at Couchbase.Core.Buckets.CouchbaseBucket.GetServer(String key, IVBucket& vBucket) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 86
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 96
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c
lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 293
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 167
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[316.56 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed


please see test's and client's logs in the attached files



 Comments   
Comment by Andrei Baranouski [ 26/Aug/14 ]
duplicate of NCBC-601




[NCBC-601] ThrowArgumentOutOfRangeException in Couchbase.Core.VBucket.LocatePrimary() after failover nodes Created: 24/Aug/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/66/console


Running ReAdd2-HYBRID: 2014-08-24 03:25:00

.....
[592.87 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@204.281
[592.87 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@204.282 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[592.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY 
[592.88 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds
[592.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[592.88 INFO] (SDKD log:137)  "DSType": "DSTYPE_SEEDED",
[592.88 INFO] (SDKD log:137)  "DS": {},
[592.88 INFO] (SDKD log:137)  "ViewParameters": {
[592.88 INFO] (SDKD log:137)  "limit": 0
[592.88 INFO] (SDKD log:137)  },
[592.88 INFO] (SDKD log:137)  "DesignName": "test_design",
[592.88 INFO] (SDKD log:137)  "Options": {
[592.88 INFO] (SDKD log:137)  "DelayMax": 10,
[592.88 INFO] (SDKD log:137)  "DelayMin": 1,
[592.89 INFO] (SDKD log:137)  "IterWait": 1,
[592.89 INFO] (SDKD log:137)  "ViewQueryCount": -1,
[592.89 INFO] (SDKD log:137)  "ViewQueryDelay": 0,
[592.89 INFO] (SDKD log:137)  "TimeRes": 1
[592.89 INFO] (SDKD log:137)  },
[592.89 INFO] (SDKD log:137)  "ViewName": "test_view"
[592.89 INFO] (SDKD log:137) } 
[622.88 INFO] (Scenario run:86) Starting CHANGE phase
[622.88 INFO] (CBCluster failoverNodes:582) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[623.18 INFO] (CBCluster failoverNodes:582) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[623.38 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds
[623.50 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 
[623.60 INFO] (SDKD log:137) 
[623.62 INFO] (SDKD log:137) Unhandled Exception: 
[623.62 INFO] (SDKD log:137) Unhandled Exception: 
[623.62 INFO] (SDKD log:137) Unhandled Exception: System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
[623.62 INFO] (SDKD log:137) Parameter name: index
[623.63 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1
[623.63 INFO] (SDKD log:137)  at System.ThrowHelper.ThrowArgumentOutOfRangeException()
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at Couchbase.Core.VBucket.LocatePrimary() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\VBucket.cs:line 20
   at Couchbase.Core.Buckets.CouchbaseBucket.GetServer(String key, IVBucket& vBucket) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 86
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 96
   at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 312
   at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()





 Comments   
Comment by Andrei Baranouski [ 24/Aug/14 ]
https://s3.amazonaws.com/bugdb/jira/NCBC-601/NCBC-601_log.zip
Comment by Jeff Morris [ 24/Aug/14 ]
Tentative patch: http://review.couchbase.org/#/c/40871/1

Discussed this with Mark Nunberg sometime back: What to do when the server returns -1 for a primary in it's VBucket map? I believe we agreed upon trying a replica (really any other available server) and if a NMV is encountered, the config will be refreshed. Let's see how this works.

We might need to handle the case of a null IServer by gracefully returning an error back to the application, instead of letting the null reference end up as a NRE.
Comment by Andrei Baranouski [ 26/Aug/14 ]
When I run with this commit I get

[270.31 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[270.32 INFO] (SDKD log:137) at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
at Couchbase.Core.VBucket.LocatePrimary() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase
\Core\VBucket.cs:line 29
at Couchbase.Core.Buckets.CouchbaseBucket.GetServer(String key, IVBucket& vBucket) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 86
at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-
andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 96
at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c
lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net
\src\Sdkd\Commands\GetCommand.cs:line 18
at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin
e 25
at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr
ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295
at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs
:line 175
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo
lean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre
serveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
[270.34 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
Comment by Jeff Morris [ 26/Aug/14 ]
Another patch:

git fetch ssh://jmorris@review.couchbase.org:29418/couchbase-net-client refs/changes/71/40871/2 && git checkout FETCH_HEAD




[NCBC-600] Update Client Client Configuration When Nodes List Changes Created: 22/Aug/14  Updated: 25/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to NCBC-597 [vpn run] Couchbase.Configuration.Ser... Resolved

 Comments   
Comment by Jeff Morris [ 22/Aug/14 ]
http://review.couchbase.org/#/c/40850/




[NCBC-599] Shuffle method changes server order resulting in NMV responses Created: 21/Aug/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Accidentally found this prepping for the CB3 webinar: if you call Bucket.IsSecure, every operation that comes after (nearly every) results in a NMV. The reason being that IsSecure grabs a random server to check and calls a Shuffle() method, which re-orders the Servers list impacting VBucket lookups.

 Comments   
Comment by Jeff Morris [ 21/Aug/14 ]
http://review.couchbase.org/#/c/40818/




[NCBC-598] ServerUnavailableException after rebalance (in CouchbaseBucket.Get) Created: 21/Aug/14  Updated: 27/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive log.txt.zip    
Issue Links:
Duplicate
duplicates NCBC-606 Couchbase.ServerUnavailableException ... Open

 Description   
git log
commit 86219ee99ccafdd00c98d577c9bc2b97efef35bb
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 20 17:44:45 2014 -0700

    NCBC-592: AuthenticationException in CouchbaseBucket.SendWithRetry


Running Rb2Out-KV: 2014-08-20 23:33:57
.....


[25.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.33
[25.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@121.43 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=10, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[25.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET 
[25.06 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds
[25.06 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[25.06 INFO] (SDKD log:137)  "DSType": "DSTYPE_SEEDED",
[25.06 INFO] (SDKD log:137)  "DS": {
[25.07 INFO] (SDKD log:137)  "KSeed": "SimpleKey",
[25.07 INFO] (SDKD log:137)  "VSeed": "SimpleValue",
[25.07 INFO] (SDKD log:137)  "Count": 10,
[25.07 INFO] (SDKD log:137)  "Continuous": true,
[25.07 INFO] (SDKD log:137)  "VSize": 128,
[25.07 INFO] (SDKD log:137)  "Repeat": "REP",
[25.07 INFO] (SDKD log:137)  "KSize": 32
[25.07 INFO] (SDKD log:137)  },
[25.07 INFO] (SDKD log:137)  "Options": {
[25.07 INFO] (SDKD log:137)  "DelayMax": 10,
[25.07 INFO] (SDKD log:137)  "DelayMin": 1,
[25.07 INFO] (SDKD log:137)  "IterWait": 1,
[25.07 INFO] (SDKD log:137)  "TimeRes": 1
[25.09 INFO] (SDKD log:137)  }
[25.09 INFO] (SDKD log:137) } 
[55.06 INFO] (Scenario run:86) Starting CHANGE phase
[55.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[56.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[57.12 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2%
[58.13 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4%
[59.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7%
.................................
[99.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 94%
[100.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96%
[101.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 98%
[102.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[103.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[104.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[105.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[106.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[107.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[108.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[109.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[110.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[111.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[112.75 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 
[112.78 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[112.79 INFO] (Scenario run:97) CHANGE phase done
[112.79 INFO] (Scenario run:103) Starting REBOUND for 90 seconds
[113.23 INFO] (SDKD log:137) 
[113.26 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavailableException' was thrown.
[125.87 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[125.88 INFO] (SDKD log:137)  at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 212
   at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194
   at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 179
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[125.89 ERROR] (BRun maybePropagate:243) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
[125.89 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.44
[125.90 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na]
at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]


 Comments   
Comment by Jeff Morris [ 22/Aug/14 ]
I think this needs to be caught and returned back to the application as a failed operation. The application can decide to retry the operation or continue on based upon the IOperationResult<T>.Status and Message properties.




[NCBC-597] [vpn run] Couchbase.Configuration.Server.Serialization.BootstrapException: Could not bootstrap from configured servers list Created: 21/Aug/14  Updated: 25/Aug/14  Resolved: 25/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-597.log.zip    
Issue Links:
Relates to
relates to NCBC-600 Update Client Client Configuration Wh... Resolved

 Description   
git log
commit 86219ee99ccafdd00c98d577c9bc2b97efef35bb
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 20 17:44:45 2014 -0700

    NCBC-592: AuthenticationException in CouchbaseBucket.SendWithRetry

c:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,17 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/count=2
    workload=HybridWorkloadGroup
    rebalance/mode=out
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1173\Rb2Out-HYBRID\08-21-14\018604\baf8737ee95c04b1942a18c39e286fd1

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-count 2 --workload HybridWorkloadGroup
 \ --rebalance-mode out --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,50 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,77 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,78 INFO] (RunContext run:124) Starting cluster and driver
[1,79 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,79 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1,79 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[2,13 INFO] (SDKD log:137) --- Logging Self-Test ---
[2,14 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[2,14 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[2,14 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[2,14 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[2,18 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,32 INFO] (SDKD log:137) Call SweepOnce()
[2,32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,32 INFO] (SDKD log:137) Call SweepOnce()
[2,32 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,32 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[2,43 INFO] (SDKD log:137) Call SweepOnce()
[2,43 INFO] (SDKD log:137) Call SweepOnce()
[2,44 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[13,36 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.145:8091,ns_1@10.3.4.145>
[13,83 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.146:8091,ns_1@10.3.4.146>
[14,26 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[14,72 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[14,72 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[14,72 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[14,73 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[16,79 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[16,79 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[16,81 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[16,83 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[16,83 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[16,83 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[16,97 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[16,98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[17,54 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@241a6856
[17,54 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@412effef
[17,55 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@1deaf20
[17,99 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@7a2f1c2a
[17,99 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@7c737a1a
[35,10 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[41,18 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[44,39 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[56,06 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[56,91 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[56,91 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[57,12 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[57,34 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[66,15 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[68,68 INFO] (RunContext run:143) Driver and cluster initialized
[68,92 INFO] (RunContext call:167) Running scenario..
[68,93 INFO] (Scenario run:72) Starting RAMP phase
[68,93 INFO] (Workload setupDesign:63) Creating design test_design
[70,20 INFO] (Workload setupDesign:80) Design creation done
[70,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[70,20 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[70,20 INFO] (SDKD log:137) Call SweepOnce()
[70,20 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[70,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[70,29 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[70,29 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[70,31 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[70,31 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[78,64 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[78,64 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {Options={DelayMin=0, TimeRes=0, IterWait=1, DelayMax=0}, Schema={InflateLevel=40, InflateContent=meh}, DSType=DSTYPE_SEEDED, DS={KSize=12, KSeed=ViewFillerSeed, VSize=512, VSeed=Vie
wFillerSeed, Continuous=false, Repeat=rep, Count=1000}}
[78,65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[78,66 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[78,66 INFO] (SDKD log:137) "Options": {
[78,66 INFO] (SDKD log:137) "DelayMin": 0,
[78,66 INFO] (SDKD log:137) "TimeRes": 0,
[78,67 INFO] (SDKD log:137) "IterWait": 1,
[78,67 INFO] (SDKD log:137) "DelayMax": 0
[78,67 INFO] (SDKD log:137) },
[78,67 INFO] (SDKD log:137) "Schema": {
[78,67 INFO] (SDKD log:137) "InflateLevel": 40,
[78,68 INFO] (SDKD log:137) "InflateContent": "meh"
[78,68 INFO] (SDKD log:137) },
[78,68 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[78,69 INFO] (SDKD log:137) "DS": {
[78,69 INFO] (SDKD log:137) "KSize": 12,
[78,69 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[78,69 INFO] (SDKD log:137) "VSize": 512,
[78,70 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[78,70 INFO] (SDKD log:137) "Continuous": false,
[78,70 INFO] (SDKD log:137) "Repeat": "rep",
[78,71 INFO] (SDKD log:137) "Count": 1000
[78,71 INFO] (SDKD log:137) }
[78,71 INFO] (SDKD log:137) }
[290,18 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[290,18 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[290,19 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[290,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[290,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[290,19 INFO] (Workload call:112) Preloading items into cluster before workload
[290,19 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 2092826221
[290,20 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[290,20 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()2092826221
[290,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[290,20 INFO] (SDKD log:137) Call SweepOnce()
[290,21 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[290,21 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[290,22 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[296,87 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[296,88 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@102.5 => {Options={DelayMin=0, TimeRes=0, IterWait=100, DelayMax=0}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=false, Repeat=REP, Coun
t=15000}}
[296,88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[296,88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[296,88 INFO] (SDKD log:137) "Options": {
[296,89 INFO] (SDKD log:137) "DelayMin": 0,
[296,89 INFO] (SDKD log:137) "TimeRes": 0,
[296,89 INFO] (SDKD log:137) "IterWait": 100,
[296,89 INFO] (SDKD log:137) "DelayMax": 0
[296,90 INFO] (SDKD log:137) },
[296,90 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[296,90 INFO] (SDKD log:137) "DS": {
[296,90 INFO] (SDKD log:137) "KSize": 32,
[296,90 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[296,91 INFO] (SDKD log:137) "VSize": 128,
[296,91 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[296,91 INFO] (SDKD log:137) "Continuous": false,
[296,91 INFO] (SDKD log:137) "Repeat": "REP",
[296,92 INFO] (SDKD log:137) "Count": 15000
[296,92 INFO] (SDKD log:137) }
[296,92 INFO] (SDKD log:137) }
[347,55 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[1868,44 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@102.5 => {[OK]: 14941, [MEMD:GENERIC]: 59}
[1868,44 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[1868,44 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[1868,44 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[1868,44 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1781340248
[1868,45 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.11 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,45 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.10 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,45 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,47 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.9 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.8 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.15 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.16 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.14 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.13 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1781340248
[1868,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.12 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1868,48 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[1868,49 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,49 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,49 INFO] (SDKD log:137) Call SweepOnce()
[1868,49 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,50 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,50 INFO] (SDKD log:137) Call SweepOnce()
[1868,50 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,50 INFO] (SDKD log:137) Call SweepOnce()
[1868,51 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,51 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,51 INFO] (SDKD log:137) Call SweepOnce()
[1868,51 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,52 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,52 INFO] (SDKD log:137) Call SweepOnce()
[1868,52 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,53 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,53 INFO] (SDKD log:137) Call SweepOnce()
[1868,54 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,54 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,54 INFO] (SDKD log:137) Call SweepOnce()
[1868,55 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,55 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,55 INFO] (SDKD log:137) Call SweepOnce()
[1868,55 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,55 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,56 INFO] (SDKD log:137) Call SweepOnce()
[1868,56 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,56 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1868,56 INFO] (SDKD log:137) Call SweepOnce()
[1868,57 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1868,57 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[1868,57 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[1868,57 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[1868,58 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[1868,58 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[1868,58 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112
[1868,58 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109
[1868,58 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111
[1868,59 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[1868,59 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110
[1874,52 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.11
[1874,62 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@107.17 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1874,62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1874,62 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.18 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1874,63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1874,63 INFO] (SDKD log:137) "Options": {
[1874,63 INFO] (SDKD log:137) "DelayMin": 1,
[1874,63 INFO] (SDKD log:137) "TimeRes": 1,
[1874,64 INFO] (SDKD log:137) "IterWait": 1,
[1874,64 INFO] (SDKD log:137) "DelayMax": 10
[1874,64 INFO] (SDKD log:137) },
[1874,64 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1874,65 INFO] (SDKD log:137) "DS": {
[1874,65 INFO] (SDKD log:137) "KSize": 32,
[1874,65 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1874,66 INFO] (SDKD log:137) "VSize": 128,
[1874,66 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1874,66 INFO] (SDKD log:137) "Continuous": true,
[1874,67 INFO] (SDKD log:137) "Repeat": "REP",
[1874,67 INFO] (SDKD log:137) "Count": 15000
[1874,67 INFO] (SDKD log:137) }
[1874,67 INFO] (SDKD log:137) }
[1874,67 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1874,68 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1874,68 INFO] (SDKD log:137) Call SweepOnce()
[1874,68 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113
[1880,35 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.10
[1880,35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@106.19 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1880,36 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1880,36 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1880,36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.20 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1880,37 INFO] (SDKD log:137) "Options": {
[1880,37 INFO] (SDKD log:137) "DelayMin": 1,
[1880,38 INFO] (SDKD log:137) "TimeRes": 1,
[1880,38 INFO] (SDKD log:137) "IterWait": 1,
[1880,38 INFO] (SDKD log:137) "DelayMax": 10
[1880,38 INFO] (SDKD log:137) },
[1880,39 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1880,39 INFO] (SDKD log:137) "DS": {
[1880,39 INFO] (SDKD log:137) "KSize": 32,
[1880,39 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1880,40 INFO] (SDKD log:137) "VSize": 128,
[1880,40 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1880,40 INFO] (SDKD log:137) "Continuous": true,
[1880,41 INFO] (SDKD log:137) "Repeat": "REP",
[1880,41 INFO] (SDKD log:137) "Count": 15000
[1880,41 INFO] (SDKD log:137) }
[1880,41 INFO] (SDKD log:137) }
[1880,42 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1880,42 INFO] (SDKD log:137) Call SweepOnce()
[1880,42 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1880,42 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114
[1886,61 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[1886,61 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@103.21 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1886,61 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1886,62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1886,62 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.22 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1886,62 INFO] (SDKD log:137) "Options": {
[1886,63 INFO] (SDKD log:137) "DelayMin": 1,
[1886,63 INFO] (SDKD log:137) "TimeRes": 1,
[1886,63 INFO] (SDKD log:137) "IterWait": 1,
[1886,64 INFO] (SDKD log:137) "DelayMax": 10
[1886,64 INFO] (SDKD log:137) },
[1886,64 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1886,65 INFO] (SDKD log:137) "DS": {
[1886,65 INFO] (SDKD log:137) "KSize": 32,
[1886,65 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1886,66 INFO] (SDKD log:137) "VSize": 128,
[1886,66 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1886,66 INFO] (SDKD log:137) "Continuous": true,
[1886,67 INFO] (SDKD log:137) "Repeat": "REP",
[1886,67 INFO] (SDKD log:137) "Count": 15000
[1886,68 INFO] (SDKD log:137) }
[1886,68 INFO] (SDKD log:137) }
[1886,68 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1886,68 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1886,68 INFO] (SDKD log:137) Call SweepOnce()
[1886,69 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115
[1892,19 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.8
[1892,19 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@104.23 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1892,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1892,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1892,20 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.24 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1892,20 INFO] (SDKD log:137) "Options": {
[1892,20 INFO] (SDKD log:137) "DelayMin": 1,
[1892,21 INFO] (SDKD log:137) "TimeRes": 1,
[1892,21 INFO] (SDKD log:137) "IterWait": 1,
[1892,21 INFO] (SDKD log:137) "DelayMax": 10
[1892,21 INFO] (SDKD log:137) },
[1892,22 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1892,22 INFO] (SDKD log:137) "DS": {
[1892,22 INFO] (SDKD log:137) "KSize": 32,
[1892,22 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1892,23 INFO] (SDKD log:137) "VSize": 128,
[1892,23 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1892,23 INFO] (SDKD log:137) "Continuous": true,
[1892,24 INFO] (SDKD log:137) "Repeat": "REP",
[1892,24 INFO] (SDKD log:137) "Count": 15000
[1892,24 INFO] (SDKD log:137) }
[1892,24 INFO] (SDKD log:137) }
[1892,25 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1892,25 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1892,25 INFO] (SDKD log:137) Call SweepOnce()
[1892,25 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116
[1897,18 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.9
[1897,18 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@105.25 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1897,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1897,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1897,19 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.26 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1897,19 INFO] (SDKD log:137) "Options": {
[1897,19 INFO] (SDKD log:137) "DelayMin": 1,
[1897,20 INFO] (SDKD log:137) "TimeRes": 1,
[1897,20 INFO] (SDKD log:137) "IterWait": 1,
[1897,20 INFO] (SDKD log:137) "DelayMax": 10
[1897,21 INFO] (SDKD log:137) },
[1897,21 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1897,21 INFO] (SDKD log:137) "DS": {
[1897,21 INFO] (SDKD log:137) "KSize": 32,
[1897,22 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1897,22 INFO] (SDKD log:137) "VSize": 128,
[1897,22 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1897,22 INFO] (SDKD log:137) "Continuous": true,
[1897,23 INFO] (SDKD log:137) "Repeat": "REP",
[1897,23 INFO] (SDKD log:137) "Count": 15000
[1897,23 INFO] (SDKD log:137) }
[1897,24 INFO] (SDKD log:137) }
[1897,24 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1897,24 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1897,25 INFO] (SDKD log:137) Call SweepOnce()
[1897,25 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117
[1903,07 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.16
[1903,07 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@112.27 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1903,08 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1903,08 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1903,08 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.28 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1903,08 INFO] (SDKD log:137) "Options": {
[1903,09 INFO] (SDKD log:137) "DelayMin": 1,
[1903,09 INFO] (SDKD log:137) "TimeRes": 1,
[1903,09 INFO] (SDKD log:137) "IterWait": 1,
[1903,09 INFO] (SDKD log:137) "DelayMax": 10
[1903,10 INFO] (SDKD log:137) },
[1903,10 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1903,10 INFO] (SDKD log:137) "DS": {
[1903,10 INFO] (SDKD log:137) "KSize": 32,
[1903,11 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1903,11 INFO] (SDKD log:137) "VSize": 128,
[1903,11 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1903,12 INFO] (SDKD log:137) "Continuous": true,
[1903,12 INFO] (SDKD log:137) "Repeat": "REP",
[1903,12 INFO] (SDKD log:137) "Count": 15000
[1903,12 INFO] (SDKD log:137) }
[1903,13 INFO] (SDKD log:137) }
[1903,13 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1903,13 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1903,13 INFO] (SDKD log:137) Call SweepOnce()
[1903,14 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118
[1909,85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.13
[1909,85 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@109.29 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1909,86 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1909,86 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.30 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1909,86 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1909,86 INFO] (SDKD log:137) "Options": {
[1909,87 INFO] (SDKD log:137) "DelayMin": 1,
[1909,87 INFO] (SDKD log:137) "TimeRes": 1,
[1909,87 INFO] (SDKD log:137) "IterWait": 1,
[1909,87 INFO] (SDKD log:137) "DelayMax": 10
[1909,88 INFO] (SDKD log:137) },
[1909,88 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1909,88 INFO] (SDKD log:137) "DS": {
[1909,89 INFO] (SDKD log:137) "KSize": 32,
[1909,89 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1909,89 INFO] (SDKD log:137) "VSize": 128,
[1909,89 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1909,90 INFO] (SDKD log:137) "Continuous": true,
[1909,90 INFO] (SDKD log:137) "Repeat": "REP",
[1909,91 INFO] (SDKD log:137) "Count": 15000
[1909,91 INFO] (SDKD log:137) }
[1909,91 INFO] (SDKD log:137) }
[1909,91 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1909,92 INFO] (SDKD log:137) Call SweepOnce()
[1909,92 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1909,92 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119
[1925,93 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.15
[1925,93 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@111.31 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1925,93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1925,93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.32 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1925,93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1925,94 INFO] (SDKD log:137) "Options": {
[1925,94 INFO] (SDKD log:137) "DelayMin": 1,
[1925,94 INFO] (SDKD log:137) "TimeRes": 1,
[1925,94 INFO] (SDKD log:137) "IterWait": 1,
[1925,95 INFO] (SDKD log:137) "DelayMax": 10
[1925,95 INFO] (SDKD log:137) },
[1925,95 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1925,95 INFO] (SDKD log:137) "DS": {
[1925,96 INFO] (SDKD log:137) "KSize": 32,
[1925,96 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1925,96 INFO] (SDKD log:137) "VSize": 128,
[1925,96 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1925,97 INFO] (SDKD log:137) "Continuous": true,
[1925,97 INFO] (SDKD log:137) "Repeat": "REP",
[1925,97 INFO] (SDKD log:137) "Count": 15000
[1925,98 INFO] (SDKD log:137) }
[1925,98 INFO] (SDKD log:137) }
[1925,98 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1925,98 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1925,99 INFO] (SDKD log:137) Call SweepOnce()
[1925,99 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120
[2031,57 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.14
[2031,57 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@110.33 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[2031,57 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[2031,57 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[2031,57 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.34 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[2031,58 INFO] (SDKD log:137) "Options": {
[2031,58 INFO] (SDKD log:137) "DelayMin": 1,
[2031,58 INFO] (SDKD log:137) "TimeRes": 1,
[2031,58 INFO] (SDKD log:137) "IterWait": 1,
[2031,58 INFO] (SDKD log:137) "DelayMax": 10
[2031,58 INFO] (SDKD log:137) },
[2031,58 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[2031,58 INFO] (SDKD log:137) "DS": {
[2031,58 INFO] (SDKD log:137) "KSize": 32,
[2031,58 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[2031,59 INFO] (SDKD log:137) "VSize": 128,
[2031,59 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[2031,59 INFO] (SDKD log:137) "Continuous": true,
[2031,59 INFO] (SDKD log:137) "Repeat": "REP",
[2031,59 INFO] (SDKD log:137) "Count": 15000
[2031,59 INFO] (SDKD log:137) }
[2031,59 INFO] (SDKD log:137) }
[2031,59 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2031,59 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[2031,59 INFO] (SDKD log:137) Call SweepOnce()
[2031,59 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121
[2133,15 INFO] (SDKD log:137)
[2133,15 INFO] (SDKD log:137) Unhandled Exception : Couchbase.Configuration.Server.Serialization.BootstrapException: Could not bootstrap from configured servers list.
[2133,15 INFO] (SDKD log:137) at Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig.Initialize() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\Streaming\HttpServerConfig.cs:line 52

[2133,16 INFO] (SDKD log:137) at Couchbase.Configuration.Server.Providers.Streaming.HttpStreamingProvider.StartProvider(String username, String password) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\
Streaming\HttpStreamingProvider.cs:line 238
[2133,16 INFO] (SDKD log:137) at Couchbase.Configuration.Server.Providers.Streaming.HttpStreamingProvider.GetConfig(String bucketName, String password) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\St
reaming\HttpStreamingProvider.cs:line 53
[2133,16 INFO] (SDKD log:137) at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159
[2133,16 INFO] (SDKD log:137) at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 84
[2133,16 INFO] (SDKD log:137) at SdkdConsole.Daemon.Handle.Run() at c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 134
[2133,16 INFO] (SDKD log:137) at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
[2133,17 INFO] (SDKD log:137) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[2133,17 INFO] (SDKD log:137) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[2133,17 INFO] (SDKD log:137) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
[2133,17 INFO] (SDKD log:137) at System.Threading.ThreadHelper.ThreadStart()
[2152,56 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_11]
        at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_11]
[2152,56 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.35
[2152,56 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]

reproduce it in each run through vpn, don't see in jenkins http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/64/console

 Comments   
Comment by Jeff Morris [ 22/Aug/14 ]
What is going on here is that the while trying to bootstrap, the client has exhausted the servers available within it's client configuration. Without looking at the sdkd-net code, I am guessing it's creating a configuration with a single bootstrap server and then eventually that bootstrap server is removed from the cluster. As evidence:

Line 19: 2014-08-20 11:19:48,001 [4] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 7120: 2014-08-20 11:21:34,247 [5] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 112230: 2014-08-20 11:46:07,977 [12] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 112298: 2014-08-20 11:46:13,909 [21] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 112523: 2014-08-20 11:46:19,922 [26] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 112936: 2014-08-20 11:46:25,401 [15] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 113567: 2014-08-20 11:46:31,724 [14] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 114433: 2014-08-20 11:46:38,069 [23] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 115409: 2014-08-20 11:46:44,009 [22] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 116591: 2014-08-20 11:46:49,994 [20] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 116654: 2014-08-21 08:11:15,680 [4] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 123744: 2014-08-21 08:14:54,350 [11] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 228894: 2014-08-21 08:41:12,600 [13] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 228959: 2014-08-21 08:41:18,672 [7] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 229187: 2014-08-21 08:41:24,526 [9] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 229665: 2014-08-21 08:41:30,757 [6] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 230256: 2014-08-21 08:41:36,329 [5] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 230950: 2014-08-21 08:41:41,370 [23] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 231963: 2014-08-21 08:41:47,217 [22] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 233352: 2014-08-21 08:41:53,990 [20] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 237408: 2014-08-21 08:42:10,063 [21] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools
Line 267746: 2014-08-21 08:43:55,722 [15] INFO Couchbase.Configuration.Server.Providers.Streaming.HttpServerConfig - Bootstrapping from http://10.3.4.144:8091/pools

The bootstrap list is shuffled, but the it always tries to bootstrap off the same server.
Comment by Jeff Morris [ 22/Aug/14 ]
Another possible issue is that the client is using the HTTP provider, which means it tried the CCCP provider and failed. The CCCP provider is preferred when the server is > 2.5.
Comment by Jeff Morris [ 25/Aug/14 ]
Depending upon how the client is used, this should resolve this: http://review.couchbase.org/#/c/40850/

However, you should add more nodes to your bootstrap list. This is the list the client will shuffle through until it either finds a node to bootstrap against or cannot bootstrap (because it can't find a candidate node in _your_ provided list). For example:

 var config = new ClientConfiguration
            {
                Servers = new List<Uri>
                {
                    new Uri("http://192.168.56.101:8091/pools"),
                    new Uri("http://192.168.56.102:8091/pools"),
                    new Uri("http://192.168.56.103:8091/pools"),
                    new Uri("http://192.168.56.104:8091/pools"),
                }
            };




Provide support for Management API (NCBC-594)

[NCBC-596] Create BucketManager class Created: 20/Aug/14  Updated: 20/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Technical task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Methods required:
-------------------------------
insertDesignDocument
updateDesignDocument
getDesignDocument
removeDesignDocument
listDesignDocuments
flush


https://docs.google.com/document/d/1HgVEJetcIfeIqviKC9zdlv_7IEkWpstatzxeydkLF3A/edit#heading=h.5pz82devlh8d
createBucket
removeBucket


https://docs.google.com/document/d/1HgVEJetcIfeIqviKC9zdlv_7IEkWpstatzxeydkLF3A/edit#heading=h.t4f3x16dy0fh







Provide support for Management API (NCBC-594)

[NCBC-595] Create and Implement ClusterManager class Created: 20/Aug/14  Updated: 20/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Technical task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Methods required:
addNode
removeNode
rebalance
listBuckets
createBucket
removeBucket


https://docs.google.com/document/d/1HgVEJetcIfeIqviKC9zdlv_7IEkWpstatzxeydkLF3A/edit#heading=h.t4f3x16dy0fh







[NCBC-594] Provide support for Management API Created: 20/Aug/14  Updated: 20/Aug/14

Status: In Progress
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Story Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Sub-Tasks:
Key
Summary
Type
Status
Assignee
NCBC-595 Create and Implement ClusterManager c... Technical task Open Jeff Morris  
NCBC-596 Create BucketManager class Technical task Open Jeff Morris  

 Description   
As an admin
I want to administrate my cluster
So that I can automate tasks such as adding and removing buckets.





[NCBC-592] AuthenticationException in CouchbaseBucket.SendWithRetry Created: 20/Aug/14  Updated: 21/Aug/14  Resolved: 20/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Andrei Baranouski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-592.log.zip    

 Description   
git log
commit 3fefdd5a17884c01d5113d73db2160bec9eb55db
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 18 22:31:37 2014 -0700

    NCBC-589: ThrowInvalidOperationException in CouchbaseConfigContext.LoadConfig during rebalance

    Change-Id: I813092f07bb6fde414ea4e05a67239173a540568

commit 732f2dcb72fe53cf7fbce27ac631a302ad9fe178
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 18 22:15:49 2014 -0700

    Check that length is positive in TypeSerializer.Deserialize

    Change-Id: Ic229fbdd0362593ac7577350cc04f549380ac7f5

commit 5c2c0e3589b5f0099fc8048de8b398c28f6eb04c
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Tue Aug 12 16:48:03 2014 -0700

    NCBC-581: System.InvalidOperationException when server list is empty

    Change-Id: I8fdef46b4057bc9f3b898b789fcccd78f689fdb6
    Reviewed-on: http://review.couchbase.org/40546
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Brett Lawson <brett19@gmail.com>

commit a132c5f921dcd9aed04d8f0d55dc5ba8ad36dd8e



C:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,14 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/count=2
    workload=HybridWorkloadGroup
    rebalance/mode=out
    testcase=RebalanceScenario
    Logging to C:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-20-14\025327\b0efa6fcc6ef738b9ac6a2109416adad

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-count 2 --workload HybridWorkloadGroup
 \ --rebalance-mode out --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,32 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,61 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,63 INFO] (RunContext run:124) Starting cluster and driver
[1,63 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1,63 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[1,64 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,93 INFO] (SDKD log:137) --- Logging Self-Test ---
[1,93 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[1,93 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[1,94 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[1,94 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[1,94 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,19 INFO] (SDKD log:137) Call SweepOnce()
[2,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,20 INFO] (SDKD log:137) Call SweepOnce()
[2,20 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,21 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[2,32 INFO] (SDKD log:137) Call SweepOnce()
[2,32 INFO] (SDKD log:137) Call SweepOnce()
[2,34 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[8,62 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.145:8091,ns_1@10.3.4.145>
[9,08 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.146:8091,ns_1@10.3.4.146>
[9,52 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[10,10 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[10,10 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,11 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,11 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[12,30 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[12,31 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[12,31 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[12,32 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[12,36 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[12,36 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[12,36 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[12,36 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[13,07 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@5417c2f
[13,07 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@65606a8d
[13,07 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@6653fd5c
[13,08 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@c890d48
[13,08 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@994c3c9
[33,08 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[39,35 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[52,86 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[55,92 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[56,79 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[56,79 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[57,01 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[57,24 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[66,46 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[69,00 INFO] (RunContext run:143) Driver and cluster initialized
[69,25 INFO] (RunContext call:167) Running scenario..
[69,25 INFO] (Scenario run:72) Starting RAMP phase
[69,25 INFO] (Workload setupDesign:63) Creating design test_design
[70,38 INFO] (Workload setupDesign:80) Design creation done
[70,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[70,39 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[70,39 INFO] (SDKD log:137) Call SweepOnce()
[70,39 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[70,40 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[70,42 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[70,43 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[70,45 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[70,46 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[78,02 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[78,02 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {Options={DelayMin=0, TimeRes=0, IterWait=1, DelayMax=0}, Schema={InflateLevel=40, InflateContent=meh}, DSType=DSTYPE_SEEDED, DS={KSize=12, KSeed=ViewFillerSeed, VSize=512, VSeed=Vie
wFillerSeed, Continuous=false, Repeat=rep, Count=1000}}
[78,03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[78,03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[78,03 INFO] (SDKD log:137) "Options": {
[78,03 INFO] (SDKD log:137) "DelayMin": 0,
[78,03 INFO] (SDKD log:137) "TimeRes": 0,
[78,04 INFO] (SDKD log:137) "IterWait": 1,
[78,04 INFO] (SDKD log:137) "DelayMax": 0
[78,04 INFO] (SDKD log:137) },
[78,05 INFO] (SDKD log:137) "Schema": {
[78,05 INFO] (SDKD log:137) "InflateLevel": 40,
[78,05 INFO] (SDKD log:137) "InflateContent": "meh"
[78,05 INFO] (SDKD log:137) },
[78,05 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[78,06 INFO] (SDKD log:137) "DS": {
[78,06 INFO] (SDKD log:137) "KSize": 12,
[78,07 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[78,07 INFO] (SDKD log:137) "VSize": 512,
[78,07 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[78,08 INFO] (SDKD log:137) "Continuous": false,
[78,08 INFO] (SDKD log:137) "Repeat": "rep",
[78,08 INFO] (SDKD log:137) "Count": 1000
[78,08 INFO] (SDKD log:137) }
[78,09 INFO] (SDKD log:137) }
[230,25 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[230,25 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[230,26 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[230,26 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[230,26 INFO] (Workload call:112) Preloading items into cluster before workload
[230,26 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[230,27 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[230,27 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[230,27 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[230,27 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[230,27 INFO] (SDKD log:137) Call SweepOnce()
[230,28 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[230,28 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[230,29 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[237,18 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[237,18 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@102.5 => {Options={DelayMin=0, TimeRes=0, IterWait=100, DelayMax=0}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=false, Repeat=REP, Coun
t=15000}}
[237,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[237,19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[237,20 INFO] (SDKD log:137) "Options": {
[237,20 INFO] (SDKD log:137) "DelayMin": 0,
[237,20 INFO] (SDKD log:137) "TimeRes": 0,
[237,20 INFO] (SDKD log:137) "IterWait": 100,
[237,21 INFO] (SDKD log:137) "DelayMax": 0
[237,21 INFO] (SDKD log:137) },
[237,21 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[237,21 INFO] (SDKD log:137) "DS": {
[237,22 INFO] (SDKD log:137) "KSize": 32,
[237,22 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[237,23 INFO] (SDKD log:137) "VSize": 128,
[237,23 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[237,23 INFO] (SDKD log:137) "Continuous": false,
[237,24 INFO] (SDKD log:137) "Repeat": "REP",
[237,24 INFO] (SDKD log:137) "Count": 15000
[237,24 INFO] (SDKD log:137) }
[237,24 INFO] (SDKD log:137) }
[412,79 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[1734,81 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@102.5 => {[OK]: 14990, [MEMD:GENERIC]: 10}
[1734,81 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[1734,81 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[1734,81 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[1734,81 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1508015768
[1734,82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1508015768
[1734,82 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.11 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,82 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.16 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,82 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.12 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.14 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.15 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.13 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.10 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.9 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.8 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1734,83 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,83 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,84 INFO] (SDKD log:137) Call SweepOnce()
[1734,84 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,84 INFO] (SDKD log:137) Call SweepOnce()
[1734,84 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,84 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,85 INFO] (SDKD log:137) Call SweepOnce()
[1734,85 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,86 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,86 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,86 INFO] (SDKD log:137) Call SweepOnce()
[1734,86 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,87 INFO] (SDKD log:137) Call SweepOnce()
[1734,87 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,87 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,88 INFO] (SDKD log:137) Call SweepOnce()
[1734,88 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,88 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,89 INFO] (SDKD log:137) Call SweepOnce()
[1734,89 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,89 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,89 INFO] (SDKD log:137) Call SweepOnce()
[1734,89 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,90 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,90 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[1734,91 INFO] (SDKD log:137) Call SweepOnce()
[1734,91 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,91 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1734,91 INFO] (SDKD log:137) Call SweepOnce()
[1734,92 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1734,92 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112
[1734,92 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[1734,93 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110
[1734,93 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111
[1734,93 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[1734,94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109
[1734,94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[1734,94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[1734,94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[1741,05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.11
[1741,10 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@107.17 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1741,10 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1741,11 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1741,11 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.18 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1741,11 INFO] (SDKD log:137) "Options": {
[1741,11 INFO] (SDKD log:137) "DelayMin": 1,
[1741,11 INFO] (SDKD log:137) "TimeRes": 1,
[1741,12 INFO] (SDKD log:137) "IterWait": 1,
[1741,12 INFO] (SDKD log:137) "DelayMax": 10
[1741,13 INFO] (SDKD log:137) },
[1741,13 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1741,13 INFO] (SDKD log:137) "DS": {
[1741,13 INFO] (SDKD log:137) "KSize": 32,
[1741,13 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1741,14 INFO] (SDKD log:137) "VSize": 128,
[1741,14 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1741,15 INFO] (SDKD log:137) "Continuous": true,
[1741,15 INFO] (SDKD log:137) "Repeat": "REP",
[1741,15 INFO] (SDKD log:137) "Count": 15000
[1741,15 INFO] (SDKD log:137) }
[1741,16 INFO] (SDKD log:137) }
[1741,16 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1741,16 INFO] (SDKD log:137) Call SweepOnce()
[1741,16 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1741,17 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113
[1746,76 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.16
[1746,76 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@112.19 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1746,76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1746,76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.20 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1746,76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1746,77 INFO] (SDKD log:137) "Options": {
[1746,77 INFO] (SDKD log:137) "DelayMin": 1,
[1746,77 INFO] (SDKD log:137) "TimeRes": 1,
[1746,77 INFO] (SDKD log:137) "IterWait": 1,
[1746,78 INFO] (SDKD log:137) "DelayMax": 10
[1746,78 INFO] (SDKD log:137) },
[1746,78 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1746,79 INFO] (SDKD log:137) "DS": {
[1746,79 INFO] (SDKD log:137) "KSize": 32,
[1746,79 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1746,80 INFO] (SDKD log:137) "VSize": 128,
[1746,80 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1746,80 INFO] (SDKD log:137) "Continuous": true,
[1746,80 INFO] (SDKD log:137) "Repeat": "REP",
[1746,80 INFO] (SDKD log:137) "Count": 15000
[1746,81 INFO] (SDKD log:137) }
[1746,81 INFO] (SDKD log:137) }
[1746,81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1746,82 INFO] (SDKD log:137) Call SweepOnce()
[1746,82 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1746,82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114
[1752,28 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.12
[1752,28 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.21 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1752,28 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1752,28 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1752,29 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.22 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1752,29 INFO] (SDKD log:137) "Options": {
[1752,29 INFO] (SDKD log:137) "DelayMin": 1,
[1752,29 INFO] (SDKD log:137) "TimeRes": 1,
[1752,29 INFO] (SDKD log:137) "IterWait": 1,
[1752,30 INFO] (SDKD log:137) "DelayMax": 10
[1752,30 INFO] (SDKD log:137) },
[1752,30 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1752,31 INFO] (SDKD log:137) "DS": {
[1752,31 INFO] (SDKD log:137) "KSize": 32,
[1752,31 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1752,31 INFO] (SDKD log:137) "VSize": 128,
[1752,32 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1752,32 INFO] (SDKD log:137) "Continuous": true,
[1752,32 INFO] (SDKD log:137) "Repeat": "REP",
[1752,33 INFO] (SDKD log:137) "Count": 15000
[1752,33 INFO] (SDKD log:137) }
[1752,33 INFO] (SDKD log:137) }
[1752,34 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1752,34 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1752,34 INFO] (SDKD log:137) Call SweepOnce()
[1752,34 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115
[1766,32 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.14
[1766,32 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@110.23 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1766,32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1766,33 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.24 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1766,33 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1766,33 INFO] (SDKD log:137) "Options": {
[1766,33 INFO] (SDKD log:137) "DelayMin": 1,
[1766,33 INFO] (SDKD log:137) "TimeRes": 1,
[1766,34 INFO] (SDKD log:137) "IterWait": 1,
[1766,34 INFO] (SDKD log:137) "DelayMax": 10
[1766,34 INFO] (SDKD log:137) },
[1766,35 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1766,35 INFO] (SDKD log:137) "DS": {
[1766,35 INFO] (SDKD log:137) "KSize": 32,
[1766,35 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1766,36 INFO] (SDKD log:137) "VSize": 128,
[1766,36 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1766,36 INFO] (SDKD log:137) "Continuous": true,
[1766,37 INFO] (SDKD log:137) "Repeat": "REP",
[1766,37 INFO] (SDKD log:137) "Count": 15000
[1766,37 INFO] (SDKD log:137) }
[1766,37 INFO] (SDKD log:137) }
[1766,37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1766,38 INFO] (SDKD log:137) Call SweepOnce()
[1766,38 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1766,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116
[1781,73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.15
[1781,73 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@111.25 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1781,74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1781,74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.26 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1781,74 INFO] (SDKD log:137) "Options": {
[1781,75 INFO] (SDKD log:137) "DelayMin": 1,
[1781,75 INFO] (SDKD log:137) "TimeRes": 1,
[1781,75 INFO] (SDKD log:137) "IterWait": 1,
[1781,75 INFO] (SDKD log:137) "DelayMax": 10
[1781,76 INFO] (SDKD log:137) },
[1781,76 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1781,76 INFO] (SDKD log:137) "DS": {
[1781,76 INFO] (SDKD log:137) "KSize": 32,
[1781,77 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1781,77 INFO] (SDKD log:137) "VSize": 128,
[1781,78 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1781,78 INFO] (SDKD log:137) "Continuous": true,
[1781,78 INFO] (SDKD log:137) "Repeat": "REP",
[1781,78 INFO] (SDKD log:137) "Count": 15000
[1781,79 INFO] (SDKD log:137) }
[1781,79 INFO] (SDKD log:137) }
[1781,79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,80 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1781,80 INFO] (SDKD log:137) Call SweepOnce()
[1781,80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117
[1787,45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.9
[1787,45 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@105.27 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1787,45 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1787,46 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.28 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1787,46 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1787,47 INFO] (SDKD log:137) "Options": {
[1787,47 INFO] (SDKD log:137) "DelayMin": 1,
[1787,47 INFO] (SDKD log:137) "TimeRes": 1,
[1787,47 INFO] (SDKD log:137) "IterWait": 1,
[1787,47 INFO] (SDKD log:137) "DelayMax": 10
[1787,48 INFO] (SDKD log:137) },
[1787,48 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1787,48 INFO] (SDKD log:137) "DS": {
[1787,49 INFO] (SDKD log:137) "KSize": 32,
[1787,49 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1787,49 INFO] (SDKD log:137) "VSize": 128,
[1787,50 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1787,50 INFO] (SDKD log:137) "Continuous": true,
[1787,50 INFO] (SDKD log:137) "Repeat": "REP",
[1787,50 INFO] (SDKD log:137) "Count": 15000
[1787,51 INFO] (SDKD log:137) }
[1787,51 INFO] (SDKD log:137) }
[1787,52 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1787,52 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1787,52 INFO] (SDKD log:137) Call SweepOnce()
[1787,52 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118
[1802,48 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.13
[1802,49 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@109.29 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1802,49 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1802,49 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.30 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1802,50 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1802,50 INFO] (SDKD log:137) "Options": {
[1802,50 INFO] (SDKD log:137) "DelayMin": 1,
[1802,50 INFO] (SDKD log:137) "TimeRes": 1,
[1802,51 INFO] (SDKD log:137) "IterWait": 1,
[1802,51 INFO] (SDKD log:137) "DelayMax": 10
[1802,51 INFO] (SDKD log:137) },
[1802,52 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1802,52 INFO] (SDKD log:137) "DS": {
[1802,52 INFO] (SDKD log:137) "KSize": 32,
[1802,52 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1802,53 INFO] (SDKD log:137) "VSize": 128,
[1802,53 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1802,53 INFO] (SDKD log:137) "Continuous": true,
[1802,53 INFO] (SDKD log:137) "Repeat": "REP",
[1802,54 INFO] (SDKD log:137) "Count": 15000
[1802,54 INFO] (SDKD log:137) }
[1802,55 INFO] (SDKD log:137) }
[1802,55 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1802,55 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1802,55 INFO] (SDKD log:137) Call SweepOnce()
[1802,55 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119
[1812,21 INFO] (SDKD log:137)
[1812,22 INFO] (SDKD log:137) Unhandle Exception: System.Security.Authentication.AuthenticationException: default
[1812,22 INFO] (SDKD log:137) at at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 78
[1812,22 INFO] (SDKD log:137) at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 43
[1812,23 INFO] (SDKD log:137) at Couchbase.Core.Server.Send[T](IOperation`1 operation) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 84
[1812,23 INFO] (SDKD log:137) at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 98
[1812,23 INFO] (SDKD log:137) at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
[1812,23 INFO] (SDKD log:137) at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) at c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18
[1812,24 INFO] (SDKD log:137) at Sdkd.Commands.KVCommand.Run() at c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
[1812,24 INFO] (SDKD log:137) at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) at c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235
[1812,24 INFO] (SDKD log:137) at SdkdConsole.Daemon.Handle.Run() at c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156
[1812,24 INFO] (SDKD log:137) at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
[1812,25 INFO] (SDKD log:137) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[1812,25 INFO] (SDKD log:137) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[1812,26 INFO] (SDKD log:137) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
[1812,26 INFO] (SDKD log:137) at System.Threading.ThreadHelper.ThreadStart()
[1977,47 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: java.net.SocketException: Connection reset
        at com.couchbase.sdkdclient.context.HarnessException.create(HarnessException.java:56) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.Workload.makeHandle(Workload.java:91) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.KeyValueWorkload.makeHandleCommon(KeyValueWorkload.java:70) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.KeyValueWorkload.access$200(KeyValueWorkload.java:24) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.KeyValueWorkload$1$1.call(KeyValueWorkload.java:119) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_11]
        at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_11]
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(Unknown Source) ~[na:1.8.0_11]
        at java.net.SocketInputStream.read(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamDecoder.implRead(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamDecoder.read(Unknown Source) ~[na:1.8.0_11]
        at java.io.InputStreamReader.read(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedReader.fill(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedReader.readLine(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedReader.readLine(Unknown Source) ~[na:1.8.0_11]
        at com.couchbase.sdkdclient.handle.Handle.receiveMessage(Handle.java:152) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.sendMessage(Handle.java:173) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.negotiate(Handle.java:237) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.HandleFactory.createHandle(HandleFactory.java:43) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.Workload.makeHandle(Workload.java:89) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        ... 7 common frames omitted
[1977,55 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.31
[1977,55 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]



 Comments   
Comment by Jeff Morris [ 20/Aug/14 ]
This is likely caused by:
a) missing or deleted bucket - the server will return an authentication error if it cannot locate the bucket.
b) IO error that's currently being hidden by the AuthenticationException
c) Bucket name and/or credentials are invalid

There is nothing I can do for "a" - the server would have to change the message it returns and the test runner may be deleting buckets. For "b", i am working on a patch to handle this - the result may still be an uncaught exception though in SDKD, for "c" that's the domain of the caller to manage.

I am not sure this would classifiy as a "test blocker" - a test blocker should be a bug (and the same bug) that appears on every run of every test and not intermittently/situation-ally.
Comment by Andrei Baranouski [ 20/Aug/14 ]
seems like a) and b) are not my cases. the test is performed without any problems for a while, and then fails(bucket is not removed)
it's reproduced every time with this scenario. will try some other.
Comment by Jeff Morris [ 20/Aug/14 ]
Andrei -

Can you describe in words this particular test (I am not a sdkd expert :))? Does this specific test fail every time (a good thing actually)?

My patch when completed, should fix the issue where the client tries to auth against a node that has been removed from the cluster - this causes most of the Auth exceptions.

Thanks,

Jeff
Comment by Jeff Morris [ 20/Aug/14 ]
This where the code failed:
2014-08-20 10:32:18,479 [21] DEBUG Couchbase.Authentication.SASL.CramMd5Mechanism - Authentication for socket fef9f34e-6c36-42a3-8970-24bd5487a28c failed: Timed out.
2014-08-20 10:32:18,479 [21] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Could not authenticate default using Couchbase.Authentication.SASL.CramMd5Mechanism - d8041012-7830-45dc-a97c-2b64a0a3215b.

It "Timed out", which means the remote host failed to respond within the grace period. Likely the server has been ejected from the cluster at that point.
Comment by Jeff Morris [ 20/Aug/14 ]
http://review.couchbase.org/#/c/40772/
Comment by Andrei Baranouski [ 21/Aug/14 ]
fixed




[NCBC-591] NullReferenceException in Couchbase.Configuration.Server.Providers.Streaming.HttpStreamingProvider.GetBucketConfig Created: 19/Aug/14  Updated: 21/Aug/14  Resolved: 19/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File config-with-1-node.json     File config-with-2-nodes.json     Zip Archive NCBC-591.log.zip    

 Description   
$ git log
commit 07c4c72cc7a34cc46149cf9be357c84913113fae
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 18 22:31:37 2014 -0700

    NCBC-589: ThrowInvalidOperationException in CouchbaseConfigContext.LoadConfig during rebalance

    Change-Id: I813092f07bb6fde414ea4e05a67239173a540568

commit 8ba11bbadd0d9d2b3621840b531e350a6b506760
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 18 22:15:49 2014 -0700

    Check that length is positive in TypeSerializer.Deserialize

    Change-Id: Ic229fbdd0362593ac7577350cc04f549380ac7f5

commit 5c2c0e3589b5f0099fc8048de8b398c28f6eb04c
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Tue Aug 12 16:48:03 2014 -0700

    NCBC-581: System.InvalidOperationException when server list is empty

    Change-Id: I8fdef46b4057bc9f3b898b789fcccd78f689fdb6
    Reviewed-on: http://review.couchbase.org/40546




I changed the sdkd-net code to a primitive version, and now each handler(read as thread of cb commands) I use separate CouchbaseCluster and CouchbaseBucket for each handler

using (var client = new CouchbaseCluster(config)){
   using (var SdkHandle = (CouchbaseBucket)client.OpenBucket("default", ""))
        {
            resp = new Sdkd.Protocol.Response(req, 0);
            SendMessage(resp);

            while (true)
            {
                req = GetMessage(true);
                if (req == null)
                {
                    HLogger.Warn("Socket closed without sending CLOSEHANDLE");
                    ReleaseHandle(SdkHandle);
                    return;
                }

                HLogger.Warn("Got command {0}", req.Command);
                HLogger.Warn("Command data: {0}", req.CommandData);

                try
                {
                    if (DispatchCommand(req, SdkHandle) == false)
                    {
                        ReleaseHandle(SdkHandle);
                        HLogger.Warn("DispatchCommand(req, SdkHandle) == false");
                        return;
                    }
                }
                catch (SdkdProtocolException exc)
                {
                    SendMessage(new Response(req, exc.ProtoErr));
                }
            }
        }


( I still face with http://www.couchbase.com/issues/browse/NCBC-575 problem when I use the same CouchbaseCluster for different CouchbaseBucket instances. will add my thoughts there)

c:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,12 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/count=2
    workload=HybridWorkloadGroup
    rebalance/mode=out
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-19-14\067477\8af0690bf5e9f06fa0c37ccbebf0379d

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-count 2 --workload HybridWorkloadGroup
 \ --rebalance-mode out --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,34 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,65 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,66 INFO] (RunContext run:124) Starting cluster and driver
[1,66 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,66 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1,66 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[1,82 INFO] (SDKD log:137) --- Logging Self-Test ---
[1,82 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[1,82 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[1,82 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[1,83 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[1,83 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,19 INFO] (SDKD log:137) Call SweepOnce()
[2,19 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,19 INFO] (SDKD log:137) Call SweepOnce()
[2,20 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,20 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[2,36 INFO] (SDKD log:137) Call SweepOnce()
[2,37 INFO] (SDKD log:137) Call SweepOnce()
[2,37 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[7,66 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.145:8091,ns_1@10.3.4.145>
[8,10 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.146:8091,ns_1@10.3.4.146>
[8,52 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[8,99 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[8,99 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[8,99 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[9,00 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,93 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[10,94 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[10,94 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[10,94 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[10,94 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[10,94 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[10,95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[10,95 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[11,65 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@325c364d
[11,65 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@447b4664
[11,66 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@2011af89
[11,66 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@2d0c6b05
[11,66 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@167a53aa
[26,07 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[34,03 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[44,85 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[50,40 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[51,26 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[51,26 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[51,47 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[51,69 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[60,67 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[63,17 INFO] (RunContext run:143) Driver and cluster initialized
[63,38 INFO] (RunContext call:167) Running scenario..
[63,38 INFO] (Scenario run:72) Starting RAMP phase
[63,38 INFO] (Workload setupDesign:63) Creating design test_design
[64,51 INFO] (Workload setupDesign:80) Design creation done
[64,51 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[64,51 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[64,51 INFO] (SDKD log:137) Call SweepOnce()
[64,52 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[64,52 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[64,54 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[64,54 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[64,57 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[64,57 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[71,93 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[71,93 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {Options={DelayMin=0, TimeRes=0, IterWait=1, DelayMax=0}, Schema={InflateLevel=40, InflateContent=meh}, DSType=DSTYPE_SEEDED, DS={KSize=12, KSeed=ViewFillerSeed, VSize=512, VSeed=Vie
wFillerSeed, Continuous=false, Repeat=rep, Count=1000}}
[71,93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[71,94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[71,94 INFO] (SDKD log:137) "Options": {
[71,94 INFO] (SDKD log:137) "DelayMin": 0,
[71,94 INFO] (SDKD log:137) "TimeRes": 0,
[71,95 INFO] (SDKD log:137) "IterWait": 1,
[71,95 INFO] (SDKD log:137) "DelayMax": 0
[71,95 INFO] (SDKD log:137) },
[71,95 INFO] (SDKD log:137) "Schema": {
[71,96 INFO] (SDKD log:137) "InflateLevel": 40,
[71,96 INFO] (SDKD log:137) "InflateContent": "meh"
[71,96 INFO] (SDKD log:137) },
[71,97 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[71,97 INFO] (SDKD log:137) "DS": {
[71,97 INFO] (SDKD log:137) "KSize": 12,
[71,97 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[71,98 INFO] (SDKD log:137) "VSize": 512,
[71,98 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[71,99 INFO] (SDKD log:137) "Continuous": false,
[71,99 INFO] (SDKD log:137) "Repeat": "rep",
[72,00 INFO] (SDKD log:137) "Count": 1000
[72,00 INFO] (SDKD log:137) }
[72,00 INFO] (SDKD log:137) }
[256,45 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[256,46 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[256,46 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[256,46 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[256,47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[256,47 INFO] (Workload call:112) Preloading items into cluster before workload
[256,47 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[256,47 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[256,48 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[256,48 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[256,48 INFO] (SDKD log:137) Call SweepOnce()
[256,49 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[256,49 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[256,49 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[265,65 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[265,65 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@102.5 => {Options={DelayMin=0, TimeRes=0, IterWait=100, DelayMax=0}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=false, Repeat=REP, Coun
t=15000}}
[265,66 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[265,66 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[265,67 INFO] (SDKD log:137) "Options": {
[265,67 INFO] (SDKD log:137) "DelayMin": 0,
[265,67 INFO] (SDKD log:137) "TimeRes": 0,
[265,67 INFO] (SDKD log:137) "IterWait": 100,
[265,67 INFO] (SDKD log:137) "DelayMax": 0
[265,67 INFO] (SDKD log:137) },
[265,68 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[265,68 INFO] (SDKD log:137) "DS": {
[265,68 INFO] (SDKD log:137) "KSize": 32,
[265,68 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[265,69 INFO] (SDKD log:137) "VSize": 128,
[265,69 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[265,69 INFO] (SDKD log:137) "Continuous": false,
[265,69 INFO] (SDKD log:137) "Repeat": "REP",
[265,69 INFO] (SDKD log:137) "Count": 15000
[265,70 INFO] (SDKD log:137) }
[265,70 INFO] (SDKD log:137) }
[405,30 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[1650,02 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@102.5 => {[OK]: 14985, [MEMD:GENERIC]: 15}
[1650,02 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[1650,02 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[1650,03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[1650,03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1744713491
[1650,03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.8 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.11 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.12 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,04 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.10 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,05 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.9 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,05 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1744713491
[1650,06 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.16 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,06 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.13 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,06 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.15 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,06 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.14 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1650,07 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[1650,07 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,07 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,08 INFO] (SDKD log:137) Call SweepOnce()
[1650,08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,08 INFO] (SDKD log:137) Call SweepOnce()
[1650,08 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,09 INFO] (SDKD log:137) Call SweepOnce()
[1650,09 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,09 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,10 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[1650,10 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[1650,10 INFO] (SDKD log:137) Call SweepOnce()
[1650,11 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,11 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,11 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[1650,12 INFO] (SDKD log:137) Call SweepOnce()
[1650,12 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,13 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,13 INFO] (SDKD log:137) Call SweepOnce()
[1650,13 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,13 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,14 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[1650,14 INFO] (SDKD log:137) Call SweepOnce()
[1650,14 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,15 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,15 INFO] (SDKD log:137) Call SweepOnce()
[1650,15 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,15 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,16 INFO] (SDKD log:137) Call SweepOnce()
[1650,16 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,16 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1650,17 INFO] (SDKD log:137) Call SweepOnce()
[1650,17 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1650,17 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[1650,18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[1650,18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112
[1650,18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109
[1650,18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111
[1650,18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110
[1655,77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[1655,78 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@103.17 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1655,78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1655,78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1655,78 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.18 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1655,78 INFO] (SDKD log:137) "Options": {
[1655,79 INFO] (SDKD log:137) "DelayMin": 1,
[1655,79 INFO] (SDKD log:137) "TimeRes": 1,
[1655,79 INFO] (SDKD log:137) "IterWait": 1,
[1655,79 INFO] (SDKD log:137) "DelayMax": 10
[1655,79 INFO] (SDKD log:137) },
[1655,79 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1655,79 INFO] (SDKD log:137) "DS": {
[1655,79 INFO] (SDKD log:137) "KSize": 32,
[1655,80 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1655,80 INFO] (SDKD log:137) "VSize": 128,
[1655,80 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1655,80 INFO] (SDKD log:137) "Continuous": true,
[1655,80 INFO] (SDKD log:137) "Repeat": "REP",
[1655,80 INFO] (SDKD log:137) "Count": 15000
[1655,80 INFO] (SDKD log:137) }
[1655,81 INFO] (SDKD log:137) }
[1655,81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1655,81 INFO] (SDKD log:137) Call SweepOnce()
[1655,81 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1655,81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113
[1661,66 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.11
[1661,66 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@107.19 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1661,67 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1661,67 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.20 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1661,67 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1661,68 INFO] (SDKD log:137) "Options": {
[1661,68 INFO] (SDKD log:137) "DelayMin": 1,
[1661,68 INFO] (SDKD log:137) "TimeRes": 1,
[1661,68 INFO] (SDKD log:137) "IterWait": 1,
[1661,69 INFO] (SDKD log:137) "DelayMax": 10
[1661,69 INFO] (SDKD log:137) },
[1661,69 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1661,69 INFO] (SDKD log:137) "DS": {
[1661,70 INFO] (SDKD log:137) "KSize": 32,
[1661,70 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1661,70 INFO] (SDKD log:137) "VSize": 128,
[1661,70 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1661,70 INFO] (SDKD log:137) "Continuous": true,
[1661,71 INFO] (SDKD log:137) "Repeat": "REP",
[1661,71 INFO] (SDKD log:137) "Count": 15000
[1661,71 INFO] (SDKD log:137) }
[1661,71 INFO] (SDKD log:137) }
[1661,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1661,72 INFO] (SDKD log:137) Call SweepOnce()
[1661,72 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1661,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114
[1667,55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.8
[1667,55 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@104.21 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1667,55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1667,55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1667,56 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.22 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1667,56 INFO] (SDKD log:137) "Options": {
[1667,56 INFO] (SDKD log:137) "DelayMin": 1,
[1667,56 INFO] (SDKD log:137) "TimeRes": 1,
[1667,57 INFO] (SDKD log:137) "IterWait": 1,
[1667,57 INFO] (SDKD log:137) "DelayMax": 10
[1667,57 INFO] (SDKD log:137) },
[1667,57 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1667,57 INFO] (SDKD log:137) "DS": {
[1667,58 INFO] (SDKD log:137) "KSize": 32,
[1667,58 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1667,58 INFO] (SDKD log:137) "VSize": 128,
[1667,58 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1667,59 INFO] (SDKD log:137) "Continuous": true,
[1667,59 INFO] (SDKD log:137) "Repeat": "REP",
[1667,59 INFO] (SDKD log:137) "Count": 15000
[1667,59 INFO] (SDKD log:137) }
[1667,60 INFO] (SDKD log:137) }
[1667,60 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1667,61 INFO] (SDKD log:137) Call SweepOnce()
[1667,61 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1667,61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115
[1672,50 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.12
[1672,51 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.23 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1672,51 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1672,51 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1672,52 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.24 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1672,52 INFO] (SDKD log:137) "Options": {
[1672,53 INFO] (SDKD log:137) "DelayMin": 1,
[1672,53 INFO] (SDKD log:137) "TimeRes": 1,
[1672,53 INFO] (SDKD log:137) "IterWait": 1,
[1672,53 INFO] (SDKD log:137) "DelayMax": 10
[1672,53 INFO] (SDKD log:137) },
[1672,53 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1672,54 INFO] (SDKD log:137) "DS": {
[1672,54 INFO] (SDKD log:137) "KSize": 32,
[1672,54 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1672,54 INFO] (SDKD log:137) "VSize": 128,
[1672,55 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1672,55 INFO] (SDKD log:137) "Continuous": true,
[1672,55 INFO] (SDKD log:137) "Repeat": "REP",
[1672,55 INFO] (SDKD log:137) "Count": 15000
[1672,56 INFO] (SDKD log:137) }
[1672,56 INFO] (SDKD log:137) }
[1672,56 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1672,56 INFO] (SDKD log:137) Call SweepOnce()
[1672,56 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1672,56 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116
[1687,65 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.10
[1687,65 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@106.25 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1687,66 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1687,66 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1687,66 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.26 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1687,66 INFO] (SDKD log:137) "Options": {
[1687,67 INFO] (SDKD log:137) "DelayMin": 1,
[1687,67 INFO] (SDKD log:137) "TimeRes": 1,
[1687,67 INFO] (SDKD log:137) "IterWait": 1,
[1687,67 INFO] (SDKD log:137) "DelayMax": 10
[1687,68 INFO] (SDKD log:137) },
[1687,68 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1687,68 INFO] (SDKD log:137) "DS": {
[1687,68 INFO] (SDKD log:137) "KSize": 32,
[1687,69 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1687,69 INFO] (SDKD log:137) "VSize": 128,
[1687,69 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1687,69 INFO] (SDKD log:137) "Continuous": true,
[1687,69 INFO] (SDKD log:137) "Repeat": "REP",
[1687,70 INFO] (SDKD log:137) "Count": 15000
[1687,70 INFO] (SDKD log:137) }
[1687,70 INFO] (SDKD log:137) }
[1687,70 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1687,70 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1687,71 INFO] (SDKD log:137) Call SweepOnce()
[1687,71 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117
[1702,13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.9
[1702,13 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@105.27 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1702,14 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1702,14 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1702,14 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.28 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1702,14 INFO] (SDKD log:137) "Options": {
[1702,14 INFO] (SDKD log:137) "DelayMin": 1,
[1702,15 INFO] (SDKD log:137) "TimeRes": 1,
[1702,15 INFO] (SDKD log:137) "IterWait": 1,
[1702,15 INFO] (SDKD log:137) "DelayMax": 10
[1702,15 INFO] (SDKD log:137) },
[1702,16 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1702,16 INFO] (SDKD log:137) "DS": {
[1702,16 INFO] (SDKD log:137) "KSize": 32,
[1702,16 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1702,16 INFO] (SDKD log:137) "VSize": 128,
[1702,17 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1702,17 INFO] (SDKD log:137) "Continuous": true,
[1702,17 INFO] (SDKD log:137) "Repeat": "REP",
[1702,17 INFO] (SDKD log:137) "Count": 15000
[1702,18 INFO] (SDKD log:137) }
[1702,18 INFO] (SDKD log:137) }
[1702,18 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1702,18 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1702,18 INFO] (SDKD log:137) Call SweepOnce()
[1702,19 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118
[1808,14 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.13
[1808,15 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@109.29 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1808,15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1808,15 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.30 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1808,15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1808,16 INFO] (SDKD log:137) "Options": {
[1808,16 INFO] (SDKD log:137) "DelayMin": 1,
[1808,16 INFO] (SDKD log:137) "TimeRes": 1,
[1808,17 INFO] (SDKD log:137) "IterWait": 1,
[1808,17 INFO] (SDKD log:137) "DelayMax": 10
[1808,17 INFO] (SDKD log:137) },
[1808,17 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1808,18 INFO] (SDKD log:137) "DS": {
[1808,18 INFO] (SDKD log:137) "KSize": 32,
[1808,19 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1808,19 INFO] (SDKD log:137) "VSize": 128,
[1808,19 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1808,19 INFO] (SDKD log:137) "Continuous": true,
[1808,20 INFO] (SDKD log:137) "Repeat": "REP",
[1808,20 INFO] (SDKD log:137) "Count": 15000
[1808,20 INFO] (SDKD log:137) }
[1808,20 INFO] (SDKD log:137) }
[1808,21 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1808,21 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[1808,22 INFO] (SDKD log:137) Call SweepOnce()
[1808,22 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119
[2014,15 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.16
[2014,15 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@112.31 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[2014,15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[2014,16 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[2014,16 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.32 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[2014,16 INFO] (SDKD log:137) "Options": {
[2014,16 INFO] (SDKD log:137) "DelayMin": 1,
[2014,16 INFO] (SDKD log:137) "TimeRes": 1,
[2014,16 INFO] (SDKD log:137) "IterWait": 1,
[2014,17 INFO] (SDKD log:137) "DelayMax": 10
[2014,17 INFO] (SDKD log:137) },
[2014,17 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[2014,17 INFO] (SDKD log:137) "DS": {
[2014,17 INFO] (SDKD log:137) "KSize": 32,
[2014,18 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[2014,18 INFO] (SDKD log:137) "VSize": 128,
[2014,18 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[2014,18 INFO] (SDKD log:137) "Continuous": true,
[2014,18 INFO] (SDKD log:137) "Repeat": "REP",
[2014,19 INFO] (SDKD log:137) "Count": 15000
[2014,19 INFO] (SDKD log:137) }
[2014,19 INFO] (SDKD log:137) }
[2014,19 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2014,19 INFO] (SDKD log:137) !!!!!!!!!!!!!!!
[2014,19 INFO] (SDKD log:137) Call SweepOnce()
[2014,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120
[2115,71 INFO] (SDKD log:137)
[2115,72 INFO] (SDKD log:137) Unhandle Exception: System.NullReferenceException: Object reference not set to an instance of the object.
[2115,72 INFO] (SDKD log:137) в Couchbase.Configuration.Server.Providers.Streaming.HttpStreamingProvider.GetBucketConfig(String bucketName, String password) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Provid
ers\Streaming\HttpStreamingProvider.cs:line 247
[2115,72 INFO] (SDKD log:137) at Couchbase.Configuration.Server.Providers.Streaming.HttpStreamingProvider.GetConfig(String bucketName, String password) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\St
reaming\HttpStreamingProvider.cs:line 54
[2115,73 INFO] (SDKD log:137) в Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159
[2115,73 INFO] (SDKD log:137) в Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 84
[2115,73 INFO] (SDKD log:137) в SdkdConsole.Daemon.Handle.Run() в c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 134
[2115,73 INFO] (SDKD log:137) в System.Threading.ThreadHelper.ThreadStart_Context(Object state)
[2115,73 INFO] (SDKD log:137) в System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[2115,74 INFO] (SDKD log:137) в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[2115,74 INFO] (SDKD log:137) в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
[2115,74 INFO] (SDKD log:137) в System.Threading.ThreadHelper.ThreadStart()
[2151,61 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_11]
        at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_11]
[2151,62 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.33
[2151,63 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]



 Comments   
Comment by Jeff Morris [ 19/Aug/14 ]
>>I changed the sdkd-net code to a primitive version, and now each handler(read as thread of cb commands) I use separate CouchbaseCluster and CouchbaseBucket for each handler

Good, this should make it easier to isolate issues. Let me dig through the logs and see what I find.
Comment by Jeff Morris [ 19/Aug/14 ]
The issue appears to be that when a cluster is a single node, it returns $HOST as the hostname in the nodes element of the config. If the cluster is greater than 1, then it returns the proper hostname for each node. The resolution for $HOST fails, thus making the BucketConfig object null when it's referenced. Please see attached JSON files.

Note this is using CB 3.0 beta 2.
Comment by Matt Ingenthron [ 19/Aug/14 ]
The $HOST is expected. You might recall that from a while ago. The client library is supposed to swap in the IP/hostname it used to retrieve that config.
Comment by Jeff Morris [ 19/Aug/14 ]
Yeah, I remember $HOST...$HOST is interesting, because it may or may not be in the config: after adding the second node, I removed it making it a single node cluster and no $HOST was present - it was using the IP again.
Comment by Jeff Morris [ 19/Aug/14 ]
http://review.couchbase.org/#/c/40742/




[NCBC-590] Refactor logging so that ODE are not raised when it references a disposed object Created: 18/Aug/14  Updated: 18/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
2014-08-16 08:53:24,635 [33] INFO Couchbase.IO.ConnectionBase - <log4net.Error>Exception rendering object type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]<stackTrace>System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
   at System.Net.Sockets.Socket.get_RemoteEndPoint()
   at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line 38
   at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString()
   at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer)
   at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer)</stackTrace></log4net.Error>




[NCBC-589] ThrowInvalidOperationException in Couchbase.Configuration.CouchbaseConfigContext.LoadConfig during rebalance Created: 16/Aug/14  Updated: 21/Aug/14  Resolved: 19/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Andrei Baranouski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-589.zip    
Issue Links:
Gantt: finish-start

 Description   
I took a few commits ahead from gerrit


commit 2c00a9d3a397f5c1eddc56238cf4a19e6e6618dd
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 11 18:41:08 2014 -0700

    Add config files to tests

    Change-Id: Ie093eac1da4ffc742796ffd4df8a9c63cc1c9c2f



c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0.32 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/mode=out
    workload=HybridWorkloadGroup
    rebalance/count=2
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-16-14\056660\742db0c42511883a3b8960d57d363be0

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-mode out --workload HybridWorkloadGroup
 \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1.52 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1.93 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1.95 INFO] (RunContext run:124) Starting cluster and driver
[1.96 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1.96 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1.96 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[2.30 INFO] (SDKD log:137) --- Logging Self-Test ---
[2.31 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[2.31 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[2.32 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[2.32 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[2.32 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2.48 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2.49 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2.78 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}}
[4.34 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[4.43 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa
[4.43 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[4.43 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa
[4.43 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[4.94 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[4.95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[4.95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[4.97 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[4.97 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[4.97 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[4.98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[4.98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[5.18 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@7b0bbd42
[5.19 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@10d4f27
[5.19 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@b22c9df
[5.20 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@74bfd10a
[5.20 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@6ae2d0b2
[17.28 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[24.00 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[36.81 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[43.42 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[43.46 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[44.47 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[44.47 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[44.48 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[44.49 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[54.57 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[54.62 INFO] (RunContext run:143) Driver and cluster initialized
[54.65 INFO] (RunContext call:167) Running scenario..
[54.66 INFO] (Scenario run:72) Starting RAMP phase
[54.66 INFO] (Workload setupDesign:63) Creating design test_design
[54.93 INFO] (Workload setupDesign:80) Design creation done
[54.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[54.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[54.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[54.96 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[54.97 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[54.97 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[55.00 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[55.01 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[55.42 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[55.42 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12
}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[55.43 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[55.43 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[55.43 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[55.43 INFO] (SDKD log:137) "DS": {
[55.43 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[55.44 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[55.44 INFO] (SDKD log:137) "Count": 1000,
[55.44 INFO] (SDKD log:137) "Continuous": false,
[55.44 INFO] (SDKD log:137) "VSize": 512,
[55.44 INFO] (SDKD log:137) "Repeat": "rep",
[55.44 INFO] (SDKD log:137) "KSize": 12
[55.45 INFO] (SDKD log:137) },
[55.45 INFO] (SDKD log:137) "Schema": {
[55.45 INFO] (SDKD log:137) "InflateContent": "meh",
[55.45 INFO] (SDKD log:137) "InflateLevel": 40
[55.45 INFO] (SDKD log:137) },
[55.46 INFO] (SDKD log:137) "Options": {
[55.46 INFO] (SDKD log:137) "DelayMax": 0,
[55.46 INFO] (SDKD log:137) "DelayMin": 0,
[55.46 INFO] (SDKD log:137) "IterWait": 1,
[55.46 INFO] (SDKD log:137) "TimeRes": 0
[55.46 INFO] (SDKD log:137) }
[55.47 INFO] (SDKD log:137) }
[56.75 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[56.76 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[56.76 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[56.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[56.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[56.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[56.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[56.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[56.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[56.79 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[56.79 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12
}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[56.79 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[56.80 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[56.80 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[56.80 INFO] (SDKD log:137) "DS": {
[56.81 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[56.81 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[56.81 INFO] (SDKD log:137) "Count": 1000,
[56.81 INFO] (SDKD log:137) "Continuous": false,
[56.81 INFO] (SDKD log:137) "VSize": 512,
[56.82 INFO] (SDKD log:137) "Repeat": "rep",
[56.82 INFO] (SDKD log:137) "KSize": 12
[56.82 INFO] (SDKD log:137) },
[56.82 INFO] (SDKD log:137) "Schema": {
[56.82 INFO] (SDKD log:137) "InflateContent": "meh",
[56.83 INFO] (SDKD log:137) "InflateLevel": 40
[56.83 INFO] (SDKD log:137) },
[56.83 INFO] (SDKD log:137) "Options": {
[56.83 INFO] (SDKD log:137) "DelayMax": 0,
[56.83 INFO] (SDKD log:137) "DelayMin": 0,
[56.84 INFO] (SDKD log:137) "IterWait": 1,
[56.84 INFO] (SDKD log:137) "TimeRes": 0
[56.84 INFO] (SDKD log:137) }
[56.84 INFO] (SDKD log:137) }
[58.30 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[58.31 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@102.5
[58.31 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[58.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[58.31 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[58.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[58.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[58.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[58.34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[58.34 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCou
nt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[58.35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[58.35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[58.35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[58.35 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[58.35 INFO] (SDKD log:137) "DS": {},
[58.36 INFO] (SDKD log:137) "ViewParameters": {},
[58.36 INFO] (SDKD log:137) "DesignName": "test_design",
[58.36 INFO] (SDKD log:137) "Options": {
[58.36 INFO] (SDKD log:137) "DelayMax": 10,
[58.36 INFO] (SDKD log:137) "DelayMin": 1,
[58.37 INFO] (SDKD log:137) "IterWait": 1,
[58.37 INFO] (SDKD log:137) "ViewQueryCount": -1,
[58.37 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[58.37 INFO] (SDKD log:137) "TimeRes": 1
[58.38 INFO] (SDKD log:137) },
[58.38 INFO] (SDKD log:137) "ViewName": "test_view"
[58.38 INFO] (SDKD log:137) }
[58.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[58.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[58.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9
[58.40 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[58.41 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[58.41 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[58.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[58.41 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[58.41 INFO] (SDKD log:137) "DS": {},
[58.42 INFO] (SDKD log:137) "ViewParameters": {},
[58.42 INFO] (SDKD log:137) "DesignName": "test_design",
[58.42 INFO] (SDKD log:137) "Options": {
[58.42 INFO] (SDKD log:137) "DelayMax": 10,
[58.42 INFO] (SDKD log:137) "DelayMin": 1,
[58.42 INFO] (SDKD log:137) "IterWait": 1,
[58.43 INFO] (SDKD log:137) "ViewQueryCount": -1,
[58.43 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[58.43 INFO] (SDKD log:137) "TimeRes": 1
[58.43 INFO] (SDKD log:137) },
[58.43 INFO] (SDKD log:137) "ViewName": "test_view"
[58.44 INFO] (SDKD log:137) }
[58.44 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.44 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[58.44 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[58.57 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11
[58.58 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[58.58 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[58.58 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[58.59 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[58.59 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[58.59 INFO] (SDKD log:137) "DS": {},
[58.59 INFO] (SDKD log:137) "ViewParameters": {},
[58.59 INFO] (SDKD log:137) "DesignName": "test_design",
[58.60 INFO] (SDKD log:137) "Options": {
[58.60 INFO] (SDKD log:137) "DelayMax": 10,
[58.60 INFO] (SDKD log:137) "DelayMin": 1,
[58.60 INFO] (SDKD log:137) "IterWait": 1,
[58.60 INFO] (SDKD log:137) "ViewQueryCount": -1,
[58.60 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[58.60 INFO] (SDKD log:137) "TimeRes": 1
[58.61 INFO] (SDKD log:137) },
[58.61 INFO] (SDKD log:137) "ViewName": "test_view"
[58.61 INFO] (SDKD log:137) }
[58.61 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[58.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[58.73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13
[58.73 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[58.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[58.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[58.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[58.74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[58.74 INFO] (SDKD log:137) "DS": {},
[58.74 INFO] (SDKD log:137) "ViewParameters": {},
[58.74 INFO] (SDKD log:137) "DesignName": "test_design",
[58.74 INFO] (SDKD log:137) "Options": {
[58.75 INFO] (SDKD log:137) "DelayMax": 10,
[58.75 INFO] (SDKD log:137) "DelayMin": 1,
[58.75 INFO] (SDKD log:137) "IterWait": 1,
[58.75 INFO] (SDKD log:137) "ViewQueryCount": -1,
[58.75 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[58.76 INFO] (SDKD log:137) "TimeRes": 1
[58.76 INFO] (SDKD log:137) },
[58.76 INFO] (SDKD log:137) "ViewName": "test_view"
[58.76 INFO] (SDKD log:137) }
[58.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[58.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[58.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15
[58.77 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[58.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[58.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[58.78 INFO] (Workload call:112) Preloading items into cluster before workload
[58.78 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[58.78 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[58.79 INFO] (SDKD log:137) "DS": {},
[58.79 INFO] (SDKD log:137) "ViewParameters": {},
[58.79 INFO] (SDKD log:137) "DesignName": "test_design",
[58.79 INFO] (SDKD log:137) "Options": {
[58.79 INFO] (SDKD log:137) "DelayMax": 10,
[58.80 INFO] (SDKD log:137) "DelayMin": 1,
[58.80 INFO] (SDKD log:137) "IterWait": 1,
[58.80 INFO] (SDKD log:137) "ViewQueryCount": -1,
[58.80 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[58.80 INFO] (SDKD log:137) "TimeRes": 1
[58.81 INFO] (SDKD log:137) },
[58.81 INFO] (SDKD log:137) "ViewName": "test_view"
[58.81 INFO] (SDKD log:137) }
[58.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[58.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[58.81 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[58.82 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17
[58.82 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}}
[58.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[58.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[58.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[58.83 INFO] (SDKD log:137) "DS": {
[58.83 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[58.84 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[58.84 INFO] (SDKD log:137) "Count": 15000,
[58.84 INFO] (SDKD log:137) "Continuous": false,
[58.84 INFO] (SDKD log:137) "VSize": 128,
[58.84 INFO] (SDKD log:137) "Repeat": "REP",
[58.85 INFO] (SDKD log:137) "KSize": 32
[58.85 INFO] (SDKD log:137) },
[58.85 INFO] (SDKD log:137) "Options": {
[58.85 INFO] (SDKD log:137) "DelayMax": 0,
[58.85 INFO] (SDKD log:137) "DelayMin": 0,
[58.85 INFO] (SDKD log:137) "IterWait": 100,
[58.86 INFO] (SDKD log:137) "TimeRes": 0
[58.86 INFO] (SDKD log:137) }
[58.86 INFO] (SDKD log:137) }
[249.74 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 4003, [MEMD:MEMD_EVBUCKET]: 10997}
[249.74 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19
[249.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[249.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[249.74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[249.74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[249.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.77 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109
[249.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114
[249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116
[249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115
[249.80 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20
[249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.81 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@109.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110
[249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113
[249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111
[249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117
[249.82 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112
[249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118
[249.82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[249.82 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[249.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[249.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[249.83 INFO] (SDKD log:137) "DS": {
[249.83 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[249.83 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[249.83 INFO] (SDKD log:137) "Count": 15000,
[249.84 INFO] (SDKD log:137) "Continuous": true,
[249.84 INFO] (SDKD log:137) "VSize": 128,
[249.84 INFO] (SDKD log:137) "Repeat": "REP",
[249.84 INFO] (SDKD log:137) "KSize": 32
[249.84 INFO] (SDKD log:137) },
[249.84 INFO] (SDKD log:137) "Options": {
[249.84 INFO] (SDKD log:137) "DelayMax": 10,
[249.85 INFO] (SDKD log:137) "DelayMin": 1,
[249.85 INFO] (SDKD log:137) "IterWait": 1,
[249.85 INFO] (SDKD log:137) "TimeRes": 1
[249.85 INFO] (SDKD log:137) }
[249.86 INFO] (SDKD log:137) }
[249.86 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25
[249.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.87 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[249.87 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119
[249.87 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[249.87 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[249.88 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[249.88 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[249.88 INFO] (SDKD log:137) "DS": {
[249.89 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[249.89 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[249.89 INFO] (SDKD log:137) "Count": 15000,
[249.89 INFO] (SDKD log:137) "Continuous": true,
[249.89 INFO] (SDKD log:137) "VSize": 128,
[249.90 INFO] (SDKD log:137) "Repeat": "REP",
[249.90 INFO] (SDKD log:137) "KSize": 32
[249.90 INFO] (SDKD log:137) },
[249.90 INFO] (SDKD log:137) "Options": {
[249.90 INFO] (SDKD log:137) "DelayMax": 10,
[249.90 INFO] (SDKD log:137) "DelayMin": 1,
[249.90 INFO] (SDKD log:137) "IterWait": 1,
[249.91 INFO] (SDKD log:137) "TimeRes": 1
[249.91 INFO] (SDKD log:137) }
[249.91 INFO] (SDKD log:137) }
[249.91 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27
[249.92 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120
[249.92 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@116.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[249.92 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[249.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[249.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[249.93 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[249.93 INFO] (SDKD log:137) "DS": {
[249.94 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[249.94 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[249.94 INFO] (SDKD log:137) "Count": 15000,
[249.94 INFO] (SDKD log:137) "Continuous": true,
[249.94 INFO] (SDKD log:137) "VSize": 128,
[249.94 INFO] (SDKD log:137) "Repeat": "REP",
[249.95 INFO] (SDKD log:137) "KSize": 32
[249.95 INFO] (SDKD log:137) },
[249.95 INFO] (SDKD log:137) "Options": {
[249.95 INFO] (SDKD log:137) "DelayMax": 10,
[249.95 INFO] (SDKD log:137) "DelayMin": 1,
[249.96 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26
[249.96 INFO] (SDKD log:137) "IterWait": 1,
[249.96 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@115.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[249.96 INFO] (SDKD log:137) "TimeRes": 1
[249.96 INFO] (SDKD log:137) }
[249.96 INFO] (SDKD log:137) }
[249.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[249.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[249.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121
[249.98 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[249.98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[249.98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[249.98 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[249.98 INFO] (SDKD log:137) "DS": {
[249.98 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[249.99 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[249.99 INFO] (SDKD log:137) "Count": 15000,
[249.99 INFO] (SDKD log:137) "Continuous": true,
[249.99 INFO] (SDKD log:137) "VSize": 128,
[249.99 INFO] (SDKD log:137) "Repeat": "REP",
[249.99 INFO] (SDKD log:137) "KSize": 32
[250.00 INFO] (SDKD log:137) },
[250.00 INFO] (SDKD log:137) "Options": {
[250.00 INFO] (SDKD log:137) "DelayMax": 10,
[250.01 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24
[250.01 INFO] (SDKD log:137) "DelayMin": 1,
[250.01 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.01 INFO] (SDKD log:137) "IterWait": 1,
[250.02 INFO] (SDKD log:137) "TimeRes": 1
[250.02 INFO] (SDKD log:137) }
[250.02 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[250.02 INFO] (SDKD log:137) }
[250.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122
[250.03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.03 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.03 INFO] (SDKD log:137) "DS": {
[250.04 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.04 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.04 INFO] (SDKD log:137) "Count": 15000,
[250.04 INFO] (SDKD log:137) "Continuous": true,
[250.04 INFO] (SDKD log:137) "VSize": 128,
[250.04 INFO] (SDKD log:137) "Repeat": "REP",
[250.05 INFO] (SDKD log:137) "KSize": 32
[250.05 INFO] (SDKD log:137) },
[250.05 INFO] (SDKD log:137) "Options": {
[250.05 INFO] (SDKD log:137) "DelayMax": 10,
[250.05 INFO] (SDKD log:137) "DelayMin": 1,
[250.05 INFO] (SDKD log:137) "IterWait": 1,
[250.06 INFO] (SDKD log:137) "TimeRes": 1
[250.07 INFO] (SDKD log:137) }
[250.07 INFO] (SDKD log:137) }
[250.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.08 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21
[250.08 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@110.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123
[250.09 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.09 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[250.10 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.11 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124
[250.12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.12 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.13 INFO] (SDKD log:137) "DS": {
[250.13 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[250.14 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.15 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.15 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[250.15 INFO] (SDKD log:137) "Count": 15000,
[250.15 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[250.15 INFO] (SDKD log:137) "Continuous": true,
[250.15 INFO] (SDKD log:137) "VSize": 128,
[250.16 INFO] (SDKD log:137) "Repeat": "REP",
[250.16 INFO] (SDKD log:137) "KSize": 32
[250.16 INFO] (SDKD log:137) },
[250.16 INFO] (SDKD log:137) "Options": {
[250.16 INFO] (SDKD log:137) "DelayMax": 10,
[250.17 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1
[250.17 INFO] (SDKD log:137) "DelayMin": 1,
[250.17 INFO] (SDKD log:137) "IterWait": 1,
[250.18 INFO] (SDKD log:137) at System.Net.Sockets.Socket.get_RemoteEndPoint()
   at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line
38
   at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString()
   at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer)
   at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer)
[250.18 INFO] (SDKD log:137) "TimeRes": 1
[250.18 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[250.19 INFO] (SDKD log:137) }
[250.19 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[250.19 INFO] (SDKD log:137) }
[250.19 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[250.20 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.21 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[250.21 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22
[250.21 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@111.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.22 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[250.23 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[250.23 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.23 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.23 INFO] (SDKD log:137) "DS": {
[250.24 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.24 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.24 INFO] (SDKD log:137) "Count": 15000,
[250.24 INFO] (SDKD log:137) "Continuous": true,
[250.25 INFO] (SDKD log:137) "VSize": 128,
[250.25 INFO] (SDKD log:137) "Repeat": "REP",
[250.25 INFO] (SDKD log:137) "KSize": 32
[250.25 INFO] (SDKD log:137) },
[250.25 INFO] (SDKD log:137) "Options": {
[250.27 INFO] (SDKD log:137) "DelayMax": 10,
[250.28 INFO] (SDKD log:137) "DelayMin": 1,
[250.28 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28
[250.28 INFO] (SDKD log:137) "IterWait": 1,
[250.29 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@117.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.29 INFO] (SDKD log:137) "TimeRes": 1
[250.29 INFO] (SDKD log:137) }
[250.29 INFO] (SDKD log:137) }
[250.30 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[250.30 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.31 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125
[250.31 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.32 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.33 INFO] (SDKD log:137) "DS": {
[250.33 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.33 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.33 INFO] (SDKD log:137) "Count": 15000,
[250.34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23
[250.34 INFO] (SDKD log:137) "Continuous": true,
[250.35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@112.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.35 INFO] (SDKD log:137) "VSize": 128,
[250.35 INFO] (SDKD log:137) "Repeat": "REP",
[250.35 INFO] (SDKD log:137) "KSize": 32
[250.35 INFO] (SDKD log:137) },
[250.36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[250.36 INFO] (SDKD log:137) "Options": {
[250.36 INFO] (SDKD log:137) "DelayMax": 10,
[250.36 INFO] (SDKD log:137) "DelayMin": 1,
[250.37 INFO] (SDKD log:137) "IterWait": 1,
[250.37 INFO] (SDKD log:137) "TimeRes": 1
[250.37 INFO] (SDKD log:137) }
[250.37 INFO] (SDKD log:137) }
[250.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126
[250.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.38 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.39 INFO] (SDKD log:137) "DS": {
[250.39 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.40 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29
[250.40 INFO] (SDKD log:137) "Count": 15000,
[250.40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@118.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.41 INFO] (SDKD log:137) "Continuous": true,
[250.42 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[250.42 INFO] (SDKD log:137) "VSize": 128,
[250.43 INFO] (SDKD log:137) "Repeat": "REP",
[250.43 INFO] (SDKD log:137) "KSize": 32
[250.43 INFO] (SDKD log:137) },
[250.43 INFO] (SDKD log:137) "Options": {
[250.44 INFO] (SDKD log:137) "DelayMax": 10,
[250.44 INFO] (SDKD log:137) "DelayMin": 1,
[250.44 INFO] (SDKD log:137) "IterWait": 1,
[250.44 INFO] (SDKD log:137) "TimeRes": 1
[250.44 INFO] (SDKD log:137) }
[250.45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31
[250.45 INFO] (SDKD log:137) }
[250.45 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127
[250.45 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.46 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.46 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[250.46 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.47 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.47 INFO] (SDKD log:137) "DS": {
[250.47 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.48 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.48 INFO] (SDKD log:137) "Count": 15000,
[250.48 INFO] (SDKD log:137) "Continuous": true,
[250.48 INFO] (SDKD log:137) "VSize": 128,
[250.49 INFO] (SDKD log:137) "Repeat": "REP",
[250.49 INFO] (SDKD log:137) "KSize": 32
[250.49 INFO] (SDKD log:137) },
[250.49 INFO] (SDKD log:137) "Options": {
[250.50 INFO] (SDKD log:137) "DelayMax": 10,
[250.50 INFO] (SDKD log:137) "DelayMin": 1,
[250.50 INFO] (SDKD log:137) "IterWait": 1,
[250.50 INFO] (SDKD log:137) "TimeRes": 1
[250.51 INFO] (SDKD log:137) }
[250.51 INFO] (SDKD log:137) }
[250.51 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128
[250.51 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33
[250.52 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35
[250.56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.56 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.56 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.56 INFO] (SDKD log:137) "DS": {
[250.57 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.57 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.57 INFO] (SDKD log:137) "Count": 15000,
[250.58 INFO] (SDKD log:137) "Continuous": true,
[250.58 INFO] (SDKD log:137) "VSize": 128,
[250.58 INFO] (SDKD log:137) "Repeat": "REP",
[250.58 INFO] (SDKD log:137) "KSize": 32
[250.59 INFO] (SDKD log:137) },
[250.59 INFO] (SDKD log:137) "Options": {
[250.59 INFO] (SDKD log:137) "DelayMax": 10,
[250.59 INFO] (SDKD log:137) +++ Received exception of ID 0
[250.60 INFO] (SDKD log:137) "DelayMin": 1,
[250.60 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[250.60 INFO] (SDKD log:137) "IterWait": 1,
[250.60 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[250.60 INFO] (SDKD log:137) "TimeRes": 1
[250.60 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[250.60 INFO] (SDKD log:137) }
[250.61 INFO] (SDKD log:137) }
[250.61 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.61 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[250.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.62 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.62 INFO] (SDKD log:137) "DS": {
[250.63 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.63 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.63 INFO] (SDKD log:137) "Count": 15000,
[250.63 INFO] (SDKD log:137) "Continuous": true,
[250.63 INFO] (SDKD log:137) "VSize": 128,
[250.63 INFO] (SDKD log:137) "Repeat": "REP",
[250.63 INFO] (SDKD log:137) "KSize": 32
[250.63 INFO] (SDKD log:137) },
[250.64 INFO] (SDKD log:137) "Options": {
[250.64 INFO] (SDKD log:137) "DelayMax": 10,
[250.64 INFO] (SDKD log:137) "DelayMin": 1,
[250.64 INFO] (SDKD log:137) "IterWait": 1,
[250.64 INFO] (SDKD log:137) "TimeRes": 1
[250.65 INFO] (SDKD log:137) }
[250.65 INFO] (SDKD log:137) }
[250.65 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.65 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.66 INFO] (SDKD log:137) "DS": {
[250.66 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.66 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.66 INFO] (SDKD log:137) "Count": 15000,
[250.66 INFO] (SDKD log:137) "Continuous": true,
[250.66 INFO] (SDKD log:137) "VSize": 128,
[250.67 INFO] (SDKD log:137) "Repeat": "REP",
[250.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37
[250.67 INFO] (SDKD log:137) "KSize": 32
[250.68 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.68 INFO] (SDKD log:137) },
[250.68 INFO] (SDKD log:137) "Options": {
[250.68 INFO] (SDKD log:137) "DelayMax": 10,
[250.68 INFO] (SDKD log:137) "DelayMin": 1,
[250.69 INFO] (SDKD log:137) "IterWait": 1,
[250.69 INFO] (SDKD log:137) "TimeRes": 1
[250.69 INFO] (SDKD log:137) }
[250.70 INFO] (SDKD log:137) }
[250.70 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.70 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[250.71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.71 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.71 INFO] (SDKD log:137) "DS": {
[250.71 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39
[250.71 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.72 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.72 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.73 INFO] (SDKD log:137) "Count": 15000,
[250.73 INFO] (SDKD log:137) "Continuous": true,
[250.73 INFO] (SDKD log:137) "VSize": 128,
[250.73 INFO] (SDKD log:137) "Repeat": "REP",
[250.73 INFO] (SDKD log:137) "KSize": 32
[250.73 INFO] (SDKD log:137) },
[250.74 INFO] (SDKD log:137) "Options": {
[250.74 INFO] (SDKD log:137) "DelayMax": 10,
[250.74 INFO] (SDKD log:137) "DelayMin": 1,
[250.75 INFO] (SDKD log:137) "IterWait": 1,
[250.75 INFO] (SDKD log:137) "TimeRes": 1
[250.75 INFO] (SDKD log:137) }
[250.76 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41
[250.76 INFO] (SDKD log:137) }
[250.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.76 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.77 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.77 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.77 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.78 INFO] (SDKD log:137) "DS": {
[250.78 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.78 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.79 INFO] (SDKD log:137) "Count": 15000,
[250.79 INFO] (SDKD log:137) "Continuous": true,
[250.79 INFO] (SDKD log:137) "VSize": 128,
[250.79 INFO] (SDKD log:137) "Repeat": "REP",
[250.79 INFO] (SDKD log:137) "KSize": 32
[250.79 INFO] (SDKD log:137) },
[250.79 INFO] (SDKD log:137) "Options": {
[250.80 INFO] (SDKD log:137) "DelayMax": 10,
[250.80 INFO] (SDKD log:137) "DelayMin": 1,
[250.80 INFO] (SDKD log:137) "IterWait": 1,
[250.81 INFO] (SDKD log:137) "TimeRes": 1
[250.81 INFO] (SDKD log:137) }
[250.81 INFO] (SDKD log:137) }
[250.81 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43
[250.82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.83 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[250.84 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.84 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@126.45
[250.85 INFO] (SDKD log:137) "DS": {
[250.85 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.85 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@126.57 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.86 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.86 INFO] (SDKD log:137) "Count": 15000,
[250.87 INFO] (SDKD log:137) "Continuous": true,
[250.87 INFO] (SDKD log:137) "VSize": 128,
[250.87 INFO] (SDKD log:137) "Repeat": "REP",
[250.87 INFO] (SDKD log:137) "KSize": 32
[250.87 INFO] (SDKD log:137) },
[250.87 INFO] (SDKD log:137) "Options": {
[250.88 INFO] (SDKD log:137) "DelayMax": 10,
[250.88 INFO] (SDKD log:137) "DelayMin": 1,
[250.88 INFO] (SDKD log:137) "IterWait": 1,
[250.88 INFO] (SDKD log:137) "TimeRes": 1
[250.88 INFO] (SDKD log:137) }
[250.88 INFO] (SDKD log:137) }
[250.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.89 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.89 INFO] (SDKD log:137) "DS": {
[250.90 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.90 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.90 INFO] (SDKD log:137) "Count": 15000,
[250.90 INFO] (SDKD log:137) "Continuous": true,
[250.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@127.47
[250.91 INFO] (SDKD log:137) "VSize": 128,
[250.91 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@127.58 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[250.91 INFO] (SDKD log:137) "Repeat": "REP",
[250.91 INFO] (SDKD log:137) "KSize": 32
[250.92 INFO] (SDKD log:137) },
[250.92 INFO] (SDKD log:137) "Options": {
[250.92 INFO] (SDKD log:137) "DelayMax": 10,
[250.93 INFO] (SDKD log:137) "DelayMin": 1,
[250.93 INFO] (SDKD log:137) "IterWait": 1,
[250.93 INFO] (SDKD log:137) "TimeRes": 1
[250.93 INFO] (SDKD log:137) }
[250.93 INFO] (SDKD log:137) }
[250.93 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[250.94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.94 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.94 INFO] (SDKD log:137) "DS": {
[250.94 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.95 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.95 INFO] (SDKD log:137) "Count": 15000,
[250.96 INFO] (SDKD log:137) "Continuous": true,
[250.96 INFO] (SDKD log:137) "VSize": 128,
[250.96 INFO] (SDKD log:137) "Repeat": "REP",
[250.96 INFO] (SDKD log:137) "KSize": 32
[250.96 INFO] (SDKD log:137) },
[250.96 INFO] (SDKD log:137) "Options": {
[250.96 INFO] (SDKD log:137) "DelayMax": 10,
[250.96 INFO] (SDKD log:137) "DelayMin": 1,
[250.96 INFO] (SDKD log:137) "IterWait": 1,
[250.96 INFO] (SDKD log:137) "TimeRes": 1
[250.96 INFO] (SDKD log:137) }
[250.97 INFO] (SDKD log:137) }
[250.97 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[250.97 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[250.97 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[250.98 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[250.98 INFO] (SDKD log:137) "DS": {
[250.99 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[250.99 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[250.99 INFO] (SDKD log:137) "Count": 15000,
[250.99 INFO] (SDKD log:137) "Continuous": true,
[250.99 INFO] (SDKD log:137) "VSize": 128,
[250.99 INFO] (SDKD log:137) "Repeat": "REP",
[250.99 INFO] (SDKD log:137) "KSize": 32
[250.99 INFO] (SDKD log:137) },
[251.00 INFO] (SDKD log:137) "Options": {
[251.00 INFO] (SDKD log:137) "DelayMax": 10,
[251.00 INFO] (SDKD log:137) "DelayMin": 1,
[251.00 INFO] (SDKD log:137) "IterWait": 1,
[251.01 INFO] (SDKD log:137) "TimeRes": 1
[251.01 INFO] (SDKD log:137) }
[251.02 INFO] (SDKD log:137) }
[251.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@128.49
[251.06 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@128.59 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[251.07 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@129.60 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[251.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[251.09 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[251.09 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[251.09 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[251.09 INFO] (SDKD log:137) +++ Received exception of ID 0
[251.09 INFO] (SDKD log:137) "DS": {
[251.09 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[251.09 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[251.09 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[251.10 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[251.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[251.10 INFO] (SDKD log:137) "Count": 15000,
[251.10 INFO] (SDKD log:137) +++ Received exception of ID 0
[251.10 INFO] (SDKD log:137) "Continuous": true,
[251.11 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[251.11 INFO] (SDKD log:137) "VSize": 128,
[251.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[251.11 INFO] (SDKD log:137) "Repeat": "REP",
[251.12 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[251.13 INFO] (SDKD log:137) "KSize": 32
[251.13 INFO] (SDKD log:137) },
[251.13 INFO] (SDKD log:137) "Options": {
[251.13 INFO] (SDKD log:137) "DelayMax": 10,
[251.14 INFO] (SDKD log:137) "DelayMin": 1,
[251.14 INFO] (SDKD log:137) "IterWait": 1,
[251.14 INFO] (SDKD log:137) "TimeRes": 1
[251.15 INFO] (SDKD log:137) }
[251.15 INFO] (SDKD log:137) }
[251.15 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 129
[251.15 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[251.17 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@129.60
[251.18 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@129.61 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=
12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[251.19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[251.20 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[251.21 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[251.21 INFO] (SDKD log:137) "DS": {
[251.21 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[251.21 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[251.21 INFO] (SDKD log:137) "Count": 1000,
[251.22 INFO] (SDKD log:137) "Continuous": false,
[251.22 INFO] (SDKD log:137) "VSize": 512,
[251.23 INFO] (SDKD log:137) "Repeat": "rep",
[251.23 INFO] (SDKD log:137) "KSize": 12
[251.23 INFO] (SDKD log:137) },
[251.23 INFO] (SDKD log:137) "Schema": {
[251.24 INFO] (SDKD log:137) "InflateContent": "meh",
[251.24 INFO] (SDKD log:137) "InflateLevel": 40
[251.24 INFO] (SDKD log:137) },
[251.25 INFO] (SDKD log:137) "Options": {
[251.25 INFO] (SDKD log:137) "DelayMax": 0,
[251.25 INFO] (SDKD log:137) "DelayMin": 0,
[251.26 INFO] (SDKD log:137) "IterWait": 1,
[251.26 INFO] (SDKD log:137) "TimeRes": 0
[251.27 INFO] (SDKD log:137) }
[251.27 INFO] (SDKD log:137) }
[544.61 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[544.61 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@129.61
[544.61 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@129.62
[544.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[544.62 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@130.63 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[544.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[544.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[544.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[544.63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[544.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[544.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 130
[544.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[544.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@130.63
[544.67 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@130.64 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[544.67 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[544.68 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@131.65 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[544.68 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[544.68 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[544.68 INFO] (SDKD log:137) "DS": {},
[544.68 INFO] (SDKD log:137) "ViewParameters": {
[544.69 INFO] (SDKD log:137) "limit": 0
[544.69 INFO] (SDKD log:137) },
[544.69 INFO] (SDKD log:137) "DesignName": "test_design",
[544.69 INFO] (SDKD log:137) "Options": {
[544.69 INFO] (SDKD log:137) "DelayMax": 10,
[544.70 INFO] (SDKD log:137) "DelayMin": 1,
[544.70 INFO] (SDKD log:137) "IterWait": 1,
[544.70 INFO] (SDKD log:137) "ViewQueryCount": -1,
[544.70 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[544.70 INFO] (SDKD log:137) "TimeRes": 1
[544.70 INFO] (SDKD log:137) },
[544.70 INFO] (SDKD log:137) "ViewName": "test_view"
[544.71 INFO] (SDKD log:137) }
[544.71 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[544.71 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 131
[544.71 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[544.73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@131.65
[544.73 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@131.66 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[544.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[544.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@132.67 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[544.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[544.74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[544.74 INFO] (SDKD log:137) "DS": {},
[544.74 INFO] (SDKD log:137) "ViewParameters": {
[544.74 INFO] (SDKD log:137) "limit": 0
[544.75 INFO] (SDKD log:137) },
[544.75 INFO] (SDKD log:137) "DesignName": "test_design",
[544.75 INFO] (SDKD log:137) "Options": {
[544.75 INFO] (SDKD log:137) "DelayMax": 10,
[544.75 INFO] (SDKD log:137) "DelayMin": 1,
[544.76 INFO] (SDKD log:137) "IterWait": 1,
[544.76 INFO] (SDKD log:137) "ViewQueryCount": -1,
[544.76 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[544.76 INFO] (SDKD log:137) "TimeRes": 1
[544.76 INFO] (SDKD log:137) },
[544.77 INFO] (SDKD log:137) "ViewName": "test_view"
[544.77 INFO] (SDKD log:137) +++ Received exception of ID 0
[544.77 INFO] (SDKD log:137) }
[544.78 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[544.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[544.78 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[544.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 132
[544.78 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[544.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[544.81 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@132.67
[544.82 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@132.68 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[544.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[544.83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@133.69 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[544.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[544.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[544.84 INFO] (SDKD log:137) "DS": {},
[544.84 INFO] (SDKD log:137) "ViewParameters": {
[544.85 INFO] (SDKD log:137) "limit": 0
[544.85 INFO] (SDKD log:137) },
[544.85 INFO] (SDKD log:137) "DesignName": "test_design",
[544.85 INFO] (SDKD log:137) "Options": {
[544.86 INFO] (SDKD log:137) "DelayMax": 10,
[544.86 INFO] (SDKD log:137) "DelayMin": 1,
[544.86 INFO] (SDKD log:137) "IterWait": 1,
[544.86 INFO] (SDKD log:137) "ViewQueryCount": -1,
[544.86 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[544.86 INFO] (SDKD log:137) +++ Received exception of ID 0
[544.87 INFO] (SDKD log:137) "TimeRes": 1
[544.87 INFO] (SDKD log:137) },
[544.88 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[544.88 INFO] (SDKD log:137) "ViewName": "test_view"
[544.88 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[544.88 INFO] (SDKD log:137) }
[544.88 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[544.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[544.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 133
[544.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[544.90 INFO] (SDKD log:137) +++ Received exception of ID 0
[544.91 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[544.92 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[544.92 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[544.93 INFO] (SDKD log:137) +++ Received exception of ID 0
[544.94 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[544.94 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[544.94 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[544.96 INFO] (SDKD log:137) +++ Received exception of ID 0
[544.97 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[544.97 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[544.97 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[544.97 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.00 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.00 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.02 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@133.69
[545.02 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[545.02 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@133.70 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[545.02 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[545.03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@134.71 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[545.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[545.03 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[545.03 INFO] (SDKD log:137) "DS": {},
[545.03 INFO] (SDKD log:137) "ViewParameters": {
[545.03 INFO] (SDKD log:137) "limit": 0
[545.03 INFO] (SDKD log:137) },
[545.04 INFO] (SDKD log:137) "DesignName": "test_design",
[545.04 INFO] (SDKD log:137) "Options": {
[545.04 INFO] (SDKD log:137) "DelayMax": 10,
[545.04 INFO] (SDKD log:137) "DelayMin": 1,
[545.04 INFO] (SDKD log:137) "IterWait": 1,
[545.04 INFO] (SDKD log:137) "ViewQueryCount": -1,
[545.05 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[545.05 INFO] (SDKD log:137) "TimeRes": 1
[545.05 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.05 INFO] (SDKD log:137) },
[545.05 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.06 INFO] (SDKD log:137) "ViewName": "test_view"
[545.06 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.06 INFO] (SDKD log:137) }
[545.06 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[545.06 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[545.06 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.07 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 134
[545.07 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.07 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[545.07 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.08 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[545.08 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.08 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.08 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.09 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[545.09 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.10 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[545.10 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.11 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[545.13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@134.71
[545.13 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@134.72 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[545.14 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds
[545.14 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[545.15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[545.15 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[545.15 INFO] (SDKD log:137) "DS": {},
[545.15 INFO] (SDKD log:137) "ViewParameters": {
[545.15 INFO] (SDKD log:137) "limit": 0
[545.15 INFO] (SDKD log:137) },
[545.16 INFO] (SDKD log:137) "DesignName": "test_design",
[545.16 INFO] (SDKD log:137) "Options": {
[545.16 INFO] (SDKD log:137) "DelayMax": 10,
[545.16 INFO] (SDKD log:137) "DelayMin": 1,
[545.16 INFO] (SDKD log:137) "IterWait": 1,
[545.16 INFO] (SDKD log:137) "ViewQueryCount": -1,
[545.17 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[545.17 INFO] (SDKD log:137) "TimeRes": 1
[545.17 INFO] (SDKD log:137) },
[545.17 INFO] (SDKD log:137) "ViewName": "test_view"
[545.18 INFO] (SDKD log:137) +++ Received exception of ID 0
[545.18 INFO] (SDKD log:137) }
[545.18 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[545.18 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[545.18 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[575.14 INFO] (Scenario run:86) Starting CHANGE phase
[575.20 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[576.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[577.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[579.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[581.33 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[583.27 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 1%
[586.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2%
[587.51 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2%
[588.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3%
[589.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3%
[590.57 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3%
[591.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4%
[592.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5%
[593.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5%
[594.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 6%
[595.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7%
[596.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7%
[597.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 8%
[598.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9%
[599.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9%
[600.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10%
[601.98 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10%
[603.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11%
[604.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11%
[605.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12%
[606.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12%
[607.16 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13%
[608.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13%
[609.28 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14%
[610.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14%
[611.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15%
[612.39 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15%
[613.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16%
[614.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16%
[615.43 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[616.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18%
[617.49 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18%
[618.50 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19%
[619.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19%
[620.53 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20%
[621.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20%
[622.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21%
[623.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21%
[624.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22%
[625.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22%
[626.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 23%
[627.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 24%
[628.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 24%
[629.79 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25%
[630.80 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25%
[631.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26%
[632.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26%
[633.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26%
[634.87 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 27%
[635.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 27%
[636.90 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28%
[637.91 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28%
[638.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29%
[639.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29%
[640.95 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 30%
[641.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31%
[643.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31%
[643.63 INFO] (SDKD log:137) +++ Received exception of ID 0
[643.63 INFO] (SDKD log:137)
[643.66 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
[644.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32%
[645.04 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32%
[646.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32%
[647.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33%
[648.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33%
[649.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 34%
[650.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35%
[651.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35%
[652.18 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36%
[653.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36%
[654.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37%
[655.23 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37%
[656.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 38%
[657.25 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39%
[658.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40%
[659.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40%
[660.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40%
[661.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 41%
[662.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42%
[663.37 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42%
[664.38 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43%
[665.40 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43%
[666.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 44%
[667.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45%
[668.43 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45%
[669.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 46%
[670.47 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 47%
[671.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 47%
[672.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48%
[673.54 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48%
[674.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49%
[675.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49%
[676.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50%
[677.59 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50%
[678.60 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51%
[679.60 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51%
[680.61 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51%
[681.62 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 52%
[682.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53%
[683.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53%
[684.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 54%
[685.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 54%
[686.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55%
[687.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55%
[688.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56%
[689.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56%
[690.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56%
[691.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57%
[692.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57%
[693.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58%
[694.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58%
[695.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59%
[696.70 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[696.71 INFO] (SDKD log:137) at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Collections.Generic.List`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line
 47
   at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.UpdateConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Co
nfiguration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 91
   at Couchbase.Core.ClusterManager.NotifyConfigPublished(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 137
   at Couchbase.Core.Buckets.CouchbaseBucket.CheckForConfigUpdates[T](IOperationResult`1 operationResult) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.c
s:line 234
   at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couc
hbase\Core\Buckets\CouchbaseBucket.cs:line 166
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[696.74 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
        at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
[696.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59%
[696.76 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73
[696.76 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]

c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>

 Comments   
Comment by Jeff Morris [ 18/Aug/14 ]
This is valid bug. Threads 26 and 48 have both received a new config and updating the config context at the same time:

Line 1398139: 2014-08-16 08:55:03,007 [26] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#2061 on thread 26
Line 1398143: 2014-08-16 08:55:03,013 [48] INFO Couchbase.IO.Operations.OperationResult`1 - Received config rev#2061
Line 1398145: 2014-08-16 08:55:03,014 [48] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#2061 on thread 48
Line 1398175: 2014-08-16 08:55:03,032 [48] INFO Couchbase.Configuration.Server.Providers.ConfigProviderBase - New config has changed new Rev#2062 | old Rev#2061 CCCP: {"name":"default","bucketType":null,"authType":null"...
Line 1398367: 2014-08-16 08:55:56,152 [26] INFO Couchbase.Configuration.Server.Providers.ConfigProviderBase - New config has changed new Rev#2062 | old Rev#2061 CCCP:

Since they are both updating the same List<T>, the enumeration is failing (you cannot modify a collection once an enumeration has begin). This should be relatively easy to fix with some thread sync techniques.
Comment by Jeff Morris [ 19/Aug/14 ]
Patch:

git fetch ssh://jmorris@review.couchbase.org:29418/couchbase-net-client refs/changes/15/40715/1 && git checkout FETCH_HEAD
Comment by Andrei Baranouski [ 21/Aug/14 ]
fixed




[NCBC-588] System.InvalidOperationException: Sequence contains no matching element in Couchbase.Configuration.ConfigContextBase.GetServer() after rebalance Created: 16/Aug/14  Updated: 27/Aug/14  Resolved: 27/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Andrei Baranouski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive NCBC-588.zip    
Issue Links:
Gantt: finish-start

 Description   
I took a few commits ahead from gerrit


commit 2c00a9d3a397f5c1eddc56238cf4a19e6e6618dd
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 11 18:41:08 2014 -0700

    Add config files to tests

    Change-Id: Ie093eac1da4ffc742796ffd4df8a9c63cc1c9c2f

commit 5b18623a7c13fb16f21476fe77eaf9a192e92eb2
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 11 17:20:19 2014 -0700

    Complete IBucket implementation for Memcached and Couchbase Buckets

    Change-Id: If6fb47b9e4a5201e625f155dadf45696b8666674

commit a6001260b0437a9779f7b7730e857b77ac9aea31
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 15:31:06 2014 -0700

    NCBC-542: Provide Observe method on IBucket

    Change-Id: Iecfbc02490b9ce237d6fbe16619e5aa252566bd7

commit 321940886cffcdc30225201ea243d8c6e4df6da4
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 12:31:12 2014 -0700

    NCBC-576: Do not retry operation when unknown status is returned.

    Change-Id: Ic9afeb1373d9e7804cad3c2df0521b162956326d
    Reviewed-on: http://review.couchbase.org/40347
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Matt Ingenthron <matt@couchbase.com>


c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0.31 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/mode=out
    workload=HybridWorkloadGroup
    rebalance/count=2
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-16-14\052918\efcd0dc3177c5d7533647e7a3fe8ee7a

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-mode out --workload HybridWorkloadGroup
 \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1.54 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1.97 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1.99 INFO] (RunContext run:124) Starting cluster and driver
[2.00 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[2.00 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[2.00 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[2.31 INFO] (SDKD log:137) --- Logging Self-Test ---
[2.33 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[2.34 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[2.34 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[2.34 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[2.35 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2.53 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2.55 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2.83 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}}
[5.06 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[5.16 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa
[5.17 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[5.17 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[5.17 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa
[5.75 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[5.76 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[5.77 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[5.79 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[5.79 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[5.80 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[5.80 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[5.80 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[6.01 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@6b96bac4
[6.02 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@3dde8aa8
[6.02 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@43a4181c
[6.02 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@56b61c3
[6.04 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@1de4eb5b
[17.65 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[28.31 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[35.77 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[51.97 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[52.01 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[53.07 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[53.08 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[53.09 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[53.11 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[61.02 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[61.37 INFO] (RunContext run:143) Driver and cluster initialized
[61.40 INFO] (RunContext call:167) Running scenario..
[61.41 INFO] (Scenario run:72) Starting RAMP phase
[61.41 INFO] (Workload setupDesign:63) Creating design test_design
[61.74 INFO] (Workload setupDesign:80) Design creation done
[61.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[61.75 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[61.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[61.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[61.79 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[61.79 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[61.82 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[61.82 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[62.22 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[62.23 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12
}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[62.24 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[62.24 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[62.24 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[62.25 INFO] (SDKD log:137) "DS": {
[62.25 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[62.25 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[62.26 INFO] (SDKD log:137) "Count": 1000,
[62.26 INFO] (SDKD log:137) "Continuous": false,
[62.26 INFO] (SDKD log:137) "VSize": 512,
[62.26 INFO] (SDKD log:137) "Repeat": "rep",
[62.27 INFO] (SDKD log:137) "KSize": 12
[62.27 INFO] (SDKD log:137) },
[62.27 INFO] (SDKD log:137) "Schema": {
[62.27 INFO] (SDKD log:137) "InflateContent": "meh",
[62.28 INFO] (SDKD log:137) "InflateLevel": 40
[62.28 INFO] (SDKD log:137) },
[62.28 INFO] (SDKD log:137) "Options": {
[62.28 INFO] (SDKD log:137) "DelayMax": 0,
[62.29 INFO] (SDKD log:137) "DelayMin": 0,
[62.29 INFO] (SDKD log:137) "IterWait": 1,
[62.29 INFO] (SDKD log:137) "TimeRes": 0
[62.29 INFO] (SDKD log:137) }
[62.30 INFO] (SDKD log:137) }
[64.44 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[64.44 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[64.45 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[64.45 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[64.46 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[64.46 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[64.46 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[64.46 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[64.47 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[64.47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[64.47 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[64.48 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[64.49 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[64.49 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12
}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[64.50 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[64.50 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[64.51 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[64.51 INFO] (SDKD log:137) "DS": {
[64.51 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[64.52 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[64.52 INFO] (SDKD log:137) "Count": 1000,
[64.52 INFO] (SDKD log:137) "Continuous": false,
[64.52 INFO] (SDKD log:137) "VSize": 512,
[64.53 INFO] (SDKD log:137) "Repeat": "rep",
[64.53 INFO] (SDKD log:137) "KSize": 12
[64.53 INFO] (SDKD log:137) },
[64.54 INFO] (SDKD log:137) "Schema": {
[64.54 INFO] (SDKD log:137) "InflateContent": "meh",
[64.54 INFO] (SDKD log:137) "InflateLevel": 40
[64.54 INFO] (SDKD log:137) },
[64.55 INFO] (SDKD log:137) "Options": {
[64.55 INFO] (SDKD log:137) "DelayMax": 0,
[64.55 INFO] (SDKD log:137) "DelayMin": 0,
[64.56 INFO] (SDKD log:137) "IterWait": 1,
[64.56 INFO] (SDKD log:137) "TimeRes": 0
[64.56 INFO] (SDKD log:137) }
[64.57 INFO] (SDKD log:137) }
[67.50 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[67.50 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@102.5
[67.51 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[67.51 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[67.51 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[67.52 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[67.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[67.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[67.52 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[67.53 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[67.53 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[67.53 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[67.55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[67.55 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCou
nt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[67.56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[67.56 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3
.4.147",8091]]}, Hostname=10.3.4.144}
[67.57 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[67.57 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[67.57 INFO] (SDKD log:137) "DS": {},
[67.58 INFO] (SDKD log:137) "ViewParameters": {},
[67.58 INFO] (SDKD log:137) "DesignName": "test_design",
[67.58 INFO] (SDKD log:137) "Options": {
[67.58 INFO] (SDKD log:137) "DelayMax": 10,
[67.59 INFO] (SDKD log:137) "DelayMin": 1,
[67.59 INFO] (SDKD log:137) "IterWait": 1,
[67.60 INFO] (SDKD log:137) "ViewQueryCount": -1,
[67.60 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9
[67.60 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[67.61 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[67.61 INFO] (SDKD log:137) "TimeRes": 1
[67.61 INFO] (SDKD log:137) },
[67.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[67.62 INFO] (SDKD log:137) "ViewName": "test_view"
[67.62 INFO] (SDKD log:137) }
[67.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[67.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[67.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[67.64 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[67.64 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[67.64 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[67.64 INFO] (SDKD log:137) "DS": {},
[67.65 INFO] (SDKD log:137) "ViewParameters": {},
[67.65 INFO] (SDKD log:137) "DesignName": "test_design",
[67.65 INFO] (SDKD log:137) "Options": {
[67.66 INFO] (SDKD log:137) "DelayMax": 10,
[67.66 INFO] (SDKD log:137) "DelayMin": 1,
[67.66 INFO] (SDKD log:137) "IterWait": 1,
[67.67 INFO] (SDKD log:137) "ViewQueryCount": -1,
[67.67 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[67.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11
[67.67 INFO] (SDKD log:137) "TimeRes": 1
[67.68 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[67.68 INFO] (SDKD log:137) },
[67.69 INFO] (SDKD log:137) "ViewName": "test_view"
[67.69 INFO] (SDKD log:137) }
[67.69 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[67.70 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[67.70 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[67.70 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[67.71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[67.71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[67.72 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[67.72 INFO] (SDKD log:137) "DS": {},
[67.72 INFO] (SDKD log:137) "ViewParameters": {},
[67.73 INFO] (SDKD log:137) "DesignName": "test_design",
[67.73 INFO] (SDKD log:137) "Options": {
[67.74 INFO] (SDKD log:137) "DelayMax": 10,
[67.74 INFO] (SDKD log:137) "DelayMin": 1,
[67.74 INFO] (SDKD log:137) "IterWait": 1,
[67.75 INFO] (SDKD log:137) "ViewQueryCount": -1,
[67.75 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[67.75 INFO] (SDKD log:137) "TimeRes": 1
[67.76 INFO] (SDKD log:137) },
[67.76 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13
[67.76 INFO] (SDKD log:137) "ViewName": "test_view"
[67.77 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[67.78 INFO] (SDKD log:137) }
[67.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[67.78 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[67.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[67.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[67.79 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[67.80 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[67.80 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[67.80 INFO] (SDKD log:137) "DS": {},
[67.81 INFO] (SDKD log:137) "ViewParameters": {},
[67.81 INFO] (SDKD log:137) "DesignName": "test_design",
[67.81 INFO] (SDKD log:137) "Options": {
[67.82 INFO] (SDKD log:137) "DelayMax": 10,
[67.82 INFO] (SDKD log:137) "DelayMin": 1,
[67.82 INFO] (SDKD log:137) "IterWait": 1,
[67.82 INFO] (SDKD log:137) "ViewQueryCount": -1,
[67.83 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15
[67.83 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[67.83 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo
unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[67.84 INFO] (SDKD log:137) "TimeRes": 1
[67.84 INFO] (SDKD log:137) },
[67.84 INFO] (SDKD log:137) "ViewName": "test_view"
[67.84 INFO] (Workload call:112) Preloading items into cluster before workload
[67.85 INFO] (SDKD log:137) }
[67.85 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.
3.4.147",8091]]}, Hostname=10.3.4.144}
[67.85 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[67.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[67.86 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[67.86 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[67.86 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[67.87 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[67.87 INFO] (SDKD log:137) "DS": {},
[67.87 INFO] (SDKD log:137) "ViewParameters": {},
[67.88 INFO] (SDKD log:137) "DesignName": "test_design",
[67.88 INFO] (SDKD log:137) "Options": {
[67.88 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17
[67.88 INFO] (SDKD log:137) "DelayMax": 10,
[67.89 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}}
[67.89 INFO] (SDKD log:137) "DelayMin": 1,
[67.89 INFO] (SDKD log:137) "IterWait": 1,
[67.90 INFO] (SDKD log:137) "ViewQueryCount": -1,
[67.91 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[67.91 INFO] (SDKD log:137) "TimeRes": 1
[67.91 INFO] (SDKD log:137) },
[67.92 INFO] (SDKD log:137) "ViewName": "test_view"
[67.92 INFO] (SDKD log:137) }
[67.92 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[67.93 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[67.93 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[67.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[67.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[67.94 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[67.94 INFO] (SDKD log:137) "DS": {
[67.94 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[67.94 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[67.95 INFO] (SDKD log:137) "Count": 15000,
[67.95 INFO] (SDKD log:137) "Continuous": false,
[67.95 INFO] (SDKD log:137) "VSize": 128,
[67.96 INFO] (SDKD log:137) "Repeat": "REP",
[67.96 INFO] (SDKD log:137) "KSize": 32
[67.96 INFO] (SDKD log:137) },
[67.96 INFO] (SDKD log:137) "Options": {
[67.97 INFO] (SDKD log:137) "DelayMax": 0,
[67.97 INFO] (SDKD log:137) "DelayMin": 0,
[67.97 INFO] (SDKD log:137) "IterWait": 100,
[67.98 INFO] (SDKD log:137) "TimeRes": 0
[67.98 INFO] (SDKD log:137) }
[67.98 INFO] (SDKD log:137) }
[270.59 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 4231, [MEMD:MEMD_EVBUCKET]: 10769}
[270.59 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19
[270.59 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[270.60 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[270.60 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.60 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.60 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.60 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.61 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[270.61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[270.63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[270.64 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.64 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.64 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110
[270.64 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114
[270.64 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115
[270.64 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113
[270.65 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109
[270.65 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112
[270.65 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111
[270.65 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.65 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.66 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116
[270.66 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.66 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117
[270.66 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118
[270.67 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21
[270.67 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@110.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.68 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.68 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[270.69 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[270.69 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[270.69 INFO] (SDKD log:137) "DS": {
[270.69 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[270.69 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[270.69 INFO] (SDKD log:137) "Count": 15000,
[270.69 INFO] (SDKD log:137) "Continuous": true,
[270.70 INFO] (SDKD log:137) "VSize": 128,
[270.70 INFO] (SDKD log:137) "Repeat": "REP",
[270.70 INFO] (SDKD log:137) "KSize": 32
[270.70 INFO] (SDKD log:137) },
[270.70 INFO] (SDKD log:137) "Options": {
[270.70 INFO] (SDKD log:137) "DelayMax": 10,
[270.71 INFO] (SDKD log:137) "DelayMin": 1,
[270.71 INFO] (SDKD log:137) "IterWait": 1,
[270.71 INFO] (SDKD log:137) "TimeRes": 1
[270.71 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25
[270.71 INFO] (SDKD log:137) }
[270.72 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@114.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.72 INFO] (SDKD log:137) }
[270.72 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.72 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119
[270.73 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.73 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[270.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[270.73 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[270.73 INFO] (SDKD log:137) "DS": {
[270.73 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[270.74 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[270.74 INFO] (SDKD log:137) "Count": 15000,
[270.74 INFO] (SDKD log:137) "Continuous": true,
[270.74 INFO] (SDKD log:137) "VSize": 128,
[270.75 INFO] (SDKD log:137) "Repeat": "REP",
[270.75 INFO] (SDKD log:137) "KSize": 32
[270.75 INFO] (SDKD log:137) },
[270.75 INFO] (SDKD log:137) "Options": {
[270.75 INFO] (SDKD log:137) "DelayMax": 10,
[270.76 INFO] (SDKD log:137) "DelayMin": 1,
[270.76 INFO] (SDKD log:137) "IterWait": 1,
[270.76 INFO] (SDKD log:137) "TimeRes": 1
[270.76 INFO] (SDKD log:137) }
[270.76 INFO] (SDKD log:137) }
[270.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120
[270.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26
[270.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.78 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@115.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[270.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[270.79 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[270.79 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.79 INFO] (SDKD log:137) "DS": {
[270.79 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[270.80 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22
[270.80 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[270.80 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@111.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.80 INFO] (SDKD log:137) "Count": 15000,
[270.80 INFO] (SDKD log:137) "Continuous": true,
[270.81 INFO] (SDKD log:137) "VSize": 128,
[270.81 INFO] (SDKD log:137) "Repeat": "REP",
[270.81 INFO] (SDKD log:137) "KSize": 32
[270.81 INFO] (SDKD log:137) },
[270.81 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.82 INFO] (SDKD log:137) "Options": {
[270.82 INFO] (SDKD log:137) "DelayMax": 10,
[270.82 INFO] (SDKD log:137) "DelayMin": 1,
[270.82 INFO] (SDKD log:137) "IterWait": 1,
[270.82 INFO] (SDKD log:137) "TimeRes": 1
[270.83 INFO] (SDKD log:137) }
[270.83 INFO] (SDKD log:137) }
[270.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.83 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121
[270.84 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[270.84 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[270.84 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[270.84 INFO] (SDKD log:137) "DS": {
[270.84 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[270.85 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[270.85 INFO] (SDKD log:137) "Count": 15000,
[270.85 INFO] (SDKD log:137) "Continuous": true,
[270.85 INFO] (SDKD log:137) "VSize": 128,
[270.85 INFO] (SDKD log:137) "Repeat": "REP",
[270.86 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23
[270.86 INFO] (SDKD log:137) "KSize": 32
[270.86 INFO] (SDKD log:137) },
[270.86 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@112.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.86 INFO] (SDKD log:137) "Options": {
[270.87 INFO] (SDKD log:137) "DelayMax": 10,
[270.87 INFO] (SDKD log:137) "DelayMin": 1,
[270.87 INFO] (SDKD log:137) "IterWait": 1,
[270.87 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.87 INFO] (SDKD log:137) "TimeRes": 1
[270.88 INFO] (SDKD log:137) }
[270.88 INFO] (SDKD log:137) }
[270.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122
[270.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[270.90 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20
[270.90 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[270.90 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@109.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.90 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[270.91 INFO] (SDKD log:137) "DS": {
[270.91 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.92 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[270.92 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[270.92 INFO] (SDKD log:137) "Count": 15000,
[270.92 INFO] (SDKD log:137) "Continuous": true,
[270.92 INFO] (SDKD log:137) "VSize": 128,
[270.92 INFO] (SDKD log:137) "Repeat": "REP",
[270.93 INFO] (SDKD log:137) "KSize": 32
[270.93 INFO] (SDKD log:137) },
[270.93 INFO] (SDKD log:137) "Options": {
[270.93 INFO] (SDKD log:137) "DelayMax": 10,
[270.93 INFO] (SDKD log:137) "DelayMin": 1,
[270.94 INFO] (SDKD log:137) "IterWait": 1,
[270.94 INFO] (SDKD log:137) "TimeRes": 1
[270.94 INFO] (SDKD log:137) }
[270.94 INFO] (SDKD log:137) }
[270.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123
[270.94 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[270.95 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[270.95 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29
[270.95 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[270.96 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@118.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[270.96 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[270.96 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[270.97 INFO] (SDKD log:137) "DS": {
[270.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[270.97 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[270.98 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[270.98 INFO] (SDKD log:137) "Count": 15000,
[270.99 INFO] (SDKD log:137) "Continuous": true,
[270.99 INFO] (SDKD log:137) "VSize": 128,
[270.99 INFO] (SDKD log:137) "Repeat": "REP",
[271.00 INFO] (SDKD log:137) "KSize": 32
[271.00 INFO] (SDKD log:137) },
[271.00 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.01 INFO] (SDKD log:137) "Options": {
[271.01 INFO] (SDKD log:137) "DelayMax": 10,
[271.01 INFO] (SDKD log:137) "DelayMin": 1,
[271.03 INFO] (SDKD log:137) "IterWait": 1,
[271.03 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.04 INFO] (SDKD log:137) "TimeRes": 1
[271.04 INFO] (SDKD log:137) }
[271.04 INFO] (SDKD log:137) }
[271.04 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124
[271.05 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.05 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.05 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[271.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[271.06 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.06 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.06 INFO] (SDKD log:137) "DS": {
[271.06 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.06 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.07 INFO] (SDKD log:137) "Count": 15000,
[271.07 INFO] (SDKD log:137) "Continuous": true,
[271.07 INFO] (SDKD log:137) "VSize": 128,
[271.07 INFO] (SDKD log:137) "Repeat": "REP",
[271.07 INFO] (SDKD log:137) "KSize": 32
[271.07 INFO] (SDKD log:137) },
[271.08 INFO] (SDKD log:137) "Options": {
[271.08 INFO] (SDKD log:137) "DelayMax": 10,
[271.08 INFO] (SDKD log:137) "DelayMin": 1,
[271.08 INFO] (SDKD log:137) "IterWait": 1,
[271.08 INFO] (SDKD log:137) "TimeRes": 1
[271.09 INFO] (SDKD log:137) }
[271.09 INFO] (SDKD log:137) }
[271.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125
[271.09 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[271.10 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28
[271.10 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.10 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@117.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.10 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[271.11 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.11 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.11 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[271.11 INFO] (SDKD log:137) "DS": {
[271.11 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.11 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.12 INFO] (SDKD log:137) "Count": 15000,
[271.13 INFO] (SDKD log:137) "Continuous": true,
[271.13 INFO] (SDKD log:137) "VSize": 128,
[271.14 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24
[271.14 INFO] (SDKD log:137) "Repeat": "REP",
[271.14 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.14 INFO] (SDKD log:137) "KSize": 32
[271.14 INFO] (SDKD log:137) },
[271.15 INFO] (SDKD log:137) "Options": {
[271.15 INFO] (SDKD log:137) "DelayMax": 10,
[271.15 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[271.15 INFO] (SDKD log:137) "DelayMin": 1,
[271.15 INFO] (SDKD log:137) "IterWait": 1,
[271.15 INFO] (SDKD log:137) "TimeRes": 1
[271.15 INFO] (SDKD log:137) }
[271.16 INFO] (SDKD log:137) }
[271.16 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[271.16 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126
[271.17 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.17 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[271.17 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.17 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.17 INFO] (SDKD log:137) "DS": {
[271.17 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.17 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.17 INFO] (SDKD log:137) "Count": 15000,
[271.18 INFO] (SDKD log:137) "Continuous": true,
[271.18 INFO] (SDKD log:137) "VSize": 128,
[271.18 INFO] (SDKD log:137) "Repeat": "REP",
[271.18 INFO] (SDKD log:137) "KSize": 32
[271.18 INFO] (SDKD log:137) },
[271.18 INFO] (SDKD log:137) "Options": {
[271.19 INFO] (SDKD log:137) "DelayMax": 10,
[271.19 INFO] (SDKD log:137) "DelayMin": 1,
[271.19 INFO] (SDKD log:137) "IterWait": 1,
[271.19 INFO] (SDKD log:137) "TimeRes": 1
[271.19 INFO] (SDKD log:137) }
[271.19 INFO] (SDKD log:137) }
[271.20 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[271.20 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127
[271.20 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.20 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27
[271.20 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@116.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.21 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[271.22 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[271.22 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.22 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.23 INFO] (SDKD log:137) "DS": {
[271.23 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.23 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.23 INFO] (SDKD log:137) "Count": 15000,
[271.23 INFO] (SDKD log:137) "Continuous": true,
[271.23 INFO] (SDKD log:137) "VSize": 128,
[271.23 INFO] (SDKD log:137) "Repeat": "REP",
[271.23 INFO] (SDKD log:137) "KSize": 32
[271.24 INFO] (SDKD log:137) },
[271.24 INFO] (SDKD log:137) "Options": {
[271.24 INFO] (SDKD log:137) "DelayMax": 10,
[271.24 INFO] (SDKD log:137) "DelayMin": 1,
[271.24 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31
[271.25 INFO] (SDKD log:137) "IterWait": 1,
[271.25 INFO] (SDKD log:137) "TimeRes": 1
[271.25 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.25 INFO] (SDKD log:137) }
[271.25 INFO] (SDKD log:137) }
[271.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[271.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128
[271.26 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.26 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[271.26 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.26 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.27 INFO] (SDKD log:137) "DS": {
[271.27 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.27 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.27 INFO] (SDKD log:137) "Count": 15000,
[271.27 INFO] (SDKD log:137) "Continuous": true,
[271.27 INFO] (SDKD log:137) "VSize": 128,
[271.27 INFO] (SDKD log:137) "Repeat": "REP",
[271.28 INFO] (SDKD log:137) "KSize": 32
[271.28 INFO] (SDKD log:137) },
[271.28 INFO] (SDKD log:137) "Options": {
[271.28 INFO] (SDKD log:137) "DelayMax": 10,
[271.29 INFO] (SDKD log:137) "DelayMin": 1,
[271.29 INFO] (SDKD log:137) "IterWait": 1,
[271.29 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33
[271.29 INFO] (SDKD log:137) "TimeRes": 1
[271.29 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.29 INFO] (SDKD log:137) }
[271.29 INFO] (SDKD log:137) }
[271.30 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.30 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[271.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.31 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.31 INFO] (SDKD log:137) "DS": {
[271.31 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.31 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1
[271.31 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.31 INFO] (SDKD log:137) at System.Net.Sockets.Socket.get_RemoteEndPoint()
   at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line
38
   at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString()
   at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer)
   at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer)
[271.32 INFO] (SDKD log:137) "Count": 15000,
[271.32 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.32 INFO] (SDKD log:137) "Continuous": true,
[271.33 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.33 INFO] (SDKD log:137) "VSize": 128,
[271.33 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.33 INFO] (SDKD log:137) "Repeat": "REP",
[271.33 INFO] (SDKD log:137) "KSize": 32
[271.33 INFO] (SDKD log:137) },
[271.34 INFO] (SDKD log:137) "Options": {
[271.34 INFO] (SDKD log:137) "DelayMax": 10,
[271.34 INFO] (SDKD log:137) "DelayMin": 1,
[271.34 INFO] (SDKD log:137) "IterWait": 1,
[271.34 INFO] (SDKD log:137) "TimeRes": 1
[271.34 INFO] (SDKD log:137) }
[271.34 INFO] (SDKD log:137) }
[271.35 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.35 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.35 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.36 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.38 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35
[271.39 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.39 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[271.39 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.39 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.39 INFO] (SDKD log:137) "DS": {
[271.40 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.40 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.40 INFO] (SDKD log:137) "Count": 15000,
[271.40 INFO] (SDKD log:137) "Continuous": true,
[271.40 INFO] (SDKD log:137) "VSize": 128,
[271.41 INFO] (SDKD log:137) "Repeat": "REP",
[271.41 INFO] (SDKD log:137) "KSize": 32
[271.41 INFO] (SDKD log:137) },
[271.41 INFO] (SDKD log:137) "Options": {
[271.41 INFO] (SDKD log:137) "DelayMax": 10,
[271.41 INFO] (SDKD log:137) "DelayMin": 1,
[271.41 INFO] (SDKD log:137) "IterWait": 1,
[271.41 INFO] (SDKD log:137) "TimeRes": 1
[271.42 INFO] (SDKD log:137) }
[271.42 INFO] (SDKD log:137) }
[271.47 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.47 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37
[271.48 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.48 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[271.48 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.48 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.49 INFO] (SDKD log:137) "DS": {
[271.49 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.51 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.51 INFO] (SDKD log:137) "Count": 15000,
[271.51 INFO] (SDKD log:137) "Continuous": true,
[271.51 INFO] (SDKD log:137) "VSize": 128,
[271.51 INFO] (SDKD log:137) "Repeat": "REP",
[271.51 INFO] (SDKD log:137) "KSize": 32
[271.52 INFO] (SDKD log:137) },
[271.52 INFO] (SDKD log:137) "Options": {
[271.52 INFO] (SDKD log:137) "DelayMax": 10,
[271.52 INFO] (SDKD log:137) "DelayMin": 1,
[271.52 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.52 INFO] (SDKD log:137) "IterWait": 1,
[271.52 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.53 INFO] (SDKD log:137) "TimeRes": 1
[271.53 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.53 INFO] (SDKD log:137) }
[271.53 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.53 INFO] (SDKD log:137) }
[271.55 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39
[271.55 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[271.55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.56 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.56 INFO] (SDKD log:137) "DS": {
[271.56 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.56 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.56 INFO] (SDKD log:137) "Count": 15000,
[271.56 INFO] (SDKD log:137) "Continuous": true,
[271.57 INFO] (SDKD log:137) "VSize": 128,
[271.57 INFO] (SDKD log:137) "Repeat": "REP",
[271.57 INFO] (SDKD log:137) "KSize": 32
[271.57 INFO] (SDKD log:137) },
[271.57 INFO] (SDKD log:137) "Options": {
[271.58 INFO] (SDKD log:137) "DelayMax": 10,
[271.58 INFO] (SDKD log:137) "DelayMin": 1,
[271.58 INFO] (SDKD log:137) "IterWait": 1,
[271.58 INFO] (SDKD log:137) "TimeRes": 1
[271.58 INFO] (SDKD log:137) }
[271.58 INFO] (SDKD log:137) }
[271.61 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.62 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41
[271.62 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[271.63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.63 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.63 INFO] (SDKD log:137) "DS": {
[271.63 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.64 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.64 INFO] (SDKD log:137) "Count": 15000,
[271.64 INFO] (SDKD log:137) "Continuous": true,
[271.64 INFO] (SDKD log:137) "VSize": 128,
[271.64 INFO] (SDKD log:137) "Repeat": "REP",
[271.64 INFO] (SDKD log:137) "KSize": 32
[271.64 INFO] (SDKD log:137) },
[271.65 INFO] (SDKD log:137) "Options": {
[271.65 INFO] (SDKD log:137) "DelayMax": 10,
[271.65 INFO] (SDKD log:137) "DelayMin": 1,
[271.65 INFO] (SDKD log:137) "IterWait": 1,
[271.65 INFO] (SDKD log:137) "TimeRes": 1
[271.65 INFO] (SDKD log:137) }
[271.65 INFO] (SDKD log:137) }
[271.67 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.67 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.67 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.67 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.68 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.68 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.68 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.68 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.68 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.68 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.68 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.69 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.69 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.69 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.69 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.69 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.69 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.69 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.71 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.72 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.73 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.73 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.75 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.76 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.76 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.78 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.79 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.79 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43
[271.79 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.80 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.80 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[271.80 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.80 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.81 INFO] (SDKD log:137) "DS": {
[271.81 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.81 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.82 INFO] (SDKD log:137) "Count": 15000,
[271.82 INFO] (SDKD log:137) "Continuous": true,
[271.82 INFO] (SDKD log:137) "VSize": 128,
[271.82 INFO] (SDKD log:137) "Repeat": "REP",
[271.82 INFO] (SDKD log:137) "KSize": 32
[271.82 INFO] (SDKD log:137) },
[271.82 INFO] (SDKD log:137) "Options": {
[271.82 INFO] (SDKD log:137) "DelayMax": 10,
[271.83 INFO] (SDKD log:137) "DelayMin": 1,
[271.83 INFO] (SDKD log:137) "IterWait": 1,
[271.83 INFO] (SDKD log:137) "TimeRes": 1
[271.83 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@126.45
[271.83 INFO] (SDKD log:137) }
[271.84 INFO] (SDKD log:137) }
[271.84 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@126.57 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[271.84 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[271.85 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[271.86 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.87 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[271.87 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.87 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[271.87 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.87 INFO] (SDKD log:137) "DS": {
[271.87 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.87 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[271.88 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[271.88 INFO] (SDKD log:137) "Count": 15000,
[271.88 INFO] (SDKD log:137) "Continuous": true,
[271.88 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.88 INFO] (SDKD log:137) "VSize": 128,
[271.88 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.88 INFO] (SDKD log:137) "Repeat": "REP",
[271.88 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.89 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.89 INFO] (SDKD log:137) "KSize": 32
[271.89 INFO] (SDKD log:137) },
[271.89 INFO] (SDKD log:137) "Options": {
[271.90 INFO] (SDKD log:137) "DelayMax": 10,
[271.90 INFO] (SDKD log:137) "DelayMin": 1,
[271.90 INFO] (SDKD log:137) "IterWait": 1,
[271.90 INFO] (SDKD log:137) "TimeRes": 1
[271.90 INFO] (SDKD log:137) }
[271.90 INFO] (SDKD log:137) }
[271.91 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.92 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.92 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.92 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.92 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.92 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.93 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.93 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.93 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.93 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.93 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.93 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.94 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.94 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.95 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.95 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.96 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.96 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.96 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.96 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.97 INFO] (SDKD log:137) +++ Received exception of ID 0
[271.97 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[271.97 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[271.97 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[271.99 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[272.00 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@127.47
[272.00 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@127.58 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32},
 Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[272.01 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.01 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.01 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.02 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.02 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.02 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.02 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.02 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.03 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.03 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.03 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.03 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.03 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.04 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.04 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.04 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[272.04 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.04 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[272.04 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.04 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[272.04 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.04 INFO] (SDKD log:137) "DS": {
[272.05 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.05 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[272.05 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.05 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[272.05 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.05 INFO] (SDKD log:137) "Count": 15000,
[272.05 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.05 INFO] (SDKD log:137) "Continuous": true,
[272.05 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.06 INFO] (SDKD log:137) "VSize": 128,
[272.06 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.07 INFO] (SDKD log:137) "Repeat": "REP",
[272.07 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.07 INFO] (SDKD log:137) "KSize": 32
[272.07 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.07 INFO] (SDKD log:137) },
[272.07 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.07 INFO] (SDKD log:137) "Options": {
[272.07 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.07 INFO] (SDKD log:137) "DelayMax": 10,
[272.08 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.08 INFO] (SDKD log:137) "DelayMin": 1,
[272.08 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.08 INFO] (SDKD log:137) "IterWait": 1,
[272.08 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.08 INFO] (SDKD log:137) "TimeRes": 1
[272.08 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.09 INFO] (SDKD log:137) }
[272.09 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.09 INFO] (SDKD log:137) }
[272.09 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.09 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.10 INFO] (SDKD log:137) +++ Received exception of ID 0
[272.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[272.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[272.11 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[272.15 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@128.49
[272.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@128.59 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option
s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}}
[272.17 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[272.17 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[272.17 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@129.60 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[272.17 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[272.18 INFO] (SDKD log:137) "DS": {
[272.18 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[272.18 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[272.19 INFO] (SDKD log:137) "Count": 15000,
[272.19 INFO] (SDKD log:137) "Continuous": true,
[272.19 INFO] (SDKD log:137) "VSize": 128,
[272.19 INFO] (SDKD log:137) "Repeat": "REP",
[272.19 INFO] (SDKD log:137) "KSize": 32
[272.20 INFO] (SDKD log:137) },
[272.20 INFO] (SDKD log:137) "Options": {
[272.20 INFO] (SDKD log:137) "DelayMax": 10,
[272.20 INFO] (SDKD log:137) "DelayMin": 1,
[272.20 INFO] (SDKD log:137) "IterWait": 1,
[272.21 INFO] (SDKD log:137) "TimeRes": 1
[272.21 INFO] (SDKD log:137) }
[272.21 INFO] (SDKD log:137) }
[272.21 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[272.22 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 129
[272.22 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[272.24 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@129.60
[272.24 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@129.61 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=
12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[272.25 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[272.25 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[272.26 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[272.26 INFO] (SDKD log:137) "DS": {
[272.26 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[272.26 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[272.27 INFO] (SDKD log:137) "Count": 1000,
[272.27 INFO] (SDKD log:137) "Continuous": false,
[272.27 INFO] (SDKD log:137) "VSize": 512,
[272.27 INFO] (SDKD log:137) "Repeat": "rep",
[272.27 INFO] (SDKD log:137) "KSize": 12
[272.27 INFO] (SDKD log:137) },
[272.28 INFO] (SDKD log:137) "Schema": {
[272.28 INFO] (SDKD log:137) "InflateContent": "meh",
[272.28 INFO] (SDKD log:137) "InflateLevel": 40
[272.28 INFO] (SDKD log:137) },
[272.28 INFO] (SDKD log:137) "Options": {
[272.28 INFO] (SDKD log:137) "DelayMax": 0,
[272.28 INFO] (SDKD log:137) "DelayMin": 0,
[272.29 INFO] (SDKD log:137) "IterWait": 1,
[272.29 INFO] (SDKD log:137) "TimeRes": 0
[272.29 INFO] (SDKD log:137) }
[272.29 INFO] (SDKD log:137) }
[439.04 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[439.04 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@129.61
[439.05 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@129.62
[439.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[439.05 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@130.63 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[439.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[439.05 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411
[439.06 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411
[439.06 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[439.06 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[439.06 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 130
[439.06 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[439.11 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@130.63
[439.11 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@130.64 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[439.12 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@131.65 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[439.12 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[439.13 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[439.13 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[439.13 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[439.13 INFO] (SDKD log:137) "DS": {},
[439.14 INFO] (SDKD log:137) "ViewParameters": {
[439.14 INFO] (SDKD log:137) "limit": 0
[439.14 INFO] (SDKD log:137) },
[439.14 INFO] (SDKD log:137) "DesignName": "test_design",
[439.15 INFO] (SDKD log:137) "Options": {
[439.15 INFO] (SDKD log:137) "DelayMax": 10,
[439.15 INFO] (SDKD log:137) "DelayMin": 1,
[439.15 INFO] (SDKD log:137) "IterWait": 1,
[439.16 INFO] (SDKD log:137) "ViewQueryCount": -1,
[439.16 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[439.16 INFO] (SDKD log:137) "TimeRes": 1
[439.17 INFO] (SDKD log:137) },
[439.17 INFO] (SDKD log:137) "ViewName": "test_view"
[439.17 INFO] (SDKD log:137) }
[439.17 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 131
[439.17 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[439.21 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@131.65
[439.21 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@131.66 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[439.21 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[439.22 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[439.22 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@132.67 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[439.22 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[439.22 INFO] (SDKD log:137) "DS": {},
[439.23 INFO] (SDKD log:137) "ViewParameters": {
[439.23 INFO] (SDKD log:137) "limit": 0
[439.23 INFO] (SDKD log:137) },
[439.23 INFO] (SDKD log:137) "DesignName": "test_design",
[439.23 INFO] (SDKD log:137) "Options": {
[439.23 INFO] (SDKD log:137) "DelayMax": 10,
[439.24 INFO] (SDKD log:137) "DelayMin": 1,
[439.24 INFO] (SDKD log:137) "IterWait": 1,
[439.24 INFO] (SDKD log:137) "ViewQueryCount": -1,
[439.24 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[439.25 INFO] (SDKD log:137) "TimeRes": 1
[439.25 INFO] (SDKD log:137) },
[439.25 INFO] (SDKD log:137) "ViewName": "test_view"
[439.25 INFO] (SDKD log:137) }
[439.25 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[439.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 132
[439.26 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[439.28 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@132.67
[439.28 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@132.68 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[439.29 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[439.29 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@133.69 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[439.29 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[439.29 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[439.30 INFO] (SDKD log:137) "DS": {},
[439.30 INFO] (SDKD log:137) "ViewParameters": {
[439.30 INFO] (SDKD log:137) "limit": 0
[439.30 INFO] (SDKD log:137) },
[439.30 INFO] (SDKD log:137) "DesignName": "test_design",
[439.30 INFO] (SDKD log:137) "Options": {
[439.31 INFO] (SDKD log:137) "DelayMax": 10,
[439.31 INFO] (SDKD log:137) "DelayMin": 1,
[439.31 INFO] (SDKD log:137) "IterWait": 1,
[439.31 INFO] (SDKD log:137) "ViewQueryCount": -1,
[439.32 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[439.32 INFO] (SDKD log:137) "TimeRes": 1
[439.32 INFO] (SDKD log:137) },
[439.32 INFO] (SDKD log:137) "ViewName": "test_view"
[439.32 INFO] (SDKD log:137) }
[439.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[439.33 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 133
[439.33 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[439.38 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@133.69
[439.38 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@133.70 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[439.39 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[439.39 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@134.71 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10
.3.4.147",8091]]}, Hostname=10.3.4.144}
[439.39 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[439.40 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[439.40 INFO] (SDKD log:137) "DS": {},
[439.40 INFO] (SDKD log:137) "ViewParameters": {
[439.40 INFO] (SDKD log:137) "limit": 0
[439.40 INFO] (SDKD log:137) },
[439.41 INFO] (SDKD log:137) "DesignName": "test_design",
[439.41 INFO] (SDKD log:137) "Options": {
[439.41 INFO] (SDKD log:137) "DelayMax": 10,
[439.41 INFO] (SDKD log:137) "DelayMin": 1,
[439.41 INFO] (SDKD log:137) "IterWait": 1,
[439.41 INFO] (SDKD log:137) "ViewQueryCount": -1,
[439.42 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[439.42 INFO] (SDKD log:137) "TimeRes": 1
[439.42 INFO] (SDKD log:137) },
[439.42 INFO] (SDKD log:137) "ViewName": "test_view"
[439.42 INFO] (SDKD log:137) }
[439.42 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[439.43 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 134
[439.43 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[439.43 INFO] (SDKD log:137) +++ Received exception of ID 0
[439.44 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[439.44 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object.
[439.44 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'.
[439.46 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@134.71
[439.47 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@134.72 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie
wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view}
[439.48 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[439.50 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[439.50 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds
[439.51 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[439.51 INFO] (SDKD log:137) "DS": {},
[439.51 INFO] (SDKD log:137) "ViewParameters": {
[439.51 INFO] (SDKD log:137) "limit": 0
[439.51 INFO] (SDKD log:137) },
[439.52 INFO] (SDKD log:137) "DesignName": "test_design",
[439.52 INFO] (SDKD log:137) "Options": {
[439.52 INFO] (SDKD log:137) "DelayMax": 10,
[439.52 INFO] (SDKD log:137) "DelayMin": 1,
[439.52 INFO] (SDKD log:137) "IterWait": 1,
[439.52 INFO] (SDKD log:137) "ViewQueryCount": -1,
[439.52 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[439.53 INFO] (SDKD log:137) "TimeRes": 1
[439.53 INFO] (SDKD log:137) },
[439.53 INFO] (SDKD log:137) "ViewName": "test_view"
[439.53 INFO] (SDKD log:137) }
[469.51 INFO] (Scenario run:86) Starting CHANGE phase
[469.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[470.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[471.59 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[472.62 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0%
[473.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 1%
[474.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2%
[475.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2%
[476.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2%
[477.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3%
[478.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4%
[479.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5%
[480.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5%
[481.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 6%
[482.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 6%
[483.78 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7%
[484.79 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7%
[485.80 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 8%
[486.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9%
[487.82 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9%
[488.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10%
[489.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10%
[490.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11%
[491.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12%
[492.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12%
[493.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13%
[494.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14%
[495.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15%
[496.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15%
[497.95 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16%
[498.95 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[499.96 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[500.97 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[501.97 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[502.98 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18%
[503.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18%
[505.00 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18%
[506.01 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19%
[507.01 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19%
[508.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20%
[509.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20%
[510.04 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20%
[511.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21%
[512.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22%
[513.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 23%
[514.09 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25%
[515.09 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26%
[516.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28%
[517.11 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29%
[518.13 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 30%
[519.15 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31%
[520.15 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31%
[521.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32%
[522.18 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32%
[523.19 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33%
[524.20 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33%
[525.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 34%
[526.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35%
[527.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36%
[528.37 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37%
[529.38 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 38%
[530.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 38%
[531.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39%
[532.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40%
[533.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40%
[534.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 41%
[535.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 41%
[536.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42%
[537.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43%
[538.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 44%
[539.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 44%
[540.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45%
[541.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 46%
[542.78 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48%
[543.79 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49%
[544.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50%
[545.82 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51%
[546.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 52%
[547.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53%
[548.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55%
[549.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56%
[551.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56%
[552.39 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56%
[553.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57%
[554.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57%
[555.49 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58%
[556.50 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58%
[557.51 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59%
[558.53 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 60%
[559.54 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 61%
[560.59 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 61%
[561.61 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 62%
[562.61 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 63%
[563.62 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 64%
[564.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 64%
[565.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 65%
[566.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 65%
[567.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 66%
[568.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 67%
[569.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 67%
[570.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 68%
[571.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 68%
[572.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 69%
[573.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 69%
[574.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 70%
[575.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 70%
[576.78 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 71%
[577.79 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 71%
[578.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 72%
[579.82 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 73%
[580.82 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 73%
[581.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 74%
[582.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 74%
[583.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 75%
[584.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 75%
[585.87 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 75%
[587.26 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 76%
[588.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 76%
[590.00 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 77%
[591.01 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 77%
[592.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 77%
[593.04 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 77%
[594.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 77%
[595.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 77%
[596.09 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 78%
[597.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 78%
[599.20 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 78%
[600.21 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[601.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[602.23 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[603.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[604.25 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[605.26 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[606.27 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 79%
[607.28 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 80%
[608.29 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 80%
[609.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 80%
[610.32 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 80%
[611.33 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 80%
[612.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[613.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[614.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[615.38 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[616.39 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[617.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[618.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[619.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[620.46 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 81%
[621.47 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 82%
[622.47 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 82%
[623.48 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 82%
[625.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 82%
[626.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 82%
[627.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 83%
[628.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 83%
[629.80 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 83%
[630.82 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 83%
[631.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 84%
[632.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 84%
[633.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 85%
[634.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 85%
[635.90 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 85%
[636.91 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 85%
[637.91 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 86%
[638.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 86%
[639.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 87%
[640.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 87%
[641.95 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 87%
[642.96 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 88%
[643.97 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 88%
[644.97 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 89%
[645.98 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 89%
[646.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 89%
[648.00 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 90%
[649.00 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 90%
[650.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 90%
[651.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 90%
[652.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 91%
[653.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 91%
[654.04 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 91%
[655.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 91%
[656.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 92%
[657.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 92%
[658.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 92%
[659.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 93%
[660.09 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 93%
[661.09 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 94%
[662.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96%
[662.81 INFO] (SDKD log:137) [Sdkd.ViewQuery|Error] ... System.NullReferenceException Object reference not set to an instance of an object. at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace-n
et2\sdkd-net\src\Sdkd\Views\ViewCommand.cs:line 268
[663.11 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 97%
[664.12 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 99%
[665.13 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[666.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[667.15 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[668.16 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[669.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[670.19 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[671.47 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[672.48 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[673.49 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[674.50 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100%
[675.52 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[675.52 INFO] (Scenario run:97) CHANGE phase done
[675.52 INFO] (Scenario run:103) Starting REBOUND for 90 seconds
[676.23 INFO] (SDKD log:137) +++ Received exception of ID 0
[676.23 INFO] (SDKD log:137)
[676.23 INFO] (SDKD log:137) Unhandled Exception:
[676.24 INFO] (SDKD log:137) Unhandled Exception:
[676.24 INFO] (SDKD log:137)
[676.24 INFO] (SDKD log:137)
[676.24 INFO] (SDKD log:137) Unhandled Exception: Unhandled Exception: Unhandled Exception:
[676.25 INFO] (SDKD log:137)
[676.25 INFO] (SDKD log:137) Unhandled Exception:
[676.26 INFO] (SDKD log:137) Unhandled Exception:
[676.26 INFO] (SDKD log:137) Unhandled Exception: Unhandled Exception:
[676.26 INFO] (SDKD log:137)
[676.26 INFO] (SDKD log:137)
[676.26 INFO] (SDKD log:137) Unhandled Exception:
[676.26 INFO] (SDKD log:137)
[676.26 INFO] (SDKD log:137) Unhandled Exception: Unhandled Exception:
[676.27 INFO] (SDKD log:137) Unhandled Exception: Unhandled Exception: Unhandled Exception: Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.27 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 3
[676.27 INFO] (SDKD log:137) at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 172
   at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:li
ne 194
   at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couc
hbase\Core\Buckets\CouchbaseBucket.cs:line 179
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104
   at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 312
   at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[676.28 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.28 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.28 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.28 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 5
[676.28 INFO] (SDKD log:137) at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 172
   at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:li
ne 194
   at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couc
hbase\Core\Buckets\CouchbaseBucket.cs:line 179
   at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104
   at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888
   at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18
   at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[676.29 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.29 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.29 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.30 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.30 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.30 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.30 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.30 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.30 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.31 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.31 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.31 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.31 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.31 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.32 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.32 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.32 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.32 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.32 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.32 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.32 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.33 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.33 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.33 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.33 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.33 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.33 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.34 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.34 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.34 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.34 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.34 INFO] (SDKD log:137) +++ Received exception of ID 2
[676.34 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.35 INFO] (SDKD log:137) +++ Received exception of ID 4
[676.36 INFO] (SDKD log:137)
[676.36 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.49 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 7
[676.49 INFO] (SDKD log:137) at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 172
   at Couchbase.Core.Buckets.CouchbaseBucket.CreateQuery(Boolean development, String designdoc, String viewname) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseB
ucket.cs:line 1165
   at Sdkd.Views.ViewCommand.GetViewObject(CouchbaseBucket cli, Request req, ViewOptions vo) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Views\ViewCommand.cs:line 161
   at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Views\ViewCommand.cs:line 265
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[676.50 INFO] (SDKD log:137)
[676.50 INFO] (SDKD log:137)
[676.50 INFO] (SDKD log:137) Unhandled Exception: Unhandled Exception: Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.50 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.50 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.50 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.50 INFO] (SDKD log:137) System.InvalidOperationException: Sequence contains no matching element
[676.51 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.51 INFO] (SDKD log:137)
[676.51 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.51 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.51 INFO] (SDKD log:137)
[676.51 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.51 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.51 INFO] (SDKD log:137)
[676.52 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.52 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.52 INFO] (SDKD log:137)
[676.52 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.53 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.53 INFO] (SDKD log:137)
[676.53 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[676.53 INFO] (SDKD log:137) +++ Received exception of ID 6
[676.53 INFO] (SDKD log:137)
[676.53 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
[765.52 INFO] (Scenario run:105) REBOUND done. Will collect results
[765.52 DEBUG] (Handle sendMessageAsync:183) > CANCEL@103.72
[871.65 INFO] (SDKD log:137) [Sdkd.Control|Info] Cancelling command 72 for Handle 103
[871.65 DEBUG] (Handle receiveMessage:158) < CANCEL@103.72
[871.65 DEBUG] (Handle sendMessageAsync:183) > CANCEL@104.72
[872.57 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer
[872.58 INFO] (SDKD log:137) at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 172
   at Couchbase.Core.Buckets.CouchbaseBucket.CreateQuery(Boolean development, String designdoc, String viewname) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseB
ucket.cs:line 1165
   at Sdkd.Views.ViewCommand.GetViewObject(CouchbaseBucket cli, Request req, ViewOptions vo) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Views\ViewCommand.cs:line 161
   at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Views\ViewCommand.cs:line 265
   at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231
   at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
[872.60 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: java.net.SocketException: Connection reset
        at com.couchbase.sdkdclient.context.HarnessException.create(HarnessException.java:56) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.scenario.PhasedScenario.run(PhasedScenario.java:115) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:169) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:165) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27]
        at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27]
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.implRead(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamDecoder.read(Unknown Source) ~[na:1.6.0_27]
        at java.io.InputStreamReader.read(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedReader.fill(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedReader.readLine(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedReader.readLine(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.receiveMessage(Handle.java:152) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.sendMessage(Handle.java:173) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.cancel(Handle.java:271) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.cancelCurrent(Handle.java:283) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.Workload.cancel(Workload.java:110) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.workload.WorkloadGroup.cancel(WorkloadGroup.java:63) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.scenario.PhasedScenario.run(PhasedScenario.java:112) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        ... 10 common frames omitted
[872.65 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73
[872.65 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]



 Comments   
Comment by Jeff Morris [ 16/Aug/14 ]
This should be fixed here: git fetch ssh://jmorris@review.couchbase.org:29418/couchbase-net-client refs/changes/46/40546/1 && git checkout FETCH_HEAD
Comment by Jeff Morris [ 19/Aug/14 ]
Test against github, please. The changeset has been merged already.
Comment by Andrei Baranouski [ 27/Aug/14 ]
fixed




[NCBC-587] Cannot access a disposed object in DefaultIOStrategy.Finalize() Created: 16/Aug/14  Updated: 27/Aug/14  Resolved: 27/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Andrei Baranouski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 3.0.0-1105

Attachments: Zip Archive NCBC-587.log.zip    
Issue Links:
Gantt: finish-start

 Description   
maybe it's the continuation of NCBC-575


I took a few commits ahead from gerrit


commit 2c00a9d3a397f5c1eddc56238cf4a19e6e6618dd
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 11 18:41:08 2014 -0700

    Add config files to tests

    Change-Id: Ie093eac1da4ffc742796ffd4df8a9c63cc1c9c2f

commit 5b18623a7c13fb16f21476fe77eaf9a192e92eb2
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Mon Aug 11 17:20:19 2014 -0700

    Complete IBucket implementation for Memcached and Couchbase Buckets

    Change-Id: If6fb47b9e4a5201e625f155dadf45696b8666674

commit a6001260b0437a9779f7b7730e857b77ac9aea31
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 15:31:06 2014 -0700

    NCBC-542: Provide Observe method on IBucket

    Change-Id: Iecfbc02490b9ce237d6fbe16619e5aa252566bd7

commit 321940886cffcdc30225201ea243d8c6e4df6da4
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 12:31:12 2014 -0700

    NCBC-576: Do not retry operation when unknown status is returned.

    Change-Id: Ic9afeb1373d9e7804cad3c2df0521b162956326d
    Reviewed-on: http://review.couchbase.org/40347
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Matt Ingenthron <matt@couchbase.com>




c:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,12 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/count=2
    workload=HybridWorkloadGroup
    rebalance/mode=out
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-16-14\025009\de7a81baca576c481ce29a22fed68ed4

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-count 2 --workload HybridWorkloadGroup
 \ --rebalance-mode out --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,23 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,48 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,49 INFO] (RunContext run:124) Starting cluster and driver
[1,49 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1,49 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,49 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[1,68 INFO] (SDKD log:137) --- Logging Self-Test ---
[1,68 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[1,68 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[1,68 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[1,68 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[1,69 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,04 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,04 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,22 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[8,65 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.145:8091,ns_1@10.3.4.145>
[9,10 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.146:8091,ns_1@10.3.4.146>
[9,56 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[10,04 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[10,05 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,05 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,05 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Andrei\.ssh\id_rsa
[12,16 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[12,16 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[12,17 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[12,17 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[12,18 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[12,18 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[12,18 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[12,19 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[12,90 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@4b54d870
[12,90 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@7f7d4e51
[12,90 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@40c718fc
[12,91 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@16c96e22
[12,91 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@3e598bc2
[27,29 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[40,62 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[49,85 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[58,58 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[59,47 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[59,47 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[59,69 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[59,92 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[67,00 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[69,58 INFO] (RunContext run:143) Driver and cluster initialized
[69,82 INFO] (RunContext call:167) Running scenario..
[69,82 INFO] (Scenario run:72) Starting RAMP phase
[69,82 INFO] (Workload setupDesign:63) Creating design test_design
[70,98 INFO] (Workload setupDesign:80) Design creation done
[70,98 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[70,98 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[70,99 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[71,00 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[71,02 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[71,02 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[71,05 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[71,05 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[78,07 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[78,07 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {Options={DelayMin=0, TimeRes=0, IterWait=1, DelayMax=0}, Schema={InflateLevel=40, InflateContent=meh}, DSType=DSTYPE_SEEDED, DS={KSize=12, KSeed=ViewFillerSeed, VSize=512, VSeed=Vie
wFillerSeed, Continuous=false, Repeat=rep, Count=1000}}
[78,08 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[78,08 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[78,08 INFO] (SDKD log:137) "Options": {
[78,09 INFO] (SDKD log:137) "DelayMin": 0,
[78,09 INFO] (SDKD log:137) "TimeRes": 0,
[78,09 INFO] (SDKD log:137) "IterWait": 1,
[78,09 INFO] (SDKD log:137) "DelayMax": 0
[78,10 INFO] (SDKD log:137) },
[78,10 INFO] (SDKD log:137) "Schema": {
[78,10 INFO] (SDKD log:137) "InflateLevel": 40,
[78,11 INFO] (SDKD log:137) "InflateContent": "meh"
[78,11 INFO] (SDKD log:137) },
[78,11 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[78,11 INFO] (SDKD log:137) "DS": {
[78,12 INFO] (SDKD log:137) "KSize": 12,
[78,12 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[78,12 INFO] (SDKD log:137) "VSize": 512,
[78,13 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[78,13 INFO] (SDKD log:137) "Continuous": false,
[78,13 INFO] (SDKD log:137) "Repeat": "rep",
[78,13 INFO] (SDKD log:137) "Count": 1000
[78,14 INFO] (SDKD log:137) }
[78,14 INFO] (SDKD log:137) }
[293,46 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[293,46 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[293,46 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[293,47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[293,47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[293,47 INFO] (Workload call:112) Preloading items into cluster before workload
[293,47 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[293,48 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[293,48 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[293,49 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[293,49 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[293,49 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[293,49 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[298,80 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[298,80 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@102.5 => {Options={DelayMin=0, TimeRes=0, IterWait=100, DelayMax=0}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=false, Repeat=REP, Coun
t=15000}}
[298,81 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[298,81 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[298,81 INFO] (SDKD log:137) "Options": {
[298,81 INFO] (SDKD log:137) "DelayMin": 0,
[298,82 INFO] (SDKD log:137) "TimeRes": 0,
[298,82 INFO] (SDKD log:137) "IterWait": 100,
[298,82 INFO] (SDKD log:137) "DelayMax": 0
[298,82 INFO] (SDKD log:137) },
[298,82 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[298,83 INFO] (SDKD log:137) "DS": {
[298,83 INFO] (SDKD log:137) "KSize": 32,
[298,83 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[298,83 INFO] (SDKD log:137) "VSize": 128,
[298,84 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[298,84 INFO] (SDKD log:137) "Continuous": false,
[298,84 INFO] (SDKD log:137) "Repeat": "REP",
[298,85 INFO] (SDKD log:137) "Count": 15000
[298,85 INFO] (SDKD log:137) }
[298,85 INFO] (SDKD log:137) }
[554,47 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[1781,34 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@102.5 => {[OK]: 14984, [MEMD:GENERIC]: 16}
[1781,34 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[1781,34 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[1781,35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[1781,35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.9 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.8 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.16 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.15 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.14 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.13 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.12 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,37 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.11 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,37 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.10 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1781,37 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[1781,38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[1781,38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[1781,38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[1781,39 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1781,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[1781,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[1781,39 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,40 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1781,40 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112
[1781,40 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111
[1781,40 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110
[1781,40 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109
[1781,41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[1781,41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[1781,41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[1787,97 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1787,97 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.9
[1787,98 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@105.17 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1787,98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1787,98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1787,98 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.18 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1787,99 INFO] (SDKD log:137) "Options": {
[1787,99 INFO] (SDKD log:137) "DelayMin": 1,
[1787,99 INFO] (SDKD log:137) "TimeRes": 1,
[1787,99 INFO] (SDKD log:137) "IterWait": 1,
[1787,99 INFO] (SDKD log:137) "DelayMax": 10
[1788,00 INFO] (SDKD log:137) },
[1788,00 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1788,00 INFO] (SDKD log:137) "DS": {
[1788,01 INFO] (SDKD log:137) "KSize": 32,
[1788,01 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1788,01 INFO] (SDKD log:137) "VSize": 128,
[1788,01 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1788,02 INFO] (SDKD log:137) "Continuous": true,
[1788,02 INFO] (SDKD log:137) "Repeat": "REP",
[1788,02 INFO] (SDKD log:137) "Count": 15000
[1788,03 INFO] (SDKD log:137) }
[1788,03 INFO] (SDKD log:137) }
[1788,03 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1788,03 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113
[1793,55 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1793,56 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[1793,56 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@103.19 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1793,56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1793,56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1793,57 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.20 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1793,57 INFO] (SDKD log:137) "Options": {
[1793,57 INFO] (SDKD log:137) "DelayMin": 1,
[1793,57 INFO] (SDKD log:137) "TimeRes": 1,
[1793,58 INFO] (SDKD log:137) "IterWait": 1,
[1793,58 INFO] (SDKD log:137) "DelayMax": 10
[1793,58 INFO] (SDKD log:137) },
[1793,58 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1793,58 INFO] (SDKD log:137) "DS": {
[1793,59 INFO] (SDKD log:137) "KSize": 32,
[1793,59 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1793,59 INFO] (SDKD log:137) "VSize": 128,
[1793,59 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1793,60 INFO] (SDKD log:137) "Continuous": true,
[1793,60 INFO] (SDKD log:137) "Repeat": "REP",
[1793,60 INFO] (SDKD log:137) "Count": 15000
[1793,60 INFO] (SDKD log:137) }
[1793,60 INFO] (SDKD log:137) }
[1793,61 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1793,61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114
[1799,53 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1799,54 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.8
[1799,54 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@104.21 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1799,55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1799,55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1799,55 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.22 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1799,55 INFO] (SDKD log:137) "Options": {
[1799,56 INFO] (SDKD log:137) "DelayMin": 1,
[1799,56 INFO] (SDKD log:137) "TimeRes": 1,
[1799,56 INFO] (SDKD log:137) "IterWait": 1,
[1799,56 INFO] (SDKD log:137) "DelayMax": 10
[1799,56 INFO] (SDKD log:137) },
[1799,57 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1799,57 INFO] (SDKD log:137) "DS": {
[1799,58 INFO] (SDKD log:137) "KSize": 32,
[1799,58 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1799,58 INFO] (SDKD log:137) "VSize": 128,
[1799,58 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1799,59 INFO] (SDKD log:137) "Continuous": true,
[1799,59 INFO] (SDKD log:137) "Repeat": "REP",
[1799,59 INFO] (SDKD log:137) "Count": 15000
[1799,60 INFO] (SDKD log:137) }
[1799,60 INFO] (SDKD log:137) }
[1799,60 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1799,61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115
[1804,71 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1804,71 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.16
[1804,72 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@112.23 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1804,72 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1804,72 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1804,72 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.24 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1804,73 INFO] (SDKD log:137) "Options": {
[1804,73 INFO] (SDKD log:137) "DelayMin": 1,
[1804,73 INFO] (SDKD log:137) "TimeRes": 1,
[1804,73 INFO] (SDKD log:137) "IterWait": 1,
[1804,74 INFO] (SDKD log:137) "DelayMax": 10
[1804,74 INFO] (SDKD log:137) },
[1804,74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1804,74 INFO] (SDKD log:137) "DS": {
[1804,74 INFO] (SDKD log:137) "KSize": 32,
[1804,75 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1804,75 INFO] (SDKD log:137) "VSize": 128,
[1804,75 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1804,75 INFO] (SDKD log:137) "Continuous": true,
[1804,76 INFO] (SDKD log:137) "Repeat": "REP",
[1804,76 INFO] (SDKD log:137) "Count": 15000
[1804,76 INFO] (SDKD log:137) }
[1804,76 INFO] (SDKD log:137) }
[1804,77 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1804,77 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116
[1809,89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1809,89 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.15
[1809,90 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@111.25 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1809,90 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1809,90 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1809,90 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.26 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1809,91 INFO] (SDKD log:137) "Options": {
[1809,91 INFO] (SDKD log:137) "DelayMin": 1,
[1809,91 INFO] (SDKD log:137) "TimeRes": 1,
[1809,91 INFO] (SDKD log:137) "IterWait": 1,
[1809,92 INFO] (SDKD log:137) "DelayMax": 10
[1809,92 INFO] (SDKD log:137) },
[1809,92 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1809,92 INFO] (SDKD log:137) "DS": {
[1809,92 INFO] (SDKD log:137) "KSize": 32,
[1809,93 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1809,93 INFO] (SDKD log:137) "VSize": 128,
[1809,93 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1809,93 INFO] (SDKD log:137) "Continuous": true,
[1809,94 INFO] (SDKD log:137) "Repeat": "REP",
[1809,94 INFO] (SDKD log:137) "Count": 15000
[1809,94 INFO] (SDKD log:137) }
[1809,94 INFO] (SDKD log:137) }
[1809,94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1809,95 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117
[1814,99 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1814,99 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.12
[1814,99 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@108.27 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1815,00 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1815,00 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1815,00 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.28 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1815,00 INFO] (SDKD log:137) "Options": {
[1815,00 INFO] (SDKD log:137) "DelayMin": 1,
[1815,01 INFO] (SDKD log:137) "TimeRes": 1,
[1815,01 INFO] (SDKD log:137) "IterWait": 1,
[1815,01 INFO] (SDKD log:137) "DelayMax": 10
[1815,01 INFO] (SDKD log:137) },
[1815,02 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[1815,02 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1815,02 INFO] (SDKD log:137) "DS": {
[1815,02 INFO] (SDKD log:137) "KSize": 32,
[1815,03 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1815,03 INFO] (SDKD log:137) "VSize": 128,
[1815,03 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1815,03 INFO] (SDKD log:137) "Continuous": true,
[1815,03 INFO] (SDKD log:137) "Repeat": "REP",
[1815,03 INFO] (SDKD log:137) "Count": 15000
[1815,04 INFO] (SDKD log:137) }
[1815,04 INFO] (SDKD log:137) }
[1815,04 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1815,04 INFO] (SDKD log:137) System.ObjectDisposedException: Доступ к ликвидированному объекту невозможен.
[1815,05 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118
[1815,05 INFO] (SDKD log:137) Имя объекта: "System.Net.Sockets.Socket".
[1815,05 INFO] (SDKD log:137) at System.Net.Sockets.Socket.get_RemoteEndPoint()
[1815,05 INFO] (SDKD log:137) at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:строка 38
[1815,06 INFO] (SDKD log:137) at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString()
[1815,06 INFO] (SDKD log:137) at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer)
[1815,06 INFO] (SDKD log:137) at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer)
[1815,06 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage]
[1815,07 INFO] (SDKD log:137) System.ObjectDisposedException: Доступ к ликвидированному объекту невозможен.
[1815,07 INFO] (SDKD log:137) Имя объекта: "System.Net.Sockets.Socket".
[1815,07 INFO] (SDKD log:137) at System.Net.Sockets.Socket.get_RemoteEndPoint()
[1815,07 INFO] (SDKD log:137) at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:строка 38
[1815,07 INFO] (SDKD log:137) at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString()
[1815,08 INFO] (SDKD log:137) at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer)
[1815,08 INFO] (SDKD log:137) at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer)
[1820,02 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1820,02 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.11
[1820,02 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@107.29 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1820,02 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1820,03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1820,03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.30 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1820,03 INFO] (SDKD log:137) "Options": {
[1820,03 INFO] (SDKD log:137) "DelayMin": 1,
[1820,03 INFO] (SDKD log:137) "TimeRes": 1,
[1820,04 INFO] (SDKD log:137) "IterWait": 1,
[1820,04 INFO] (SDKD log:137) "DelayMax": 10
[1820,04 INFO] (SDKD log:137) },
[1820,04 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1820,05 INFO] (SDKD log:137) "DS": {
[1820,05 INFO] (SDKD log:137) "KSize": 32,
[1820,05 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1820,05 INFO] (SDKD log:137) "VSize": 128,
[1820,05 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1820,06 INFO] (SDKD log:137) "Continuous": true,
[1820,06 INFO] (SDKD log:137) "Repeat": "REP",
[1820,06 INFO] (SDKD log:137) "Count": 15000
[1820,06 INFO] (SDKD log:137) }
[1820,06 INFO] (SDKD log:137) }
[1820,06 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1820,07 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119
[1821,53 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 23344
[1825,14 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1825,14 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.14
[1825,15 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@110.31 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1825,15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1825,15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1825,15 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.32 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1825,16 INFO] (SDKD log:137) "Options": {
[1825,16 INFO] (SDKD log:137) "DelayMin": 1,
[1825,16 INFO] (SDKD log:137) "TimeRes": 1,
[1825,16 INFO] (SDKD log:137) "IterWait": 1,
[1825,17 INFO] (SDKD log:137) "DelayMax": 10
[1825,17 INFO] (SDKD log:137) },
[1825,17 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1825,17 INFO] (SDKD log:137) "DS": {
[1825,17 INFO] (SDKD log:137) "KSize": 32,
[1825,18 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1825,18 INFO] (SDKD log:137) "VSize": 128,
[1825,18 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1825,18 INFO] (SDKD log:137) "Continuous": true,
[1825,18 INFO] (SDKD log:137) "Repeat": "REP",
[1825,19 INFO] (SDKD log:137) "Count": 15000
[1825,19 INFO] (SDKD log:137) }
[1825,19 INFO] (SDKD log:137) }
[1825,19 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1825,19 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120
[1830,16 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1830,16 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.13
[1830,16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@109.33 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1830,17 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1830,17 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1830,17 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.34 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1830,17 INFO] (SDKD log:137) "Options": {
[1830,17 INFO] (SDKD log:137) "DelayMin": 1,
[1830,18 INFO] (SDKD log:137) "TimeRes": 1,
[1830,18 INFO] (SDKD log:137) "IterWait": 1,
[1830,18 INFO] (SDKD log:137) "DelayMax": 10
[1830,18 INFO] (SDKD log:137) },
[1830,18 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1830,19 INFO] (SDKD log:137) "DS": {
[1830,19 INFO] (SDKD log:137) "KSize": 32,
[1830,19 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1830,19 INFO] (SDKD log:137) "VSize": 128,
[1830,20 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1830,20 INFO] (SDKD log:137) "Continuous": true,
[1830,20 INFO] (SDKD log:137) "Repeat": "REP",
[1830,20 INFO] (SDKD log:137) "Count": 15000
[1830,20 INFO] (SDKD log:137) }
[1830,21 INFO] (SDKD log:137) }
[1830,21 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1830,21 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121
[1836,39 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1836,39 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.10
[1836,40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@106.35 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1836,40 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1836,40 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1836,40 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.36 => {Options={OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]], Username=default, SSL=false, Password=}, Bucket=default, Port=8091, Hostname=10.3.4.144}
[1836,41 INFO] (SDKD log:137) "Options": {
[1836,41 INFO] (SDKD log:137) "DelayMin": 1,
[1836,41 INFO] (SDKD log:137) "TimeRes": 1,
[1836,41 INFO] (SDKD log:137) "IterWait": 1,
[1836,42 INFO] (SDKD log:137) "DelayMax": 10
[1836,42 INFO] (SDKD log:137) },
[1836,42 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1836,42 INFO] (SDKD log:137) "DS": {
[1836,43 INFO] (SDKD log:137) "KSize": 32,
[1836,43 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1836,43 INFO] (SDKD log:137) "VSize": 128,
[1836,43 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1836,44 INFO] (SDKD log:137) "Continuous": true,
[1836,44 INFO] (SDKD log:137) "Repeat": "REP",
[1836,44 INFO] (SDKD log:137) "Count": 15000
[1836,44 INFO] (SDKD log:137) }
[1836,44 INFO] (SDKD log:137) }
[1836,45 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[1836,45 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122
[1841,55 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1841,55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.18
[1841,55 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.37 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1841,56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1841,56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1841,56 INFO] (SDKD log:137) "Options": {
[1841,56 INFO] (SDKD log:137) "DelayMin": 1,
[1841,57 INFO] (SDKD log:137) "TimeRes": 1,
[1841,57 INFO] (SDKD log:137) "IterWait": 1,
[1841,57 INFO] (SDKD log:137) "DelayMax": 10
[1841,57 INFO] (SDKD log:137) },
[1841,58 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1841,58 INFO] (SDKD log:137) "DS": {
[1841,58 INFO] (SDKD log:137) "KSize": 32,
[1841,58 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1841,58 INFO] (SDKD log:137) "VSize": 128,
[1841,59 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1841,59 INFO] (SDKD log:137) "Continuous": true,
[1841,59 INFO] (SDKD log:137) "Repeat": "REP",
[1841,59 INFO] (SDKD log:137) "Count": 15000
[1841,59 INFO] (SDKD log:137) }
[1841,60 INFO] (SDKD log:137) }
[1846,71 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1846,71 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.20
[1846,71 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.38 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Count=15000
}}
[1846,72 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET
[1846,72 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1846,72 INFO] (SDKD log:137) "Options": {
[1846,73 INFO] (SDKD log:137) "DelayMin": 1,
[1846,73 INFO] (SDKD log:137) "TimeRes": 1,
[1846,73 INFO] (SDKD log:137) "IterWait": 1,
[1846,73 INFO] (SDKD log:137) "DelayMax": 10
[1846,73 INFO] (SDKD log:137) },
[1846,74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1846,74 INFO] (SDKD log:137) "DS": {
[1846,74 INFO] (SDKD log:137) "KSize": 32,
[1846,74 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1846,74 INFO] (SDKD log:137) "VSize": 128,
[1846,75 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1846,75 INFO] (SDKD log:137) "Continuous": true,
[1846,75 INFO] (SDKD log:137) "Repeat": "REP",
[1846,75 INFO] (SDKD log:137) "Count": 15000
[1846,75 INFO] (SDKD log:137) }
[1846,76 INFO] (SDKD log:137) }
[1851,69 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[1851,69 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.22
[1851,69 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@115.39 => {Options={DelayMin=1, TimeRes=1, IterWait=1, DelayMax=10}, DSType=DSTYPE_SEEDED, DS={KSize=32, KSeed=SimpleKey, VSize=128, VSeed=SimpleValue, Continuous=true, Repeat=REP, Coun
t=15000}}
[1851,70 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[1851,70 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[1851,70 INFO] (SDKD log:137) "Options": {
[1851,71 INFO] (SDKD log:137) "DelayMin": 1,
[1851,71 INFO] (SDKD log:137) "TimeRes": 1,
[1851,71 INFO] (SDKD log:137) "IterWait": 1,
[1851,71 INFO] (SDKD log:137) "DelayMax": 10
[1851,71 INFO] (SDKD log:137) },
[1851,72 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[1851,72 INFO] (SDKD log:137) "DS": {
[1851,72 INFO] (SDKD log:137) "KSize": 32,
[1851,72 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[1851,72 INFO] (SDKD log:137) "VSize": 128,
[1851,73 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[1851,73 INFO] (SDKD log:137) "Continuous": true,
[1851,73 INFO] (SDKD log:137) "Repeat": "REP",
[1851,73 INFO] (SDKD log:137) "Count": 15000
[1851,73 INFO] (SDKD log:137) }
[1851,74 INFO] (SDKD log:137) }
[1854,01 INFO] (SDKD log:137)
[1854,01 INFO] (SDKD log:137) unhandled Exception: System.ObjectDisposedException: Cannot access a disposed object
[1854,01 INFO] (SDKD log:137) Object name: "Object ThreadLocal has been dispose.".
[1854,01 INFO] (SDKD log:137) at System.Threading.ThreadLocal`1.GetValueSlow()
[1854,01 INFO] (SDKD log:137) at System.Threading.ThreadLocal`1.get_Value()
[1854,02 INFO] (SDKD log:137) at System.Collections.Concurrent.ConcurrentBag`1.GetThreadList(Boolean forceCreate)
[1854,02 INFO] (SDKD log:137) at System.Collections.Concurrent.ConcurrentBag`1.TryTakeOrPeek(T& result, Boolean take)
[1854,02 INFO] (SDKD log:137) at System.Collections.Concurrent.ConcurrentBag`1.TryTake(T& result)
[1854,02 INFO] (SDKD log:137) at Couchbase.IO.ConnectionPool`1.Dispose(Boolean disposing) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\ConnectionPool.cs:строка 162
[1854,02 INFO] (SDKD log:137) at Couchbase.IO.ConnectionPool`1.Dispose() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\ConnectionPool.cs:строка 147
[1854,03 INFO] (SDKD log:137) at Couchbase.IO.Strategies.DefaultIOStrategy.Dispose(Boolean disposing) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:строка 113
[1854,03 INFO] (SDKD log:137) at Couchbase.IO.Strategies.DefaultIOStrategy.Finalize() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:строка 122
[2244,19 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_11]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_11]
        at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_11]
[2244,22 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.40
[2244,22 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.8.0_11]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.8.0_11]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.8.0_11]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]


client logs attached

 Comments   
Comment by Jeff Morris [ 19/Aug/14 ]
Try running again with what is on github. We pushed some commits through code review today. Attach the log and also the SDKQE code that's doing the testing.

Thanks,

Jeff
Comment by Andrei Baranouski [ 27/Aug/14 ]
fixed/verified




[NCBC-586] AddRef and Release on IBucket must not be public Created: 15/Aug/14  Updated: 15/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
AddRef and Release belong to a public interface, IBucket, thus are public methods. They are intended to be internal methods, so this needs to be refactored a bit to make them non-public.




[NCBC-585] Refactor finalization logic so that Exceptions are not thrown on finalizer thread Created: 15/Aug/14  Updated: 15/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Bug Priority: Critical
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to

 Description   
Reviewing MemcachedClient, there are a few issues that need to be resolved:
- The dispose/finalize logic itself is not correct in MemcachedClient. GCSuppressFinalize shouldn't be called during finalization for example.
- The finalizer contains code (logging) that could throw an exception if the config was not setup correctly or if It cannot create the instance for some reason




[NCBC-584] GetVBucket throws not implemented exception. Created: 14/Aug/14  Updated: 14/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
From Q&A: http://www.couchbase.com/communities/jp/q-and-a/weird-error
Weird Error

Hi Everyone,

Thanks in advance for the help. I have been getting the following error periodically.

GetVBucket is implemented only for VBucketNodeLocator implementation of IMemcachedNodeLocator

I am using couchbase as a session state offload and a caching utility.

The conditions under which I was receiving this error are as follows:

Couchbase 2.5.1 enterprise.
2 node system
when I am calling the following method:

var answer = _couchBaseClient.KeyExists("Key_Name");

For some reason the KeyExists method throws this error.

I am running this code in a two IIS server environment. The error essentially prevents the site from being usable.

I was able to remove this error but one of the two ways (neither of which is preferable to me):

removing the use of the KeyExists method which I don't want to do.
remove 1 node of my 2 node setup which makes fail over not possible.....again something I don't want to do.

So my question(s) is:

What I am doing wrong?
Configuration Issue?
Not best practice?

Thanks,
- See more at: http://www.couchbase.com/communities/jp/q-and-a/weird-error#sthash.cMQqY9An.Ytj2AA3q.dpuf




[NCBC-583] Ensure Dispose is call on every MD5CryptoServiceProvider class usage Created: 14/Aug/14  Updated: 18/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.8
Fix Version/s: 1.3.9

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to

 Description   
Every Get or Set call invokes KetamaNodeLocator.Locate(string key). The memcached keys are hashed through MD5 algorithm to find node they are supposed to be stored on. The current implementation creates a new MD5CryptoServiceProvider class which has underlying SafeHashHandle that registers into finalizer queue. Because the code does not call Dispose on the MD5CryptoServiceProvider instance, the SafeHashHandle remains in finalizer queue till garbage collector disposes it properly.





[NCBC-582] Provide PDB Files with Binaries Created: 13/Aug/14  Updated: 13/Aug/14  Resolved: 13/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.7
Fix Version/s: 1.3.8

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Jeff Morris [ 13/Aug/14 ]
Now included within the zip on S3




[NCBC-581]  System.InvalidOperationException when server list is empty Created: 12/Aug/14  Updated: 13/Aug/14  Resolved: 12/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Changing the code to make several attempts (and pauses) before throwing a new exception class: ServerUnavailableException



 Comments   
Comment by Jeff Morris [ 12/Aug/14 ]
http://review.couchbase.org/#/c/40546/
Comment by Andrei Baranouski [ 13/Aug/14 ]
initially found in https://www.couchbase.com/issues/browse/NCBC-576?focusedCommentId=97618&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-97618




[NCBC-580] not able to openBucket after certificate uninstalled/regenerated when ClientConfiguration uses SSL, BucketConfig doesn't Created: 12/Aug/14  Updated: 14/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: None
Affects Version/s: 2.0 DP3
Fix Version/s: backlog-2.0

Type: Bug Priority: Minor
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File log.txt    

 Description   
test
https://github.com/couchbase/sdkd-net/blob/client-2.0/src/FTests/SSLFeatures.cs#L342
https://github.com/couchbase/sdkd-net/blob/client-2.0/src/FTests/SSLFeatures.cs#L364

code

 ClientConfiguration extConfig = new ClientConfiguration
           {
               UseSsl = true,
               Servers = new List<Uri>
            {
                new Uri("http://testenv:8091/pools")
            }
           };

extConfig.BucketConfigs[bucketName].UseSsl = false;


[Test]
        public void Test_NotSSL_reOpenBucket_When_Cert_Removed()
        {
            installCert(certPath);
            using (var cluster = new CouchbaseCluster(extConfig))
            {
                using (var bucket = cluster.OpenBucket(bucketName))
                {

                    var result = bucket.Upsert(Path.GetRandomFileName(), Path.GetRandomFileName());
                    Assert.IsTrue(result.Success);
                    cluster.CloseBucket(bucket);
                }
                removeCert(certPath);

                var bucketRefreshed = cluster.OpenBucket(bucketName);//FAILED
                bucketRefreshed.Upsert(Path.GetRandomFileName(), Path.GetRandomFileName());
            }
        }


FTests.SSLFeatures.Test_NotSSL_reOpenBucket_When_Cert_Removed:
Couchbase.Configuration.ConfigException : Could not bootstrap default. See log for details.
at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) at d:\couchbase_src\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 206
at Couchbase.Core.ClusterManager.CreateBucket(String bucketName) в d:\couchbase_src\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 137
at Couchbase.CouchbaseCluster.OpenBucket(String bucketname) в d:\couchbase_src\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 103
at FTests.SSLFeatures.Test_NotSSL_reOpenBucket_When_Cert_Removed() в d:\couchbase_src\sdkd-net\src\FTests\SSLFeatures.cs:line 341


so, we try to openBucket when CouchbaseCluster is still "active" but initial certificate doesn't exist anymore


 Comments   
Comment by Jeff Morris [ 12/Aug/14 ]
>>See log for details.

No logs, so hard to say what the problem is here.
Comment by Andrei Baranouski [ 13/Aug/14 ]
client logs
Comment by Jeff Morris [ 13/Aug/14 ]
I can't read the actual reason why the ssl connection is failing, because of some encoding issue (probably because your localization settings?):

2014-08-13 11:29:46,830 [TestRunnerThread] WARN Couchbase.IO.ConnectionBase - Starting SSL encryption on 10.3.121.134
2014-08-13 11:29:47,385 [TestRunnerThread] ERROR Couchbase.IO.ConnectionBase - System.Security.Authentication.AuthenticationException: ��������� ���������� �������������� �������� ����������� �������� �����������.
   � System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception)
   � System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)

The problem could be somewhere with your installation/reinstallation of the client certificate. The client itself cannot make the connection with the newly installed cert.

As a feature installing/removing SSL certificates isn't something we have talked about supporting from the client. In most cases the cert would be installed and managed by itops outside of the application layer. That being said, it's an important test and will help automation later on - I do think we should lower the priority though to perhapa minor?
Comment by Andrei Baranouski [ 13/Aug/14 ]
I agree that it is a rare negative case and we can set lower priority

"As a feature installing/removing SSL certificates isn't something we have talked about supporting from the client" I remove/reinstall/regenerate certificate inside the tests to check client behaviour for different ssl feature scenarios we described in https://docs.google.com/document/d/1J3vWQnYxCtbD4WJbcSP4OqYiePAlUdCCKMSj7jtbzxs/edit#heading=h.o6lvbcwhi5gc

i've uploaded new client logs, the error says: System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the results of authentication.
Comment by Don Pinto [ 13/Aug/14 ]
For beta 2 might be OK, but for GA we should address this issue - I would see this scenario hit when you do cert refreshes or if there is a breach and you need to roll out a new certificate.

In any case, we might need to document this as well if there are any particular cases users should think about.

Thanks
Comment by Jeff Morris [ 13/Aug/14 ]
Agreed, but perhaps we add the functionality as a feature to the Management API?




[NCBC-579] Make unsupported memcached methods throw NotSupportedException Created: 11/Aug/14  Updated: 14/Aug/14

Status: Reopened
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Both Memcache and Couchbase buckets implement IBucket interface, but Memcached buckets only support a subset of the methods defined on that interface. For features that are not supported on memcached buckets, they currently throw NotImplementedExceptions, this should be change to NotSupportedExceptions.




[NCBC-578] Non Obtrusive JSON Document Deseralization Created: 11/Aug/14  Updated: 19/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.7
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: bchavez Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: all


 Description   
Hi there,

There was some discussion about the 1.3x SDK not forcefully injecting the "ID" property with the object's key.

The previous discussion was briefly mentioned here:
https://www.couchbase.com/issues/browse/NCBC-564

For convenience:
------------------------------
I am having problems with this section of code:

            if (!IsArrayOrCollection(typeof(T)))
            {
                value = DocHelper.InsertId(value, key);
            }

I am managing all the document IDs myself in my application. But DocHelper keeps interfering with my ID during deserialization. Is there a way we can possibly set a flag that wont interfere with my JSON ?

One example: All my domain objects have a strongly typed Guid as their ID.

The KEY to an Account object is "acct:aaaa-bb-cc-dd".

POCO in C#:
public class Account{
    public Guid Id{get;set;}
}

I format the key to this object inside my App; however, during deserialization DocHelper interferes with the JSON by injecting the KEY: "acct:aaaa-bb-cc-dd" into the ID property in the JSON before deseralization.

So when Newtonsoft deserializes the JSON with the injected DocHelper Id:Key:"acct:aaaa-bb-cc-dd" cannot be converted to a Guid because of the prefix "acct:" is an invalid Guid format. I'd prefer the SDK not interfere with any of my ser/deser JSON.

Perhaps we can add some flag that can short circuit the DocHelper injection?
------------------------------

Any suggestions on implementation?

As a static configuration variable on DocHelper or via App.config?

Of course, the default behavior would be to keep things as is for a non-breaking change.

Using something like a static property:

    public static class DocHelper
    {
        public static bool DisableIdInsert{get;set;}

        public static string InsertId(string json, string id)
        {
            if (!DisableIdInsert && !json.Contains("\"id\""))
            {
                return json.Insert(1, string.Concat("\"id\":", "\"", id, "\","));
            }
            return json;
        }
    }

Thanks,
Brian

 Comments   
Comment by Jeff Morris [ 19/Aug/14 ]
Hi Brian -

Sorry for not getting back to you sooner. What you have proposed looks reasonable, could you create a PR so we could push it through to review?

Thanks,

-Jeff




[NCBC-577] SDK 1.3.8 doesn't support uuid with bucket name in View query against 3.0 Created: 07/Aug/14  Updated: 21/Aug/14  Resolved: 12/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.6
Fix Version/s: 1.3.8

Type: Bug Priority: Test Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Microsoft Word encoding-issues-dotnet-sdk-137-couchabase-server-3.docx     PNG File jmorris-vs2012.png     Zip Archive system.uri-diff-output-in-console-vs-library-assemblies.zip     PNG File uri-mystery.png    

 Description   
run https://github.com/couchbase/couchbase-net-client/commit/86fd1c588df15611b557bfa7e401581e005c188e
against 3.0.0-1105

http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng-andrei/2/console

[176.39 INFO] (SDKD log:137) [Couchbase.DefaultHttpClient|Warn] System.Net.WebException: The remote server returned an error: (404) Not Found.
[176.39 INFO] (SDKD log:137) +++ Received exception of ID 0
[176.39 INFO] (SDKD log:137) [Couchbase.CouchbaseViewHandler|Warn] Design document not found, body: {"error":"not_found","reason":"missing"}
[176.39 INFO] (SDKD log:137)  


Jeff, as I understand 1.3.8 should support 3.0 CB as well?

then, the same fix for https://www.couchbase.com/issues/browse/NCBC-545 ( https://github.com/couchbase/couchbase-net-client/commit/e5ead27c1efa5d33bb4d0ba9f55d424ffd9d3c79) should be adjusted for 1.3.8?



 Comments   
Comment by Jeff Morris [ 07/Aug/14 ]
Yes, this is a bug specific to 3.0.
Comment by Jeff Morris [ 08/Aug/14 ]
Andrei -

From my unit tests this appears to not be a client error. The cluster you are running the tests against is missing the design doc you are querying: "Design document not found, body: {"error":"not_found","reason":"missing"}"

What I would do:

a) Check that the cluster indeed has the design doc and views sdkqe-net is using
b) Make sure it's not an issue with dev vs production: publish the view if it is

Also, including the url that has failed really helps debugging if you can locate it (along with the view name).

Assigning back to you.

Thanks,

Jeff

Comment by Andrei Baranouski [ 09/Aug/14 ]
Hi Jeff,

src/Couchbase/DefaultHttpClient.cs

I print out uri requests for HttpWebRequest

@@ -198,10 +198,11 @@ namespace Couchbase
                 queryString += string.Join("&", select);
 #endif
                 lock (client)
                 {
                     var uri = new Uri(client.BaseAddress + "/" + path + queryString);
+Console.WriteLine(uri);
                     var request = client.GetWebRequest(uri, client.BaseAddress.GetHashCode().ToString()) as HttpWebRequest;
                     request.Accept = "application/json";
                     request.ContentType = "application/json; charset=utf-8";
                     return request;
                 }


46.97 INFO] (SDKD log:137) http://10.3.121.136:8092/default+0218b57a1309c1d9a9bdf94f90057d27/_design/test_design/_view/test_view?
http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng-andrei/17/console


http://10.3.121.136:8092/default+0218b57a1309c1d9a9bdf94f90057d27/_design/test_design/_view/test_view - with such URI I get {"error":"not_found","reason":"missing"}
http://10.3.121.136:8092/default%2B0218b57a1309c1d9a9bdf94f90057d27/_design/test_design/_view/test_view it's a valid URI


"couchApiBase":"http://10.3.3.206:8092/default%2B0218b57a1309c1d9a9bdf94f90057d27"
Comment by Jeff Morris [ 09/Aug/14 ]
Andrei -

Thanks for the details, i'll investigate further!

-Jeff
Comment by Jeff Morris [ 09/Aug/14 ]
This is the output I get when I write the url to stdout: VIEW QUERY: http://192.168.56.101:8092/default%2Beac6cf9b998068ca424c5b48baaee12a/_design/cities/?

Are you sure your not generating that url (with the "+") from sdkqe-net? Could also be perhaps some environment difference?
Comment by Andrei Baranouski [ 09/Aug/14 ]
added logs in CouchbasePool.cs
        protected IMemcachedNode CreateNode(IPEndPoint endpoint, ISaslAuthenticationProvider auth, Dictionary<string, object> nodeInfo)
        {
            log.WarnFormat("Creating node {0}", this);
            string couchApiBase;
            if (!nodeInfo.TryGetValue("couchApiBase", out couchApiBase) || String.IsNullOrEmpty(couchApiBase))
            {
                return new CouchbaseNode(endpoint, configuration.SocketPool, auth);
            }
            Console.WriteLine("in CreateNode" + couchApiBase);
            Console.WriteLine("in CreateNode" + new Uri(couchApiBase));
            return new CouchbaseNode(endpoint, new Uri(couchApiBase), this.configuration, auth);
        }

[37.33 INFO] (SDKD log:137) in CreateNodehttp://10.3.121.134:8092/default%2Bd2da2be3dfa338b5bea90d3f63a40362
[37.33 INFO] (SDKD log:137) in CreateNodehttp://10.3.121.134:8092/default+d2da2be3dfa338b5bea90d3f63a40362

it means that new Uri() encode "%2B" in "+"

http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng-andrei/22/console

Comment by Jeff Morris [ 09/Aug/14 ]
My output:

in CreateNodehttp://192.168.56.101:8092/default%2Beac6cf9b998068ca424c5b48baaee12a
in CreateNodehttp://192.168.56.102:8092/default%2Beac6cf9b998068ca424c5b48baaee12a

The problem is your probably using an older version of the server and this was fixed at the server level!

My cluster is using:

Version: 3.0.0 Enterprise Edition (build-1098)
Cluster State ID: 048-041-39

I'll try to find the MB. We should probably testing against the latest server builds for 3.0, since a lot of this stuff is probably fixed.

Comment by Andrei Baranouski [ 09/Aug/14 ]
it's strange

in run http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng-andrei/17/consoleFull I used 3.0.0-1105-rel (Beta2 build)
I install CB directly in jenkins run
c:\jenkins\workspace\.net-sdkdclient-ng-andrei\testrunner>python.exe scripts/install.py -i net-sdkdclient-ng-andrei.ini -p version=3.0.0-1105-rel,product=cb,parallel=True,vbuckets=1024

and in this version I see :
[46.97 INFO] (SDKD log:137) http://10.3.3.206:8092/default+0218b57a1309c1d9a9bdf94f90057d27/_design/test_design/_view/test_view?
Comment by Jeff Morris [ 09/Aug/14 ]
Andrei - try this: http://review.couchbase.org/#/c/40485/1

This should be safe and sane fix.
Comment by Andrei Baranouski [ 10/Aug/14 ]
Jeff, it doesn't work because
couchApiBase contains "%2B" but not "+"
couchApiBase = couchApiBase.Replace("+", "%2B"); ie it does not affect the result.

believe that the solution is simple but not obvious)
Comment by Jeff Morris [ 10/Aug/14 ]
The mystery continues: depending upon the version of the .NET Framework you compile against, you get different output from Uri.ToString().

Note that the ".NET Version" output to console isn't indicating which .NET version, but really just the CLR version (my code sucks).

Even stranger, according to this test, when I run the Couchbase.Tests project (it's set to use .NET Framework 4.0), I should be getting the url encoded with "+" - but I don't (lines 325 and 326 of CouchbasePool.cs):

Console.WriteLine("in CreateNode 1: " + couchApiBase);
Console.WriteLine("in CreateNode 2: " + new Uri(couchApiBase));

[log4net.Repository.Hierarchy.Hierarchy]
in CreateNode 1: http://192.168.56.101:8092/default%2Beac6cf9b998068ca424c5b48baaee12a
in CreateNode 2: http://192.168.56.101:8092/default%2Beac6cf9b998068ca424c5b48baaee12a

The best solution is probably to check and convert the url string going out. I'll upload a patch.
Comment by Jeff Morris [ 10/Aug/14 ]
It appears that the System.Uri behaves differently in different types of assemblies. Attached is a zipped solution with two projects: a) a console application w/a main() method and b) client library w/unit test. Each runs roughly the same code, however how System.Uri encodes urls is different.

In the console project:
CLR Version: 4.0.30319.34014
.NET Version: 4.0.0.0
http://10.3.121.134:8092/default%2Bd2da2be3dfa338b5bea90d3f63a40362
http://10.3.121.134:8092/default+d2da2be3dfa338b5bea90d3f63a40362

If you execute the unit test:
CLR Version: 4.0.30319.34014
.NET Version: 4.0.0.0
http://10.3.121.134:8092/default%2Bd2da2be3dfa338b5bea90d3f63a40362
http://10.3.121.134:8092/default%2Bd2da2be3dfa338b5bea90d3f63a40362

This is why the unit tests run correctly, but the situational tests fail (SDKQE-NET is a console app).


Comment by Jeff Morris [ 11/Aug/14 ]
Hi Andrei -

Try this: http://review.couchbase.org/#/c/40511/

It should do the trick.

Thanks,

Jeff
Comment by Andrei Baranouski [ 12/Aug/14 ]
Hi Jeff,

URI is still built like

[357.18 INFO] (SDKD log:137) http://10.3.3.206:8092/default+4aa397ed239d973a8ccac45320abd80c/_design/test_design/_view/test_view?
[357.18 INFO] (SDKD log:137) http://10.3.3.206:8092/default+4aa397ed239d973a8ccac45320abd80c/_design/test_design/_view/test_view?limit=1

http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng-andrei/69/console

to build couchbase-net-client & sdkd-net I use the following settings
set MSBUILD_PATH=C:\Program Files (x86)\MSBuild\12.0\Bin\amd64\;C:\Windows\Microsoft.NET\Framework64\v4.0.30319"
SET PATH=%MSBUILD_PATH%;%PATH%

IP: 10.3.121.13
Comment by Jeff Morris [ 12/Aug/14 ]
Looking deeper into this, I may not be able to fix it easily - in System.Net.WebClient.cs (from decompiled .NET source):

public string BaseAddress
    {
      get
      {
        if (!(this.m_baseAddress == (Uri) null))
          return this.m_baseAddress.ToString();
        else
          return string.Empty;
      }
      set
      {
        if (value != null)
        {
          if (value.Length != 0)
          {
            try
            {
              this.m_baseAddress = new Uri(value);
              return;
            }
            catch (UriFormatException ex)
            {
              throw new ArgumentException(SR.GetString("net_webclient_invalid_baseaddress"), "value", (Exception) ex);
            }
          }
        }
        this.m_baseAddress = (Uri) null;
      }
    }

When you set the BaseAddress string property, it casts it to a Uri (which contains the bug) and calls it's ToString method. It's kinda sad MS didn't/doesn't expose the m_baseAddress field.

Comment by Jeff Morris [ 12/Aug/14 ]
It looks like I would have to rewrite nearly all of the System.Net namespace. I think it might be better to handle this via documentation and support: the .NET 1.X SDK only supports Couchbase Server 3.0 from .NET Framework 4.5 onward.

Comment by Jeff Morris [ 12/Aug/14 ]
Or another approach would be to simply remove the uuid and %2B characters. Does the server require it now?
Comment by Jeff Morris [ 12/Aug/14 ]
Andrei -

Try putting this configuration element into SDK-nets App.Config file:

<uri>
    <iriParsing enabled="true"/>
</uri>

After you have done this, run the tests again. If this works, I'll try a programmatic solution (if it exists).

Thanks,

Jeff
Comment by Jeff Morris [ 12/Aug/14 ]
New patchset: git fetch ssh://jmorris@review.couchbase.org:29418/couchbase-net-client refs/changes/11/40511/5 && git checkout FETCH_HEAD

I tested locally on clients using .NET 4, 4.5 and 4.5.1. This is the "programmatic" implementation mentioned in previous comment.
Comment by Andrei Baranouski [ 13/Aug/14 ]
awesome, it works! http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng-andrei/71/




[NCBC-576] System.ArgumentOutOfRangeException in CanRetryOperation in case Timed out error/network issues Created: 06/Aug/14  Updated: 13/Aug/14  Resolved: 13/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive log.zip     Zip Archive NCBC-576.log.zip    

 Description   
SHA: 940183971ea8d0570033ed28bba2e7c84ddc2b71

I run the tests through the vpn and sdkd tests probably are not ideal but...

c:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,14 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/mode=out
    workload=HybridWorkloadGroup
    rebalance/count=2
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-06-14\047299\d8869f6da67dff794890340838f386d3

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-mode out --workload HybridWorkloadGroup
 \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,57 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,91 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,92 INFO] (RunContext run:124) Starting cluster and driver
[1,92 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,92 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1,92 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[2,09 INFO] (SDKD log:137) --- Logging Self-Test ---
[2,09 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[2,09 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[2,09 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[2,10 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[2,10 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,44 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,44 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,57 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[9,02 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.145:8091,ns_1@10.3.4.145>
[9,51 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.146:8091,ns_1@10.3.4.146>
[10,00 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[10,50 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,50 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[12,56 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[12,56 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[12,57 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[12,57 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[12,58 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[12,58 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[12,58 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[12,58 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[13,42 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@54db08f1
[13,42 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@782831bc
[13,42 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@13d3589e
[13,43 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@2b9d24b
[13,43 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@2548d5b2
[22,50 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[28,06 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[37,69 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[42,32 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[43,29 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[43,29 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[43,54 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[43,79 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[52,92 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[55,76 INFO] (RunContext run:143) Driver and cluster initialized
[56,01 INFO] (RunContext call:167) Running scenario..
[56,01 INFO] (Scenario run:72) Starting RAMP phase
[56,01 INFO] (Workload setupDesign:63) Creating design test_design
[57,28 INFO] (Workload setupDesign:80) Design creation done
[57,29 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[57,29 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[57,30 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[57,31 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[57,33 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[57,33 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[57,36 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[57,36 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[64,97 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[64,98 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, O
ptions={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[64,98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[64,99 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[64,99 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[64,99 INFO] (SDKD log:137) "DS": {
[64,99 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[64,99 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[65,00 INFO] (SDKD log:137) "Count": 1000,
[65,00 INFO] (SDKD log:137) "Continuous": false,
[65,00 INFO] (SDKD log:137) "VSize": 512,
[65,00 INFO] (SDKD log:137) "Repeat": "rep",
[65,00 INFO] (SDKD log:137) "KSize": 12
[65,01 INFO] (SDKD log:137) },
[65,01 INFO] (SDKD log:137) "Schema": {
[65,01 INFO] (SDKD log:137) "InflateContent": "meh",
[65,01 INFO] (SDKD log:137) "InflateLevel": 40
[65,02 INFO] (SDKD log:137) },
[65,02 INFO] (SDKD log:137) "Options": {
[65,02 INFO] (SDKD log:137) "DelayMax": 0,
[65,03 INFO] (SDKD log:137) "DelayMin": 0,
[65,03 INFO] (SDKD log:137) "IterWait": 1,
[65,03 INFO] (SDKD log:137) "TimeRes": 0
[65,03 INFO] (SDKD log:137) }
[65,04 INFO] (SDKD log:137) }
[301,88 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[301,89 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[301,89 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[301,89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[301,89 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[301,89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[301,90 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[301,90 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[301,90 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[301,90 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[301,91 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[301,91 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[308,11 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[308,11 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40},
Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[308,12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[308,12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[308,12 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[308,12 INFO] (SDKD log:137) "DS": {
[308,12 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[308,13 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[308,13 INFO] (SDKD log:137) "Count": 1000,
[308,13 INFO] (SDKD log:137) "Continuous": false,
[308,14 INFO] (SDKD log:137) "VSize": 512,
[308,14 INFO] (SDKD log:137) "Repeat": "rep",
[308,14 INFO] (SDKD log:137) "KSize": 12
[308,14 INFO] (SDKD log:137) },
[308,14 INFO] (SDKD log:137) "Schema": {
[308,14 INFO] (SDKD log:137) "InflateContent": "meh",
[308,15 INFO] (SDKD log:137) "InflateLevel": 40
[308,15 INFO] (SDKD log:137) },
[308,15 INFO] (SDKD log:137) "Options": {
[308,15 INFO] (SDKD log:137) "DelayMax": 0,
[308,15 INFO] (SDKD log:137) "DelayMin": 0,
[308,16 INFO] (SDKD log:137) "IterWait": 1,
[308,16 INFO] (SDKD log:137) "TimeRes": 0
[308,16 INFO] (SDKD log:137) }
[308,17 INFO] (SDKD log:137) }
[510,64 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[510,64 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@102.5
[510,65 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[510,65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[510,65 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[510,65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[510,65 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[510,65 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[510,66 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[510,66 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[510,66 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[510,66 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[516,85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[516,86 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=tes
t_view}
[516,86 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[516,87 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[516,87 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[516,87 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[516,87 INFO] (SDKD log:137) "DS": {},
[516,87 INFO] (SDKD log:137) "ViewParameters": {},
[516,87 INFO] (SDKD log:137) "DesignName": "test_design",
[516,88 INFO] (SDKD log:137) "Options": {
[516,88 INFO] (SDKD log:137) "DelayMax": 10,
[516,88 INFO] (SDKD log:137) "DelayMin": 1,
[516,88 INFO] (SDKD log:137) "IterWait": 1,
[516,89 INFO] (SDKD log:137) "ViewQueryCount": -1,
[516,89 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[516,89 INFO] (SDKD log:137) "TimeRes": 1
[516,89 INFO] (SDKD log:137) },
[516,89 INFO] (SDKD log:137) "ViewName": "test_view"
[516,89 INFO] (SDKD log:137) }
[516,90 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[516,90 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[516,90 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[522,58 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9
[522,58 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[522,59 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[522,59 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[522,59 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[522,59 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[522,59 INFO] (SDKD log:137) "DS": {},
[522,60 INFO] (SDKD log:137) "ViewParameters": {},
[522,60 INFO] (SDKD log:137) "DesignName": "test_design",
[522,61 INFO] (SDKD log:137) "Options": {
[522,61 INFO] (SDKD log:137) "DelayMax": 10,
[522,61 INFO] (SDKD log:137) "DelayMin": 1,
[522,61 INFO] (SDKD log:137) "IterWait": 1,
[522,61 INFO] (SDKD log:137) "ViewQueryCount": -1,
[522,61 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[522,62 INFO] (SDKD log:137) "TimeRes": 1
[522,62 INFO] (SDKD log:137) },
[522,62 INFO] (SDKD log:137) "ViewName": "test_view"
[522,62 INFO] (SDKD log:137) }
[522,63 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[522,63 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[522,63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[528,32 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11
[528,32 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[528,33 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[528,33 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[528,33 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[528,34 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[528,34 INFO] (SDKD log:137) "DS": {},
[528,34 INFO] (SDKD log:137) "ViewParameters": {},
[528,34 INFO] (SDKD log:137) "DesignName": "test_design",
[528,35 INFO] (SDKD log:137) "Options": {
[528,35 INFO] (SDKD log:137) "DelayMax": 10,
[528,35 INFO] (SDKD log:137) "DelayMin": 1,
[528,35 INFO] (SDKD log:137) "IterWait": 1,
[528,35 INFO] (SDKD log:137) "ViewQueryCount": -1,
[528,36 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[528,36 INFO] (SDKD log:137) "TimeRes": 1
[528,36 INFO] (SDKD log:137) },
[528,36 INFO] (SDKD log:137) "ViewName": "test_view"
[528,36 INFO] (SDKD log:137) }
[528,37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[528,37 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[528,37 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[533,76 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13
[533,76 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[533,76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[533,76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[533,77 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[533,77 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[533,77 INFO] (SDKD log:137) "DS": {},
[533,77 INFO] (SDKD log:137) "ViewParameters": {},
[533,77 INFO] (SDKD log:137) "DesignName": "test_design",
[533,77 INFO] (SDKD log:137) "Options": {
[533,78 INFO] (SDKD log:137) "DelayMax": 10,
[533,78 INFO] (SDKD log:137) "DelayMin": 1,
[533,78 INFO] (SDKD log:137) "IterWait": 1,
[533,78 INFO] (SDKD log:137) "ViewQueryCount": -1,
[533,78 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[533,78 INFO] (SDKD log:137) "TimeRes": 1
[533,79 INFO] (SDKD log:137) },
[533,79 INFO] (SDKD log:137) "ViewName": "test_view"
[533,79 INFO] (SDKD log:137) }
[533,79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[533,79 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[533,80 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[539,57 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15
[539,57 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[539,58 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[539,58 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[539,58 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[539,58 INFO] (Workload call:112) Preloading items into cluster before workload
[539,59 INFO] (SDKD log:137) "DS": {},
[539,59 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[539,59 INFO] (SDKD log:137) "ViewParameters": {},
[539,59 INFO] (SDKD log:137) "DesignName": "test_design",
[539,59 INFO] (SDKD log:137) "Options": {
[539,60 INFO] (SDKD log:137) "DelayMax": 10,
[539,60 INFO] (SDKD log:137) "DelayMin": 1,
[539,60 INFO] (SDKD log:137) "IterWait": 1,
[539,60 INFO] (SDKD log:137) "ViewQueryCount": -1,
[539,61 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[539,61 INFO] (SDKD log:137) "TimeRes": 1
[539,61 INFO] (SDKD log:137) },
[539,61 INFO] (SDKD log:137) "ViewName": "test_view"
[539,61 INFO] (SDKD log:137) }
[539,62 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[539,62 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[539,62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[544,94 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17
[544,94 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, T
imeRes=0}}
[544,95 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[544,95 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[544,95 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[544,95 INFO] (SDKD log:137) "DS": {
[544,96 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[544,96 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[544,96 INFO] (SDKD log:137) "Count": 15000,
[544,96 INFO] (SDKD log:137) "Continuous": false,
[544,96 INFO] (SDKD log:137) "VSize": 128,
[544,97 INFO] (SDKD log:137) "Repeat": "REP",
[544,97 INFO] (SDKD log:137) "KSize": 32
[544,97 INFO] (SDKD log:137) },
[544,97 INFO] (SDKD log:137) "Options": {
[544,97 INFO] (SDKD log:137) "DelayMax": 0,
[544,98 INFO] (SDKD log:137) "DelayMin": 0,
[544,98 INFO] (SDKD log:137) "IterWait": 100,
[544,98 INFO] (SDKD log:137) "TimeRes": 0
[544,98 INFO] (SDKD log:137) }
[544,98 INFO] (SDKD log:137) }
[547,24 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[573,65 INFO] (SDKD log:137)
[573,66 INFO] (SDKD log:137) Необработанное исключение: System.ArgumentOutOfRangeException: Заданный аргумент находится вне диапазона допустимых значений.
[573,66 INFO] (SDKD log:137) at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\Couc
hbaseBucket.cs:line 180
[573,66 INFO] (SDKD log:137) at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 102
[573,66 INFO] (SDKD log:137) at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 272
[573,67 INFO] (SDKD log:137) at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) at c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77
[573,67 INFO] (SDKD log:137) at Sdkd.Commands.KVCommand.Run() at c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25
[573,67 INFO] (SDKD log:137) at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) at c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231
[573,67 INFO] (SDKD log:137) at SdkdConsole.Daemon.Handle.Run() at c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153
[573,68 INFO] (SDKD log:137) at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
[573,68 INFO] (SDKD log:137) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[573,68 INFO] (SDKD log:137) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[573,68 INFO] (SDKD log:137) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
[573,69 INFO] (SDKD log:137) at System.Threading.ThreadHelper.ThreadStart()
[3758,06 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.7.0_60]
        at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_60]
[3758,09 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.19
[3758,09 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.7.0_60]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.7.0_60]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.7.0_60]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.7.0_60]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.7.0_60]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]



please take a look on client logs related to SimpleKeyREP51REP51REP51REP51REP51REP51REP51 key


2014-08-06 16:17:50,486 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: 74bf91e3-557b-471a-9273-6e836b9bb1d2 on 10.3.4.145:11210 - b61217ce-5d73-411f-b9f6-d7f55928c393
2014-08-06 16:17:50,486 [27] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation doesn't support retries for key SimpleKeyREP50REP50REP50REP50REP50REP50REP50
2014-08-06 16:17:50,486 [27] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation for key SimpleKeyREP50REP50REP50REP50REP50REP50REP50 failed after 0 retries. Reason: Timed out.
2014-08-06 16:17:50,486 [27] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#44 on thread 27
2014-08-06 16:17:50,486 [27] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 180 for key SimpleKeyREP51REP51REP51REP51REP51REP51REP51 - rev0
2014-08-06 16:17:50,486 [27] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP51REP51REP51REP51REP51REP51REP51 using server 10.3.4.144:11210
2014-08-06 16:17:50,486 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: 840581c8-e22d-43c8-b7fb-e0b43cdff84c | 10.3.4.144:11210 | [0, 2] - 4596dec1-2c02-41bc-a1ef-ae9b058d0fcd - Disposed: False
2014-08-06 16:17:50,486 [27] INFO Couchbase.IO.ConnectionBase - Sending key SimpleKeyREP51REP51REP51REP51REP51REP51REP51 using 180 on 10.3.4.144:11210
2014-08-06 16:17:50,661 [12] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:50,695 [22] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.147:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:50,927 [21] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:50,928 [26] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.147:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:50,928 [12] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.145:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:50,932 [17] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:50,987 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: 840581c8-e22d-43c8-b7fb-e0b43cdff84c on 10.3.4.144:11210 - 4596dec1-2c02-41bc-a1ef-ae9b058d0fcd
2014-08-06 16:17:50,987 [27] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation doesn't support retries for key SimpleKeyREP51REP51REP51REP51REP51REP51REP51
2014-08-06 16:17:50,987 [27] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation for key SimpleKeyREP51REP51REP51REP51REP51REP51REP51 failed after 0 retries. Reason: Timed out.
2014-08-06 16:17:50,987 [27] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#44 on thread 27
2014-08-06 16:17:50,987 [27] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 354 for key SimpleKeyREP52REP52REP52REP52REP52REP52REP52 - rev0
2014-08-06 16:17:50,987 [27] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP52REP52REP52REP52REP52REP52REP52 using server 10.3.4.147:11210
2014-08-06 16:17:50,987 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: dddae2e9-fcca-44a7-a96b-6fb075d86c8e | 10.3.4.147:11210 | [0, 2] - 35f5a35e-087a-421d-a8ed-2d450d3b1cc2 - Disposed: False
2014-08-06 16:17:50,987 [27] INFO Couchbase.IO.ConnectionBase - Sending key SimpleKeyREP52REP52REP52REP52REP52REP52REP52 using 354 on 10.3.4.147:11210
2014-08-06 16:17:51,169 [21] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.145:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,173 [22] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.145:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,180 [17] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,419 [26] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,424 [12] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,440 [17] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.147:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,487 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: dddae2e9-fcca-44a7-a96b-6fb075d86c8e on 10.3.4.147:11210 - 35f5a35e-087a-421d-a8ed-2d450d3b1cc2
2014-08-06 16:17:51,487 [27] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation doesn't support retries for key SimpleKeyREP52REP52REP52REP52REP52REP52REP52
2014-08-06 16:17:51,487 [27] DEBUG Couchbase.Core.Buckets.CouchbaseBucket - Operation for key SimpleKeyREP52REP52REP52REP52REP52REP52REP52 failed after 0 retries. Reason: Timed out.
2014-08-06 16:17:51,487 [27] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#44 on thread 27
2014-08-06 16:17:51,487 [27] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 255 for key SimpleKeyREP53REP53REP53REP53REP53REP53REP53 - rev0
2014-08-06 16:17:51,487 [27] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP53REP53REP53REP53REP53REP53REP53 using server 10.3.4.147:11210
2014-08-06 16:17:51,487 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: dddae2e9-fcca-44a7-a96b-6fb075d86c8e | 10.3.4.147:11210 | [0, 2] - 35f5a35e-087a-421d-a8ed-2d450d3b1cc2 - Disposed: False
2014-08-06 16:17:51,487 [27] INFO Couchbase.IO.ConnectionBase - Sending key SimpleKeyREP53REP53REP53REP53REP53REP53REP53 using 255 on 10.3.4.147:11210
2014-08-06 16:17:51,670 [21] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,671 [22] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,677 [12] DEBUG Couchbase.Views.ViewQuery - http://10.3.4.144:8092/default/_design/test_design/_view/test_view?
2014-08-06 16:17:51,743 [27] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: dddae2e9-fcca-44a7-a96b-6fb075d86c8e on 10.3.4.147:11210 - 35f5a35e-087a-421d-a8ed-2d450d3b1cc2


cleint logs are attached


 Comments   
Comment by Jeff Morris [ 06/Aug/14 ]
Fix is here: 0d26d7aae6ddd6510634bd90965043a7b0e9e2f3 (in gerrit)

Comment by Andrei Baranouski [ 12/Aug/14 ]
Hi Jeff,

I tested latest code based on gerrit

$ git log
commit 1f7de92ea61547813982f315ca114e1d2be33ebc
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 15:31:06 2014 -0700

    NCBC-542: Provide Observe method on IBucket (WIP)

    Change-Id: Iecfbc02490b9ce237d6fbe16619e5aa252566bd7

commit 2c918b4a5b74ca35d8ddfb24bb5a88faa5b057be
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 12:31:12 2014 -0700

    NCBC-576: Do not retry operation when unknown status is returned.

    Change-Id: Ic9afeb1373d9e7804cad3c2df0521b162956326d

commit 1bd6684e6b3cba22c6fb03be01cfc3434c60cc3f
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Wed Aug 6 10:56:39 2014 -0700

    NCBC-575: Ensure finalization only occurs on non-disposed connections

    Change-Id: I25f7e718936d671a1227da4ce1e346fa2129dd81

commit 940183971ea8d0570033ed28bba2e7c84ddc2b71
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Fri Aug 1 10:23:50 2014 -0700

    Refactor CouchbaseBucket.LoadConfig and make it a proper IConfigInfo member

    Change-Id: I99c895cd57966e0e5892a5c750dd68f0bad6d954
    Reviewed-on: http://review.couchbase.org/40187
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Mark Nunberg <mnunberg@haskalah.org>



the problem has been reproduced when my vpn dropped:

[538,82 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[1114,81 INFO] (SDKD log:137)
[1114,83 INFO] (SDKD log:137) Необработанное исключение: System.InvalidOperationException: Последовательность не содержит соответствующий элемент
[1114,84 INFO] (SDKD log:137) в System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
[1114,84 INFO] (SDKD log:137) в Couchbase.Configuration.ConfigContextBase.GetServer() в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:строка 172
[1114,84 INFO] (SDKD log:137) в Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:строка 193
[1114,84 INFO] (SDKD log:137) в Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\Cou
chbaseBucket.cs:строка 178
[1114,84 INFO] (SDKD log:137) в Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:строка 103
[1114,85 INFO] (SDKD log:137) в Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:строка 310
[1114,85 INFO] (SDKD log:137) в Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) в c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\SetCommand.cs:строка 77
[1114,85 INFO] (SDKD log:137) в Sdkd.Commands.KVCommand.Run() в c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:строка 25
[1114,85 INFO] (SDKD log:137) в SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) в c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:строка 231
[1114,85 INFO] (SDKD log:137) в SdkdConsole.Daemon.Handle.Run() в c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:строка 153
[1114,86 INFO] (SDKD log:137) в System.Threading.ThreadHelper.ThreadStart_Context(Object state)
[1114,86 INFO] (SDKD log:137) в System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[1114,86 INFO] (SDKD log:137) в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[1114,86 INFO] (SDKD log:137) в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
[1114,86 INFO] (SDKD log:137) в System.Threading.ThreadHelper.ThreadStart()

new client logs attached
Comment by Jeff Morris [ 12/Aug/14 ]
The problem with testing the latest code on gerrit is that your probably not testing what you think you are. This is what you need to test:

git fetch ssh://jmorris@review.couchbase.org:29418/couchbase-net-client refs/changes/47/40347/1 && git checkout FETCH_HEAD

For this specific ticket, you only want to verify that that a ArgumentOutOfRangeException is not thrown within CanRetryOperation.

Also, this is a different issue:

[1114,83 INFO] (SDKD log:137) Необработанное исключение: System.InvalidOperationException: Последовательность не содержит соответствующий элемент
[1114,84 INFO] (SDKD log:137) в System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
[1114,84 INFO] (SDKD log:137) в Couchbase.Configuration.ConfigContextBase.GetServer() в c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:строка 172

System.ArgumentOutOfRangeException vs System.InvalidOperationException. This should be a seperate ticket.

>>the problem has been reproduced when my vpn dropped:

This will cause many problems!
Comment by Jeff Morris [ 12/Aug/14 ]
New ticket for the InvalidOperationException: https://www.couchbase.com/issues/browse/NCBC-581

I am pretty sure this ticket (NCBC-576) can be closed.
Comment by Andrei Baranouski [ 13/Aug/14 ]
closed based on Jeff's comment




[NCBC-575] Cannot access a disposed object in ConnectionPool`1.Finalize() Created: 06/Aug/14  Updated: 21/Aug/14  Resolved: 06/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive log.zip    

 Description   
SHA: 940183971ea8d0570033ed28bba2e7c84ddc2b71

c:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,14 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/mode=out
    workload=HybridWorkloadGroup
    rebalance/count=2
    testcase=RebalanceScenario
    Logging to c:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-06-14\038335\7c3fdd009753895e6cb649625d7e8267

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-mode out --workload HybridWorkloadGroup
 \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145
 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,55 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,87 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,88 INFO] (RunContext run:124) Starting cluster and driver
[1,88 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now
[1,88 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,89 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[2,08 INFO] (SDKD log:137) --- Logging Self-Test ---
[2,08 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[2,08 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[2,08 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[2,08 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[2,09 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,41 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,41 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,63 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[8,77 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.145:8091,ns_1@10.3.4.145>
[9,26 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.146:8091,ns_1@10.3.4.146>
[9,84 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147>
[10,34 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[10,34 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[12,37 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091
[12,38 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091
[12,38 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091
[12,39 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091
[12,41 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146
[12,41 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144
[12,41 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145
[12,42 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147
[13,25 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@5d458f36
[13,26 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@3456dafb
[13,26 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@9b981da
[13,26 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@5c921914
[13,27 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@40439a1f
[26,77 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091
[31,48 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091
[37,64 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091
[42,31 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[43,29 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[43,29 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[43,54 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[43,79 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[53,15 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[56,05 INFO] (RunContext run:143) Driver and cluster initialized
[56,31 INFO] (RunContext call:167) Running scenario..
[56,31 INFO] (Scenario run:72) Starting RAMP phase
[56,32 INFO] (Workload setupDesign:63) Creating design test_design
[57,60 INFO] (Workload setupDesign:80) Design creation done
[57,60 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[57,61 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[57,61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[57,62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[57,62 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[57,63 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[57,64 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[57,64 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[65,96 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[65,96 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, O
ptions={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[65,96 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[65,96 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[65,96 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[65,97 INFO] (SDKD log:137) "DS": {
[65,97 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[65,97 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[65,97 INFO] (SDKD log:137) "Count": 1000,
[65,97 INFO] (SDKD log:137) "Continuous": false,
[65,98 INFO] (SDKD log:137) "VSize": 512,
[65,98 INFO] (SDKD log:137) "Repeat": "rep",
[65,98 INFO] (SDKD log:137) "KSize": 12
[65,98 INFO] (SDKD log:137) },
[65,98 INFO] (SDKD log:137) "Schema": {
[65,99 INFO] (SDKD log:137) "InflateContent": "meh",
[65,99 INFO] (SDKD log:137) "InflateLevel": 40
[65,99 INFO] (SDKD log:137) },
[65,99 INFO] (SDKD log:137) "Options": {
[65,99 INFO] (SDKD log:137) "DelayMax": 0,
[65,99 INFO] (SDKD log:137) "DelayMin": 0,
[66,00 INFO] (SDKD log:137) "IterWait": 1,
[66,00 INFO] (SDKD log:137) "TimeRes": 0
[66,00 INFO] (SDKD log:137) }
[66,00 INFO] (SDKD log:137) }
[308,72 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[308,73 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2
[308,73 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[308,73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[308,73 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[308,73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[308,74 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[308,74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[308,74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[308,74 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[308,75 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[308,75 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[315,95 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[315,95 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40},
Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[315,95 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD
[315,95 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[315,96 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[315,96 INFO] (SDKD log:137) "DS": {
[315,96 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed",
[315,96 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed",
[315,96 INFO] (SDKD log:137) "Count": 1000,
[315,97 INFO] (SDKD log:137) "Continuous": false,
[315,97 INFO] (SDKD log:137) "VSize": 512,
[315,97 INFO] (SDKD log:137) "Repeat": "rep",
[315,97 INFO] (SDKD log:137) "KSize": 12
[315,98 INFO] (SDKD log:137) },
[315,98 INFO] (SDKD log:137) "Schema": {
[315,98 INFO] (SDKD log:137) "InflateContent": "meh",
[315,98 INFO] (SDKD log:137) "InflateLevel": 40
[315,98 INFO] (SDKD log:137) },
[315,98 INFO] (SDKD log:137) "Options": {
[315,99 INFO] (SDKD log:137) "DelayMax": 0,
[315,99 INFO] (SDKD log:137) "DelayMin": 0,
[315,99 INFO] (SDKD log:137) "IterWait": 1,
[315,99 INFO] (SDKD log:137) "TimeRes": 0
[315,99 INFO] (SDKD log:137) }
[315,99 INFO] (SDKD log:137) }
[548,71 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {}
[548,71 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@102.5
[548,71 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[548,71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE
[548,71 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[548,71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {}
[548,72 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket -788777045
[548,72 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()-788777045
[548,72 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false
[548,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[548,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[548,72 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[555,52 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[555,52 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=tes
t_view}
[555,53 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[555,53 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[555,53 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[555,53 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[555,53 INFO] (SDKD log:137) "DS": {},
[555,53 INFO] (SDKD log:137) "ViewParameters": {},
[555,54 INFO] (SDKD log:137) "DesignName": "test_design",
[555,54 INFO] (SDKD log:137) "Options": {
[555,54 INFO] (SDKD log:137) "DelayMax": 10,
[555,54 INFO] (SDKD log:137) "DelayMin": 1,
[555,54 INFO] (SDKD log:137) "IterWait": 1,
[555,55 INFO] (SDKD log:137) "ViewQueryCount": -1,
[555,55 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[555,55 INFO] (SDKD log:137) "TimeRes": 1
[555,55 INFO] (SDKD log:137) },
[555,55 INFO] (SDKD log:137) "ViewName": "test_view"
[555,56 INFO] (SDKD log:137) }
[555,56 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[555,56 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[555,56 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[561,36 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9
[561,37 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[561,37 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[561,38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[561,38 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[561,38 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[561,38 INFO] (SDKD log:137) "DS": {},
[561,38 INFO] (SDKD log:137) "ViewParameters": {},
[561,39 INFO] (SDKD log:137) "DesignName": "test_design",
[561,39 INFO] (SDKD log:137) "Options": {
[561,39 INFO] (SDKD log:137) "DelayMax": 10,
[561,39 INFO] (SDKD log:137) "DelayMin": 1,
[561,39 INFO] (SDKD log:137) "IterWait": 1,
[561,40 INFO] (SDKD log:137) "ViewQueryCount": -1,
[561,40 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[561,40 INFO] (SDKD log:137) "TimeRes": 1
[561,40 INFO] (SDKD log:137) },
[561,40 INFO] (SDKD log:137) "ViewName": "test_view"
[561,41 INFO] (SDKD log:137) }
[561,41 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[561,41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[561,41 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[566,92 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11
[566,92 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[566,93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[566,93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[566,93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[566,93 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[566,93 INFO] (SDKD log:137) "DS": {},
[566,93 INFO] (SDKD log:137) "ViewParameters": {},
[566,94 INFO] (SDKD log:137) "DesignName": "test_design",
[566,94 INFO] (SDKD log:137) "Options": {
[566,94 INFO] (SDKD log:137) "DelayMax": 10,
[566,94 INFO] (SDKD log:137) "DelayMin": 1,
[566,94 INFO] (SDKD log:137) "IterWait": 1,
[566,95 INFO] (SDKD log:137) "ViewQueryCount": -1,
[566,95 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[566,95 INFO] (SDKD log:137) "TimeRes": 1
[566,95 INFO] (SDKD log:137) },
[566,95 INFO] (SDKD log:137) "ViewName": "test_view"
[566,95 INFO] (SDKD log:137) }
[566,96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[566,96 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[566,96 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[572,34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13
[572,34 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[572,34 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[572,34 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[572,35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[572,35 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[572,35 INFO] (SDKD log:137) "DS": {},
[572,35 INFO] (SDKD log:137) "ViewParameters": {},
[572,35 INFO] (SDKD log:137) "DesignName": "test_design",
[572,36 INFO] (SDKD log:137) "Options": {
[572,36 INFO] (SDKD log:137) "DelayMax": 10,
[572,36 INFO] (SDKD log:137) "DelayMin": 1,
[572,36 INFO] (SDKD log:137) "IterWait": 1,
[572,36 INFO] (SDKD log:137) "ViewQueryCount": -1,
[572,37 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[572,37 INFO] (SDKD log:137) "TimeRes": 1
[572,37 INFO] (SDKD log:137) },
[572,37 INFO] (SDKD log:137) "ViewName": "test_view"
[572,37 INFO] (SDKD log:137) }
[572,38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[572,38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[572,38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[578,37 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15
[578,37 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=te
st_view}
[578,38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY
[578,38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[578,38 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[578,38 INFO] (Workload call:112) Preloading items into cluster before workload
[578,39 INFO] (SDKD log:137) "DS": {},
[578,39 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3.4.147",8091]]}, Hostname=10.3.4.144}
[578,39 INFO] (SDKD log:137) "ViewParameters": {},
[578,39 INFO] (SDKD log:137) "DesignName": "test_design",
[578,40 INFO] (SDKD log:137) "Options": {
[578,40 INFO] (SDKD log:137) "DelayMax": 10,
[578,40 INFO] (SDKD log:137) "DelayMin": 1,
[578,40 INFO] (SDKD log:137) "IterWait": 1,
[578,40 INFO] (SDKD log:137) "ViewQueryCount": -1,
[578,41 INFO] (SDKD log:137) "ViewQueryDelay": 0,
[578,41 INFO] (SDKD log:137) "TimeRes": 1
[578,41 INFO] (SDKD log:137) },
[578,41 INFO] (SDKD log:137) "ViewName": "test_view"
[578,41 INFO] (SDKD log:137) }
[578,42 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[578,42 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[578,42 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew
[583,98 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17
[583,98 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, T
imeRes=0}}
[583,99 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET
[583,99 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {
[583,99 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED",
[583,99 INFO] (SDKD log:137) "DS": {
[583,99 INFO] (SDKD log:137) "KSeed": "SimpleKey",
[584,00 INFO] (SDKD log:137) "VSeed": "SimpleValue",
[584,00 INFO] (SDKD log:137) "Count": 15000,
[584,00 INFO] (SDKD log:137) "Continuous": false,
[584,00 INFO] (SDKD log:137) "VSize": 128,
[584,01 INFO] (SDKD log:137) "Repeat": "REP",
[584,01 INFO] (SDKD log:137) "KSize": 32
[584,01 INFO] (SDKD log:137) },
[584,01 INFO] (SDKD log:137) "Options": {
[584,01 INFO] (SDKD log:137) "DelayMax": 0,
[584,02 INFO] (SDKD log:137) "DelayMin": 0,
[584,02 INFO] (SDKD log:137) "IterWait": 100,
[584,02 INFO] (SDKD log:137) "TimeRes": 0
[584,02 INFO] (SDKD log:137) }
[584,02 INFO] (SDKD log:137) }
[587,10 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[637,81 INFO] (SDKD log:137)
[637,81 INFO] (SDKD log:137) Необработанное исключение: System.ObjectDisposedException: Cannot access a disposed object
[637,82 INFO] (SDKD log:137) Имя объекта: "Объект ThreadLocal был удален.".
[637,82 INFO] (SDKD log:137) at System.Threading.ThreadLocal`1.GetValueSlow()
[637,82 INFO] (SDKD log:137) at System.Threading.ThreadLocal`1.get_Value()
[637,82 INFO] (SDKD log:137) at System.Collections.Concurrent.ConcurrentBag`1.GetThreadList(Boolean forceCreate)
[637,82 INFO] (SDKD log:137) at System.Collections.Concurrent.ConcurrentBag`1.TryTakeOrPeek(T& result, Boolean take)
[637,83 INFO] (SDKD log:137) at System.Collections.Concurrent.ConcurrentBag`1.TryTake(T& result)
[637,83 INFO] (SDKD log:137) at Couchbase.IO.ConnectionPool`1.Dispose(Boolean disposing) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\ConnectionPool.cs:line 162
[637,83 INFO] (SDKD log:137) at Couchbase.IO.ConnectionPool`1.Finalize() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\ConnectionPool.cs:строка 173
[6551,70 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.7.0_60]
        at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_60]
[6551,73 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.19
[6551,73 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.7.0_60]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.7.0_60]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.7.0_60]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.7.0_60]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.7.0_60]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]

client logs are attached

 Comments   
Comment by Jeff Morris [ 06/Aug/14 ]
Sha (from gerrit): 6fcc6ad5c2269b5ff8cd152bfc54d6990c598e1b

There is two things going on here, one is the bug fixed in the sha1. The 2nd is that I still see a lot of finalization happening in the logs; this means that the sdkd-net is not managing the cluster objects correctly. For example:

        Line 368: 2014-08-06 11:54:49,854 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 368: 2014-08-06 11:54:49,854 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 7409: 2014-08-06 11:56:56,247 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 7409: 2014-08-06 11:56:56,247 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 14352: 2014-08-06 12:00:35,071 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
...

        Line 171421: 2014-08-06 12:42:57,004 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 171421: 2014-08-06 12:42:57,004 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 172455: 2014-08-06 12:43:02,007 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster
Line 172455: 2014-08-06 12:43:02,007 [2] DEBUG Couchbase.CouchbaseCluster - Finalizing CouchbaseCluster

I do see that the CouchbaseBucket objects are being disposed, but the CouchbaseCluster object is being leaked. This needs to be fixed in the sdqd-net handle. Once again the difference is between deterministic finalization (calling Dispose()) and non-deterministic (relying on finalization) . You cannot control or manage finalization, which means will likely see "odd" and unexpected things happening as the tests are run and in the middle of a run, the object you are using is reclaimed.

-Jeff
Comment by Jeff Morris [ 06/Aug/14 ]
See previous msg and gerrit SHA.




[NCBC-574] Collection was modified; enumeration operation may not execute in ConfigContextBase.Dispose Created: 05/Aug/14  Updated: 21/Aug/14  Resolved: 14/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Andrei Baranouski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File log.txt    

 Description   
sdkd-net is still under development to fix http://www.couchbase.com/issues/browse/NCBC-567

but i think such problems should not occur on the client side


C:\jenkins\workspace-net2\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload
[0,14 INFO] (BRun run:435) Initializing history database

============================================================
Running Rb2Out-HYBRID
    rebalance/mode=out
    workload=HybridWorkloadGroup
    rebalance/count=2
    testcase=RebalanceScenario
    Logging to C:\jenkins\workspace-net2\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1060\Rb2Out-HYBRID\08-05-14\040766\93f5df0d2920de25364b1b4eb9118243

    To re-run the test, copy/paste the following into the shell.
    You may also copy/paste (except the first line) into an argfile
    8<----------------------------------------
    ./stester \
 \ --rebalance-mode out --workload HybridWorkloadGroup
 \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec
 \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port
 \ 8675 --cluster_node=10.3.121.134 --cluster_node=10.3.121.135
 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206
        --cluster_ssh-username=root --cluster_ssh-password=couchbase
    ---------------------------------------->8
[1,49 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution
[1,77 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds.
[1,78 INFO] (RunContext run:124) Starting cluster and driver
[1,78 INFO] (CBCluster startCluster:360) Node http://10.3.121.134:8091 is master now
[1,78 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'
[1,78 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations..
[1,97 INFO] (SDKD log:137) --- Logging Self-Test ---
[1,98 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message
[1,98 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message
[1,98 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message
[1,98 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message
[1,98 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675
[2,33 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[2,33 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0
[2,51 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.34014"}}
[8,05 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.135:8091,ns_1@10.3.121.135>
[8,55 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.136:8091,ns_1@10.3.121.136>
[9,05 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.3.206:8091,ns_1@10.3.3.206>
[9,56 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[9,57 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84]
[11,58 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.136:8091
[11,59 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.135:8091
[11,59 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.134:8091
[11,59 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.3.206:8091
[11,61 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.3.206
[11,61 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.134
[11,61 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.136
[11,61 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.135
[12,45 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@1bd5211
[12,45 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@3456dafb
[12,46 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@7bac9f86
[12,46 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@76be3008
[12,46 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@53ac4ca1
[28,46 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.135:8091
[33,13 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.136:8091
[43,50 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.3.206:8091
[51,51 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance
[52,51 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete
[52,51 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups
[52,76 INFO] (CBCluster setupMainBucket:209) Creating bucket default
[53,01 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted
[60,67 INFO] (CBCluster waitForBucketReady:203) Bucket creation done
[63,56 INFO] (RunContext run:143) Driver and cluster initialized
[63,82 INFO] (RunContext call:167) Running scenario..
[63,82 INFO] (Scenario run:72) Starting RAMP phase
[63,83 INFO] (Workload setupDesign:63) Creating design test_design
[65,11 INFO] (Workload setupDesign:80) Design creation done
[65,12 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[65,12 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[65,13 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101
[65,14 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Creating new shared client for key 'U,P,H10.3.121.134:8091,Bdefault'
[65,16 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net
[65,16 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e
[65,18 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net
[65,19 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821
[69,26 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1
[69,26 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, O
ptions={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[76,93 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409
[77,18 INFO] (SDKD log:137) [Sdkd.ViewLoader|Warn] there are no OK responses at ViewLoader
[77,18 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {
[77,18 INFO] (SDKD log:137) "0": 20,
[77,18 INFO] (SDKD log:137) "264": 980
[77,19 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@101.2 => {[OK]: 20, [MEMD:GENERIC]: 980}
[77,19 INFO] (SDKD log:137) }
[77,19 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3
[77,20 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 269687548
[77,20 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[77,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[77,20 INFO] (SDKD log:137) dispose U,P,H10.3.121.134:8091,Bdefault CouchbaseBucket connection from sdkd
[77,20 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102
[77,21 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Creating new shared client for key 'U,P,H10.3.121.134:8091,Bdefault'
[81,00 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4
[81,01 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, O
ptions={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}}
[92,70 INFO] (SDKD log:137) [Sdkd.ViewLoader|Warn] there are no OK responses at ViewLoader
[92,70 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {
[92,70 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@102.5 => {[OK]: 36, [MEMD:GENERIC]: 964}
[92,70 INFO] (SDKD log:137) "0": 36,
[92,70 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6
[92,70 INFO] (SDKD log:137) "264": 964
[92,71 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[92,71 INFO] (SDKD log:137) }
[92,71 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 104406921
[92,71 INFO] (SDKD log:137) dispose U,P,H10.3.121.134:8091,Bdefault CouchbaseBucket connection from sdkd
[92,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[92,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103
[92,72 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Creating new shared client for key 'U,P,H10.3.121.134:8091,Bdefault'
[96,62 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7
[96,62 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test
_view}
[96,63 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[96,63 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[96,64 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104
[96,64 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9
[96,64 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=tes
t_view}
[96,65 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[96,65 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[96,66 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105
[96,66 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11
[96,67 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=tes
t_view}
[96,68 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[96,68 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[96,68 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106
[96,69 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13
[96,69 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=tes
t_view}
[96,69 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[96,69 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[96,70 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107
[96,70 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15
[96,70 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=tes
t_view}
[96,71 INFO] (Workload call:112) Preloading items into cluster before workload
[96,71 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle
[96,72 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134}
[96,72 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108
[96,72 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17
[96,72 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, Ti
meRes=0}}
[99,64 INFO] (SDKD log:137)
[99,64 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Collection was modified; enumeration operation may not execute
[99,65 INFO] (SDKD log:137) at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
[99,65 INFO] (SDKD log:137) at System.Collections.Generic.List`1.ForEach(Action`1 action)
[99,65 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.Dispose(Boolean disposing) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 197
[99,65 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.Dispose() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 181
[99,66 INFO] (SDKD log:137) at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.UnRegisterObserver(IConfigObserver observer) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\P
roviders\CarrierPublication\CarrierPublicationProvider.cs:line 131
[99,66 INFO] (SDKD log:137) at Couchbase.Core.ClusterManager.DestroyBucket(IBucket bucket) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 228
[99,67 INFO] (SDKD log:137) at Couchbase.Core.ClusterManager.Dispose(Boolean disposing) at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 250
[99,67 INFO] (SDKD log:137) at Couchbase.Core.ClusterManager.Dispose() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 235
[99,67 INFO] (SDKD log:137) at Couchbase.CouchbaseCluster.Finalize() at c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 165
[115,39 ERROR] (BRun maybePropagate:237) Suppressing driver-related error
com.couchbase.sdkdclient.context.HarnessException: Driver crashed
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.7.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.7.0_60]
        at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_60]
[115,42 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.19
[115,43 WARN] (STester close:170) While stopping driver
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.7.0_60]
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.7.0_60]
        at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.7.0_60]
        at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.7.0_60]
        at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.7.0_60]
        at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.7.0_60]
        at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na]
        at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na]



 Comments   
Comment by Jeff Morris [ 05/Aug/14 ]
Re-run please with this SHA: 940183971ea8d0570033ed28bba2e7c84ddc2b71

Please include logs.

Thanks,

Jeff
Comment by Don Pinto [ 13/Aug/14 ]
Andrei, can you verify this and close if it does not exist?
Comment by Andrei Baranouski [ 14/Aug/14 ]
fixed with commit 9401839
did not see it on the last run. open a new one if it will be reproduced




[NCBC-573] Handle case where client is in state where no node is available to handle request. Created: 04/Aug/14  Updated: 04/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0 DP3

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In certain scenarios, the client may be in a state where a node or server is not available to handle the operation request. This ticket is to determine and define the client behavior in this case.




[NCBC-572] When using TimeSpan objects doesn't expiry from couchbase Created: 31/Jul/14  Updated: 05/Aug/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 1.3.7
Fix Version/s: 1.3.9

Type: Bug Priority: Major
Reporter: Agnieszka Cieslak Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In version 1.3.6 here was a bug introduced when using Store method:
http://www.couchbase.com/communities/q-and-a/store-memcached-bucket-expi...

I upgraded to version 1.3.7 and it looks like this issue is still happening.

When I pass TimeStamp to Store method they don't expire at all.

our implementation:
public void Store(StoreMode mode, string key, object value, TimeSpan validFor)
        {
tryExecuteStore(() => _client.ExecuteStore(mode, key, getObjectToStore(value), validFor));
        }

 private static void tryExecuteStore(Func<IOperationResult> executeStoreTask)
        {
            tryExecuteAndConvertExceptions(executeStoreTask, "Store");
        }

private static void tryExecuteAndConvertExceptions(Func<IOperationResult> executeStoreTask, string logName)
        {
            IOperationResult result = null;
            try
            {
                result = executeStoreTask();
            }
            catch (Exception exception)
            {
                logUnsuccessfulOperationAndThrow(logName, exception.Message, exception);
            }
      
            if (!result.Success)
                logUnsuccessfulOperationAndThrow(logName, result.Message, result.Exception, result.StatusCode);
        }


private static void logUnsuccessfulOperationAndThrow(
string operationType, string message, Exception exception, int? statusCode = null)
{
message = String.Format("Error during {0} operation: {1}", operationType, message);
if (statusCode.HasValue)
message = String.Format("{0} ({1} - {2})", message, statusCode.Value,
((CouchbaseStatusCode) statusCode.Value).ToString());

_logger.Error(message, exception);
throw new CouchbaseException(message, exception,
statusCode == null ? default(CouchbaseStatusCode?) : (CouchbaseStatusCode) statusCode);
}





[NCBC-571] When TCP error encountered while authenticating, return appropriate response (not an auth failure per se). Created: 30/Jul/14  Updated: 20/Aug/14  Resolved: 20/Aug/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
See NCBC-567 for details.

 Comments   
Comment by Jeff Morris [ 20/Aug/14 ]
Fixed here: http://www.couchbase.com/issues/browse/NCBC-592




[NCBC-570] KeyNotFoundException in ConfigChangedHandler in Rb2Out test Created: 30/Jul/14  Updated: 21/Aug/14  Resolved: 14/Aug/14

Status: Closed
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Critical
Reporter: Andrei Baranouski Assignee: Andrei Baranouski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: commit 9ce880

Attachments: Zip Archive log.zip     Text File test_logs.txt    

 Description   
was found during digging https://www.couchbase.com/issues/browse/NCBC-567?focusedCommentId=96333&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-96333

so, I've applied workaround to catch AuthenticationException and retry client commands on sdkd side:

diff --git a/src/SdkdConsole/Daemon/Handle.cs b/src/SdkdConsole/Daemon/Handle.cs
index c76b291..76ef19c 100644
--- a/src/SdkdConsole/Daemon/Handle.cs
+++ b/src/SdkdConsole/Daemon/Handle.cs

@@ -187,7 +188,14 @@ public class Handle : ProtocolIOHandler
                        CurrentCommand = cmd;
                }

- cmd.Run();
+ try^M
+ {^M
+ cmd.Run();^M
+ } catch (System.Security.Authentication.AuthenticationException exc){^M
+ Console.WriteLine("!!!!!!!!!!!!!!!!System.Security.Authentication.AuthenticationException " + SdkHandle.isDisposed());^M
+ cmd.Run();^M
+^M
+ }^M


sometimes the test Rb2Out-KV failed with

[190,51 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3%
[191,77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4%
[193,01 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10%
[193,85 INFO] (SDKD log:137)
[193,85 INFO] (SDKD log:137) unhandled exception: System.Collections.Generic.KeyNotFoundException: This key is not in the dictionary.
[193,86 INFO] (SDKD log:137) in System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key)
[193,86 INFO] (SDKD log:137) in Couchbase.Configuration.Server.Providers.Streaming.HttpStreamingProvider.ConfigChangedHandler(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\
Streaming\HttpStreamingProvider.cs:line 151
[193,86 INFO] (SDKD log:137) in Couchbase.Configuration.Server.Providers.Streaming.ConfigThreadState.ListenForConfigChanges() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\Streaming\ConfigThreadState.
cs:line 104
[193,87 INFO] (SDKD log:137) in System.Threading.ThreadHelper.ThreadStart_Context(Object state)
[193,87 INFO] (SDKD log:137) in System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[193,87 INFO] (SDKD log:137) in System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
[193,87 INFO] (SDKD log:137) in System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
[193,87 INFO] (SDKD log:137) in System.Threading.ThreadHelper.ThreadStart()
[194,26 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16%
[195,50 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[196,74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[197,98 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17%
[199,14 ERROR] (BRun maybePropagate:237) Suppressing driver-related error




 Comments   
Comment by Jeff Morris [ 05/Aug/14 ]
Re-run please with this SHA: 940183971ea8d0570033ed28bba2e7c84ddc2b71

Please include logs.

Thanks,

Jeff
Comment by Andrei Baranouski [ 06/Aug/14 ]
don't see this exception with commit 94018..
Comment by Don Pinto [ 13/Aug/14 ]
Andrei, looks like we can close this then?

Thanks,
Comment by Andrei Baranouski [ 14/Aug/14 ]
fixed with commit 9401839




[NCBC-569] Client hangs on CLOSE_WAIT when node is removed from Cluster or when trying to bootstrap against node that isn't in a cluster. Created: 29/Jul/14  Updated: 29/Jul/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





Implement Replica Read Capability (NCBC-493)

[NCBC-568] Update IVBucket to support more than one replica Created: 29/Jul/14  Updated: 29/Jul/14  Resolved: 29/Jul/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 DP3
Fix Version/s: 2.0 Beta

Type: Technical task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments