[NCBC-709] Make sure all Task/Task<T> call ConfigureAwaiter(false) Created: 30/Oct/14  Updated: 30/Oct/14  Resolved: 30/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This is a general best practice for library code to disable the awaiter so that a deadlock does not occur in certain host environments (ASP.NET, UI, etc): http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html

 Comments   
Comment by Jeff Morris [ 30/Oct/14 ]
http://review.couchbase.org/#/c/42647/




[NCBC-708] Use attached config when a NMV is returned via CCCP Created: 30/Oct/14  Updated: 30/Oct/14  Resolved: 30/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, the client requests a new config when a NMV is encountered even though the payload of the operation may be a config. This changes the behavior so that the client will use the config returned as the operation's result.

 Comments   
Comment by Jeff Morris [ 30/Oct/14 ]
Apparently I changed this already, because in a NMV event, the client uses the config that comes with the failed operation:

https://github.com/couchbase/couchbase-net-client/blob/master/Src/Couchbase/CouchbaseBucket.cs#L245





[NCBC-707] Return timeout if NMV replies exceed threshold Created: 30/Oct/14  Updated: 31/Oct/14  Resolved: 30/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Jeff Morris [ 30/Oct/14 ]
http://review.couchbase.org/#/c/42641/
Comment by Matt Ingenthron [ 31/Oct/14 ]
Not directly related to this commit, but do we think the retry threshold is right since it came up in those tests?
Comment by Jeff Morris [ 31/Oct/14 ]
Not sure; maybe something we should monitor and then make a decision on once we have a better idea of what the threshold should be?




[NCBC-706] Client throws exception when first bootstrapping URI fails Created: 29/Oct/14  Updated: 29/Oct/14  Resolved: 29/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When the client attempts to bootstrap, if it chooses a URI that it cannot connect to, it throws an AggregateException with the errors it encountered, but does not continue onto the next URI in the list.

 Comments   
Comment by Jeff Morris [ 29/Oct/14 ]
http://review.couchbase.org/#/c/42594/1




[NCBC-705] When a view request 404's servers return message is overriden with exception message. Created: 29/Oct/14  Updated: 29/Oct/14  Resolved: 29/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When a view request errors out, the results that are being returned:

{"error":"not_found","reason":"Design document _design/beer-sample2 not found"}

Are overridden by the error handling logic, and return back a more generic .NET error: (404) Not Found.

 Comments   
Comment by Jeff Morris [ 29/Oct/14 ]
http://review.couchbase.org/#/c/42585/




[NCBC-704] Ensure opaque value is unique for all operation instances Created: 29/Oct/14  Updated: 29/Oct/14  Resolved: 29/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Bug Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
All operations are implementations of the base class OperationBase<T> which is generic. This class held a static variable to use as a counter base; which is flawed because for each type of T instantiated, a new static variable would be created because internally each type of T is compiled into it's own class. This moves the opaque generator into a new class, SequenceGenerator, so that the identifier is unique and incremented correctly.

 Comments   
Comment by Jeff Morris [ 29/Oct/14 ]
http://review.couchbase.org/#/c/42573/2




[NCBC-703] When an operation timesout ensure that an NRE is not thrown Created: 26/Oct/14  Updated: 30/Oct/14  Resolved: 28/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Bug Priority: Blocker
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
2014-10-26 09:59:52,548 [17] DEBUG Couchbase.CouchbaseBucket - Operation for key SimpleKeyREP7126REP7126REP7126REP7126REP7126 failed after 0 retries. Reason: Operation timed out: the timeout can be configured by changing the PoolConfiguration.OperationTimeout property. The default is 2500ms.

...

2014-10-26 09:59:53,842 [11] WARN Couchbase.IO.ConnectionBase - Opcode=Set | Key=SimpleKeyREP7126REP7126REP7126REP7126REP7126 | Host=10.3.121.136:11210
System.NullReferenceException: Object reference not set to an instance of an object.
   at Couchbase.IO.Converters.AutoByteConverter.ToByte(Byte[] buffer, Int32 offset) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Converters\AutoByteConverter.cs:line 36
   at Couchbase.IO.Operations.OperationBase`1.Read(Byte[] buffer, Int32 offset, Int32 length) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Operations\OperationBase'.cs:line 105
   at Couchbase.IO.Strategies.EapConnection.ReceiveCallback(IAsyncResult asyncResult) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line 80


 Comments   
Comment by Jeff Morris [ 26/Oct/14 ]
This is a much more serious issue than expected.
Comment by Jeff Morris [ 28/Oct/14 ]
http://review.couchbase.org/#/c/42555/
Comment by Andrei Baranouski [ 29/Oct/14 ]

perhaps it is the very best results

 with commits from gerrit:

$ git log
commit 28c44b06219811747ffb4382a05686ef99a02ffe
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Tue Oct 28 20:04:08 2014 -0700

    NCBC-703: When an operation timesout ensure that an NRE is not thrown

    Change-Id: Iba4be3d23a1d416b2c64a4d0c04d22b3ec0c9011

commit f886642d1b6e57138d424b317e1cfe87986a6628
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Sat Oct 25 19:07:23 2014 -0700

    NCBC-702: Make Release and AddRef un-callable in assemblies other than Couchbase

    and Couchbase.Tests. IRefcountable is an internal interface whose method
    implmentations should not be called externally. This commit makes them
    (mostly) un-callable from assemblies that are not 'friend' assemblies of
    Couchbase.NetClient.

    Change-Id: Ie796909a747eaaf002fcf0984624d9ed0589c5d7

commit 00441eb8036257fa22962bb55ec12beedc309100
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Sat Oct 25 18:50:59 2014 -0700

    NCBC-585: Make Cluster object use Dispose correctly

    Change-Id: If98cb3499a634488f03bf93b914f616067552ef3

commit 3f12994ec37263d6e8dfd0bbd9bdf5dc5a3f7e6c
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Sat Oct 25 18:37:42 2014 -0700

    Mark the server as dead when it is disposed

    When the server is disposed, mark as dead so that an in-progress operation
    or view request will not select it. This should eliminate some ODE's but
    not all. This commit also makes the _disposed and _isDead variables
    volatile.

    Change-Id: I88b2e12079582c66725d898a6eaeee154163ea99

commit b1dc863432c0dc0ae3231004b6bc94734f8d1953
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Sat Oct 25 13:21:49 2014 -0700

    Ensure that heartbeat configuration is completed before raising next event

    Change-Id: I0389212c2b35508ef7637e59c610fac1cdbaca45

commit 509098f2aead49c4262f98058e4f045741e2fad1
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Fri Oct 24 15:23:08 2014 -0700

    Use consistent error handling when IO fails

    Change-Id: Idfca664637d5e3db41770fe0137b11b2cab36952
    Reviewed-on: http://review.couchbase.org/42427
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Brett Lawson <brett19@gmail.com>

commit 2769f360e3d129c5ebda996117361e00b9e6677f
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Fri Oct 24 15:20:45 2014 -0700

    Ensure that each server is tried during a config heartbeat

    Change-Id: I7547effadbc5a76fb5c801450c0f069d2b272d32
    Reviewed-on: http://review.couchbase.org/42426
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Matt Ingenthron <matt@couchbase.com>

commit 37758dbdc9baf75101a7e6230a2a277c37948d38
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Thu Oct 23 19:56:49 2014 -0700

    Remove old markdown file. New one (README.md) will be updated soon

    Change-Id: I73abd2e85ba17d33d350fe0bc530bd2356c1d7cc



Run all tests on 3 different platforms:

win2008 http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/160/
win2012 http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2_win2012_64_andrei/92/
win8 http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2_win8_64_andrei/33

Rb2OutEpt-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/041893/11703c079c2f02626bbe335a9b718df5-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/041893/11703c079c2f02626bbe335a9b718df5-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/036485/d3aee6f752c82f33ea80ec199fbf9ef8-MC.html hight latency at the end(sdkd issue?)
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/036485/d3aee6f752c82f33ea80ec199fbf9ef8-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/031066/597255f7ae0fd0df2ebc684cc28d61da-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/031066/597255f7ae0fd0df2ebc684cc28d61da-CB.html


Rb2Out-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2Out-HYBRID/10-29-14/042355/09ea1fa63ef3dfb2ca4a1d698c62efff-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2Out-HYBRID/10-29-14/042355/09ea1fa63ef3dfb2ca4a1d698c62efff-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2Out-HYBRID/10-29-14/037057/ea79d54c579a1a48e0ac7f01f04bc867-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2Out-HYBRID/10-29-14/037057/ea79d54c579a1a48e0ac7f01f04bc867-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2Out-HYBRID/10-29-14/031811/f5a455baad9ffbf4c7a50276d16fb553-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2Out-HYBRID/10-29-14/031811/f5a455baad9ffbf4c7a50276d16fb553-CB.html


passthrough-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/passthrough-HYBRID/10-29-14/042844/c0039c00c1e4339b4dda19ce6038f5ee-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/passthrough-HYBRID/10-29-14/042844/c0039c00c1e4339b4dda19ce6038f5ee-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/passthrough-HYBRID/10-29-14/037626/596375dbc5053db776d148d7ce3c3037-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/passthrough-HYBRID/10-29-14/037626/596375dbc5053db776d148d7ce3c3037-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/passthrough-HYBRID/10-29-14/032415/0281d375303cb616af11a86f2ec47286-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/passthrough-HYBRID/10-29-14/032415/0281d375303cb616af11a86f2ec47286-CB.html


ReAdd2-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/ReAdd2-HYBRID/10-29-14/042934/f6982afd2f191a9d4963d4ccfe64ab12-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/ReAdd2-HYBRID/10-29-14/042934/f6982afd2f191a9d4963d4ccfe64ab12-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/ReAdd2-HYBRID/10-29-14/037833/c536fc8c73d37708c542444f578f2bc2-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/ReAdd2-HYBRID/10-29-14/037833/c536fc8c73d37708c542444f578f2bc2-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/ReAdd2-HYBRID/10-29-14/032628/b81a4faf3f7166858f26981c8b46a133-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/ReAdd2-HYBRID/10-29-14/032628/b81a4faf3f7166858f26981c8b46a133-CB.html


Rb2In-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2In-HYBRID/10-29-14/043313/249005aa696cf8685fd43625a88dd650-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2In-HYBRID/10-29-14/043313/249005aa696cf8685fd43625a88dd650-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2In-HYBRID/10-29-14/038308/eebb9e612a56c87e5e34cd84c7f06157-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2In-HYBRID/10-29-14/038308/eebb9e612a56c87e5e34cd84c7f06157-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2In-HYBRID/10-29-14/033162/907eb3081d7173890ecead73f1c608cc-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2In-HYBRID/10-29-14/033162/907eb3081d7173890ecead73f1c608cc-CB.html


FoEptRb-HYBRID
#1 win 2008
crash in sdkd
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptRb-HYBRID/10-29-14/038886/080b15823ed51ae3d09db032bba64549-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptRb-HYBRID/10-29-14/038886/080b15823ed51ae3d09db032bba64549-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptRb-HYBRID/10-29-14/033869/c599539be79d3784579fe41aa054c532-MC.html hight latency at the end(sdkd issue?)
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptRb-HYBRID/10-29-14/033869/c599539be79d3784579fe41aa054c532-CB.html


SvcRestart-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/SvcRestart-HYBRID/10-29-14/043931/f0c6e37094e045acad562f15f272412f-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/SvcRestart-HYBRID/10-29-14/043931/f0c6e37094e045acad562f15f272412f-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/SvcRestart-HYBRID/10-29-14/039260/de6ed80e8006c08ecc993b968adfdbc6-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/SvcRestart-HYBRID/10-29-14/039260/de6ed80e8006c08ecc993b968adfdbc6-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/SvcRestart-HYBRID/10-29-14/034312/3e0bae101605ee61a2f5b6241519ebd9-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/SvcRestart-HYBRID/10-29-14/034312/3e0bae101605ee61a2f5b6241519ebd9-CB.html

Rb1Swap-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb1Swap-HYBRID/10-29-14/044171/f63b74d2ea2a4d5ccf3676349fd7fe96-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb1Swap-HYBRID/10-29-14/044171/f63b74d2ea2a4d5ccf3676349fd7fe96-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb1Swap-HYBRID/10-29-14/039588/f5287ccebd186e68c07aedba66ac7270-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb1Swap-HYBRID/10-29-14/039588/f5287ccebd186e68c07aedba66ac7270-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb1Swap-HYBRID/10-29-14/034662/e6891f2c44fb46c4f165b63fb93b9fd1-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb1Swap-HYBRID/10-29-14/034662/e6891f2c44fb46c4f165b63fb93b9fd1-CB.html


FoEptEject-HYBRID
#1 win 2008
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptEject-HYBRID/10-29-14/044530/8367451661c4514c692186f5b91b7746-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptEject-HYBRID/10-29-14/044530/8367451661c4514c692186f5b91b7746-CB.html
#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptEject-HYBRID/10-29-14/040010/a490ee32d7754ac1949e5860a3145616-MC.html hight latency at the end(sdkd issue?)
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptEject-HYBRID/10-29-14/040010/a490ee32d7754ac1949e5860a3145616-CB.html
#3 win 8
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptEject-HYBRID/10-29-14/035142/cd5b688c7555a876bc92c6cbd8d7ff0f-MC.html
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/FoEptEject-HYBRID/10-29-14/035142/cd5b688c7555a876bc92c6cbd8d7ff0f-CB.html


 hope to do one more iteration with github version + ssl testing
Comment by Jeff Morris [ 29/Oct/14 ]
Andrei -

All open changesets have been merged with github.

-Jeff
Comment by Jeff Morris [ 29/Oct/14 ]
>>#2 win 2012
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-29-14/036485/d3aee6f752c82f33ea80ec199fbf9ef8-MC.html hight latency at the end(sdkd issue?)

Can you provide logs so I can see what's up?

Thanks,

Jeff
Comment by Andrei Baranouski [ 30/Oct/14 ]
https://s3.amazonaws.com/bugdb/jira/NCBC-703/c3330f3b/log.zip
http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2_win2012_64_andrei/94/console
http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.1-1443/Rb2OutEpt-HYBRID/10-30-14/046051/6c039aaa37b69a5d1c20192fbffc8e56-MC.html




[NCBC-702] Make Release and AddRef un-callable in assemblies other than Couchbase and Couchbase.Tests Created: 25/Oct/14  Updated: 26/Oct/14  Resolved: 26/Oct/14

Status: Resolved
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.0

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Jeff Morris [ 26/Oct/14 ]
http://review.couchbase.org/#/c/42452/




[NCBC-701] sometimes ops/sec drop to ~1 ops/sec Created: 25/Oct/14  Updated: 27/Oct/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.1

Type: Bug Priority: Blocker
Reporter: Andrei Baranouski Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File 1_ops_sec2.png     PNG File 1_ops_sec3.png     PNG File 1_ops_sec.png     Zip Archive NCBC-701_new.zip    

 Description   
run with

$ git log -n 5
commit 509098f2aead49c4262f98058e4f045741e2fad1
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Fri Oct 24 15:23:08 2014 -0700

    Use consistent error handling when IO fails

    Change-Id: Idfca664637d5e3db41770fe0137b11b2cab36952
    Reviewed-on: http://review.couchbase.org/42427
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Brett Lawson <brett19@gmail.com>

commit 2769f360e3d129c5ebda996117361e00b9e6677f
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Fri Oct 24 15:20:45 2014 -0700

    Ensure that each server is tried during a config heartbeat

    Change-Id: I7547effadbc5a76fb5c801450c0f069d2b272d32
    Reviewed-on: http://review.couchbase.org/42426
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Matt Ingenthron <matt@couchbase.com>

commit 37758dbdc9baf75101a7e6230a2a277c37948d38
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Thu Oct 23 19:56:49 2014 -0700

    Remove old markdown file. New one (README.md) will be updated soon

    Change-Id: I73abd2e85ba17d33d350fe0bc530bd2356c1d7cc
    Reviewed-on: http://review.couchbase.org/42412
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Simon Baslé <simon@couchbase.com>

commit 73df659c062aab8a1dd211ddbf93a78864458b09
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Thu Oct 23 19:53:42 2014 -0700

    Removing old script from the enyim days

    Change-Id: I9ed90500ae2c256c6389f072c593f686e3ed4d31
    Reviewed-on: http://review.couchbase.org/42411
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>
    Reviewed-by: Simon Baslé <simon@couchbase.com>

commit 28895f0acb807947e1e55471de3751a75e360de4
Author: jeffrymorris <jeffrymorris@gmail.com>
Date: Thu Oct 23 19:47:37 2014 -0700

    Remove 'tester' projects from min repo

    These will be refactored and moved to a seperate repo on labs.

    Change-Id: I1ef3d497a210d915f5c01f50a50baa9f17c704a8
    Reviewed-on: http://review.couchbase.org/42410
    Reviewed-by: Simon Baslé <simon@couchbase.com>
    Reviewed-by: Jeffry Morris <jeffrymorris@gmail.com>
    Tested-by: Jeffry Morris <jeffrymorris@gmail.com>

Administrator@WIN-26PC57KC7NB /c/jenkins/workspace/.net-sdkdclient-ng2_win2012_6
4_andrei/couchbase-net-client (master)




when I once again reproduced it( in command line now) I terminated sdkd execution and found that some threads has stack traces like:



  [External Code]
> Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 136 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Acquire() Line 148 C#
  Couchbase.NetClient.dll!Couchbase.IO.ConnectionPool<Couchbase.IO.Strategies.EapConnection>.Couchbase.IO.IConnectionPool.Acquire() Line 200 C#
  Couchbase.NetClient.dll!Couchbase.IO.Strategies.DefaultIOStrategy.Execute<Couchbase.Configuration.Server.Serialization.BucketConfig>(Couchbase.IO.Operations.IOperation<Couchbase.Configuration.Server.Serialization.BucketConfig> operation) Line 40 C#
  Couchbase.NetClient.dll!Couchbase.Core.Server.Send<Couchbase.Configuration.Server.Serialization.BucketConfig>(Couchbase.IO.Operations.IOperation<Couchbase.Configuration.Server.Serialization.BucketConfig> operation) Line 88 C#
  Couchbase.NetClient.dll!Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider._heartBeat_Elapsed(object sender, System.Timers.ElapsedEventArgs args) Line 50 C#
  [External Code]


 Comments   
Comment by Andrei Baranouski [ 25/Oct/14 ]
https://s3.amazonaws.com/bugdb/jira/NCBC-701/74a96698/NCBC-701.zip
Comment by Jeff Morris [ 25/Oct/14 ]
Andrei-

Your logs provided in NCBC-701.zip have a time range from 2014-10-25 02:39:22,706 to 2014-10-25 02:59:28,677, so I can't correlate with the graphs 1_ops_sec_.png and 1_ops_sec3.png.

What I see in the NCBC-701.zip logs is that:

a) Threads are waiting for connections (normal in certain situ's)
b) This is caused when other threads are taking too long per operation, thus making the wait queue _way_ too long

What would cause an operation to take a long time? Probably when a request is made on a server that has either been removed from the cluster or has shutdown. The time delay is the TCP timeout's which are all default ATM. So, instead of a making a request and failing immediately, the client waits for a response which may take seconds (actually the it will timeout at 2.5s), which is time in which another thread is waiting to make a request.

However, I can postulate that the config heartbeat checks are exasperating this, since it will check every server until it either succeeds or runs out of servers to try and it runs every 10s by default. Design wise we might want to:

- Move the config heartbeat to a separate dedicated TCP connection AND/OR
- Move it to a dedicated worker thread AND/OR
- Add a dedicated HTTPStreaming config listener to the CarrierPublicationProvider (this would solve most problems, but has some minuses) AND/OR
- Stop the config heartbeat timer when it begins a cycle and start it when a cycle completes - this would keep it from backing up - this is a flaw imo

Additionally, some of this may be mitigated by tuning:

- Increasing the MaxPool size - defaults to 2
- Tuning the socket timeout values (they are all default ATM)

-Jeff

Comment by Jeff Morris [ 25/Oct/14 ]
Andrei -

I made a patch for: "Stop the config heartbeat timer when it begins a cycle and start it when a cycle complete"

http://review.couchbase.org/#/c/42449/

You can test it if you pull it on top of master or push it through CR and do it on master.

Thanks,

Jeff
Comment by Jeff Morris [ 25/Oct/14 ]
Looking deeper into the logs provided, it looks like SDKD is not properly maintaining references to the cluster object:

Line 14767: 2014-10-25 02:39:27,697 [2] DEBUG Couchbase.Cluster - Finalizing Cluster
Line 15996: 2014-10-25 02:39:43,900 [2] DEBUG Couchbase.Cluster - Finalizing Cluster
Line 15997: 2014-10-25 02:39:43,901 [2] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Finalizing DefaultIOStrategy for 10.3.121.134:11210 - 65d70de1-7221-42e4-84fc-7ff2ed34d2d5
Line 145546: 2014-10-25 02:41:39,766 [2] DEBUG Couchbase.Cluster - Finalizing Cluster
Line 150673: 2014-10-25 02:41:40,382 [2] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Finalizing DefaultIOStrategy for 10.3.121.135:11210 - b50fa7db-a7dc-42e9-a762-09a2769e216f
Line 170419: 2014-10-25 02:53:58,109 [2] DEBUG Couchbase.Cluster - Finalizing Cluster
Line 171445: 2014-10-25 02:53:59,075 [2] DEBUG Couchbase.Cluster - Finalizing Cluster

This could or could not be causing inconsistent test results.

Things getting interesting around line 253943:

2014-10-25 02:59:21,082 [17] INFO Couchbase.Configuration.Server.Providers.ConfigProviderBase - Unregistering observer default
2014-10-25 02:59:21,083 [17] DEBUG Couchbase.Configuration.ConfigContextBase - Disposing ConfigContext
2014-10-25 02:59:21,083 [17] DEBUG Couchbase.Core.Server - Disposing Server for 10.3.121.134:11210
2014-10-25 02:59:21,083 [17] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Disposing DefaultIOStrategy for 10.3.121.134:11210 - 6d4d7c96-ef1f-4bbd-adbd-a08303fc821d
2014-10-25 02:59:21,084 [17] DEBUG Couchbase.IO.ConnectionPool`1 - Disposing ConnectionPool for 10.3.121.134:11210 - f191b6cd-6b0b-4311-bcd8-13f40bf44fb0
2014-10-25 02:59:21,091 [17] DEBUG Couchbase.IO.ConnectionBase - Disposing connection for 10.3.121.134:11210 - 6877f2f9-3f19-4f34-8903-a03ba7b64a5e
2014-10-25 02:59:21,093 [40] WARN Couchbase.IO.ConnectionBase - Opcode=GetClusterConfig | Key= | Host=10.3.121.134:11210
System.IO.IOException: Unable to read data from the transport connection: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.. ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
   at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult, SocketError& errorCode)
   at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
   at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
   at Couchbase.IO.Strategies.EapConnection.ReceiveCallback(IAsyncResult asyncResult) in c:\jenkins\workspace\.net-sdkdclient-ng2_win2012_64_andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs


The "default" bucket is disposed (closing the connections to 10.3.121.134:11210), but the client keeps trying to use those connections which then throw ODE's.

-Jeff
Comment by Andrei Baranouski [ 26/Oct/14 ]
Hi Jeff,

I run the tests with the patch http://review.couchbase.org/#/c/42449/

and it's still reproduced on win8 http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2_win8_64_andrei/31/console

timestamp on my screenshot differs for -10 hours becuase I took screenshots on my local host


maybe the problem is the following exceptions becuse I see drops in ops/sec just after them

2014-10-26 09:59:51,197 [12] ERROR Couchbase.Views.ViewClient - System.Net.WebException: The operation has timed out
   at System.Net.HttpWebRequest.GetResponse()
   at Couchbase.Views.ViewClient.Execute[T](IViewQuery query) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\Views\ViewClient.cs:line 91
2014-10-26 09:59:51,209 [12] DEBUG Couchbase.CouchbaseBucket - trying again: 1
2014-10-26 09:59:51,225 [19] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.136:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,233 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.136:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,285 [16] DEBUG Couchbase.Views.ViewQuery - http://10.3.3.206:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,303 [16] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.135:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,463 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.135:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,482 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.3.206:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,495 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,519 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,620 [15] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.136:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,628 [16] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,648 [15] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:51,653 [16] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.136:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:52,110 [10] ERROR Couchbase.Views.ViewClient - System.Net.WebException: The operation has timed out
   at System.Net.HttpWebRequest.GetResponse()
   at Couchbase.Views.ViewClient.Execute[T](IViewQuery query) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\Views\ViewClient.cs:line 91
2014-10-26 09:59:52,111 [10] DEBUG Couchbase.CouchbaseBucket - trying again: 1
2014-10-26 09:59:52,114 [9] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.135:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:52,256 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.136:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:52,548 [17] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: c214d753-0a7d-4091-8355-054e81e3884f on 10.3.121.136:11210 - a6dd7625-ee7c-4658-90aa-1d5ce487daf1
2014-10-26 09:59:52,548 [17] DEBUG Couchbase.CouchbaseBucket - Operation doesn't support retries for key SimpleKeyREP7126REP7126REP7126REP7126REP7126
2014-10-26 09:59:52,548 [17] DEBUG Couchbase.CouchbaseBucket - Operation for key SimpleKeyREP7126REP7126REP7126REP7126REP7126 failed after 0 retries. Reason: Operation timed out: the timeout can be configured by changing the PoolConfiguration.OperationTimeout property. The default is 2500ms.
2014-10-26 09:59:52,548 [17] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#4 on thread 17
2014-10-26 09:59:52,548 [17] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 175 for key SimpleKeyREP7127REP7127REP7127REP7127REP7127 - rev4
2014-10-26 09:59:52,548 [17] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP7127REP7127REP7127REP7127REP7127 using server 10.3.121.134:11210
2014-10-26 09:59:52,548 [17] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: 84627e8b-4a27-4b78-a962-3c86ec6aab7e | 10.3.121.134:11210 | [0, 2] - 397c8ced-c9bc-4df6-8030-05ff10a10b61 - Disposed: False
2014-10-26 09:59:52,548 [17] INFO Couchbase.IO.ConnectionBase - Sending key SimpleKeyREP7127REP7127REP7127REP7127REP7127 using 175 on 10.3.121.134:11210
2014-10-26 09:59:52,557 [17] DEBUG Couchbase.IO.ConnectionPool`1 - Releasing: 84627e8b-4a27-4b78-a962-3c86ec6aab7e on 10.3.121.134:11210 - 397c8ced-c9bc-4df6-8030-05ff10a10b61
2014-10-26 09:59:52,557 [17] DEBUG Couchbase.CouchbaseBucket - Operation succeeded 0 for key SimpleKeyREP7127REP7127REP7127REP7127REP7127
2014-10-26 09:59:52,557 [17] INFO Couchbase.Configuration.ConfigContextBase - Getting KeyMapper for rev#4 on thread 17
2014-10-26 09:59:52,557 [17] INFO Couchbase.Core.Buckets.VBucketKeyMapper - Using index 408 for key SimpleKeyREP7128REP7128REP7128REP7128REP7128 - rev4
2014-10-26 09:59:52,557 [17] DEBUG Couchbase.Core.Server - Sending SimpleKeyREP7128REP7128REP7128REP7128REP7128 using server 10.3.121.135:11210
2014-10-26 09:59:52,557 [17] DEBUG Couchbase.IO.ConnectionPool`1 - Acquire existing: 1ea73feb-71b4-4621-868e-14d7f23a46c3 | 10.3.121.135:11210 | [0, 2] - 46af13e1-5ed8-4ca1-87ba-61071626c3db - Disposed: False
2014-10-26 09:59:52,557 [17] INFO Couchbase.IO.ConnectionBase - Sending key SimpleKeyREP7128REP7128REP7128REP7128REP7128 using 408 on 10.3.121.135:11210
2014-10-26 09:59:52,562 [10] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.136:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:52,580 [16] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:52,589 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.3.206:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:52,641 [12] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.135:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:53,300 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:53,817 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.135:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:53,837 [14] DEBUG Couchbase.Views.ViewQuery - http://10.3.121.134:8092/default/_design/test_design/_view/test_view?
2014-10-26 09:59:53,842 [11] WARN Couchbase.IO.ConnectionBase - Opcode=Set | Key=SimpleKeyREP7126REP7126REP7126REP7126REP7126 | Host=10.3.121.136:11210
System.NullReferenceException: Object reference not set to an instance of an object.
   at Couchbase.IO.Converters.AutoByteConverter.ToByte(Byte[] buffer, Int32 offset) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Converters\AutoByteConverter.cs:line 36
   at Couchbase.IO.Operations.OperationBase`1.Read(Byte[] buffer, Int32 offset, Int32 length) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Operations\OperationBase'.cs:line 105
   at Couchbase.IO.Strategies.EapConnection.ReceiveCallback(IAsyncResult asyncResult) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line 80
2014-10-26 09:59:53,842 [7] WARN Couchbase.IO.ConnectionBase - Opcode=Set | Key=SimpleKeyREP7124REP7124REP7124REP7124REP7124 | Host=10.3.121.136:11210
System.NullReferenceException: Object reference not set to an instance of an object.
   at Couchbase.IO.Converters.AutoByteConverter.ToByte(Byte[] buffer, Int32 offset) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Converters\AutoByteConverter.cs:line 36
   at Couchbase.IO.Operations.OperationBase`1.Read(Byte[] buffer, Int32 offset, Int32 length) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Operations\OperationBase'.cs:line 105
   at Couchbase.IO.Strategies.EapConnection.ReceiveCallback(IAsyncResult asyncResult) in c:\jenkins\workspace\.net-sdkdclient-ng2_win8_64_andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line 80

see logs from the patch run https://www.couchbase.com/issues/secure/attachment/22281/NCBC-701_new.zip


If we talk about the exceptions
Object name: 'System.Net.Sockets.Socket'.. ---> System.ObjectDisposedException: Cannot access a disposed object.
it happened much later we drop performance and presumably after I stop the sdkd application




Comment by Jeff Morris [ 26/Oct/14 ]
Andrei -

Thanks for clarifying, I'll look into these NRE's. I also have a few commits that need to push through CR that may also help and possibly a few more after those :)

The WebTimeoutExceptions will definitely cause latency and it's a known issue that's been identified in the Java and LBC sdks as well.

-Jeff
Comment by Andrei Baranouski [ 26/Oct/14 ]
but this should not occur permanently after a certain point, right?
Comment by Jeff Morris [ 26/Oct/14 ]
Yeah, they should recover at some point. Do you know what specific test? i.e. Rb2OutEpt, etc?
Comment by Andrei Baranouski [ 26/Oct/14 ]
it was Rb2OutEpt test but it does not matter, because it was presetup( the same for all tests).
Comment by Jeff Morris [ 27/Oct/14 ]
If it's pre-setup it indicates a possible issue with your cluster or network issues. These timeouts indicate that the client is waiting on the _server_ to reply...during a rebalance the request may have been mapped to a server that is no longer in the cluster, in which case the timeout is expected.

That being said, i was able to trace to the NRE's and created a ticket for them: http://www.couchbase.com/issues/browse/NCBC-703

-Jeff

Comment by Andrei Baranouski [ 27/Oct/14 ]
[8:03:16 PM] Andrei Baranouski: look what I have thought
it happens from time to time on env where client is from 172 network , cluster is from 10*
perhaps some time delay occurs between networks or termination

 yes, maybe it's a network problem, but it happens in presetup(there is no any rebalance or other operations with cluster on this phase)

[8:04:44 PM] Andrei Baranouski: but in any case we should to fix this bug




[NCBC-700] Add overload for byte array values on Prepend/Append methods Created: 24/Oct/14  Updated: 24/Oct/14

Status: Open
Project: Couchbase .NET client library
Component/s: library
Affects Version/s: 2.0 Beta
Fix Version/s: 2.0.1

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Jeff Morris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





Generated at Fri Oct 31 07:21:46 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.