We use a cluster of 9 Couchbase servers 4.1.0 on windows 2012 R2 platform, Couchbase.NetClient 2.3.5.0
Cluster is hosted in the cloud. Time to time one of nodes may be shutdown because of host failure. Couchbase cluster has replica set and automatic failover configured. In case one of the nodes becomes unavailable, automatic failover happens.
I noticed that during the failure and failover application pools on web servers are crashing.
Below are 2 events I see in the event log as result of crash:
Source: ASP.NET 4.0
An unhandled exception occurred and the process was terminated.
Process ID: 4156
Exception: System.ObjectDisposedException
Message: Cannot access a disposed object.
StackTrace: at System.Threading.TimerQueueTimer.Change(UInt32 dueTime, UInt32 period)
at Couchbase.Core.Server.CheckDataNode()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.TimerQueueTimer.CallCallback()
at System.Threading.TimerQueueTimer.Fire()
at System.Threading.TimerQueue.FireNextTimers()
Source: .NET runtime
Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ObjectDisposedException
at System.Threading.TimerQueueTimer.Change(UInt32, UInt32)
at Couchbase.Core.Server.CheckDataNode()
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.TimerQueueTimer.CallCallback()
at System.Threading.TimerQueueTimer.Fire()
at System.Threading.TimerQueue.FireNextTimers()