EnsureIPEndPointsAreLoaded() is not thread-safe

.NET SDK
1

Hi,

We recently upgraded our Couchbase .NET client library to version 2.7.18. Since then we started seeing the following exception

    System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Couchbase.Configuration.Server.Serialization.VBucketServerMap.EnsureIPEndPointsAreLoaded() in C:\Jenkins\workspace\dotnet\sdk\couchbase-net-client-scripted-build-pipeline\couchbase-net-client\Src\Couchbase\Configuration\Server\Serialization\VBucketServerMap.cs:line 77
   at Couchbase.Configuration.Server.Serialization.VBucketServerMap.get_IPEndPoints() in C:\Jenkins\workspace\dotnet\sdk\couchbase-net-client-scripted-build-pipeline\couchbase-net-client\Src\Couchbase\Configuration\Server\Serialization\VBucketServerMap.cs:line 44
   at Couchbase.Core.VBucket.LocatePrimary() in C:\Jenkins\workspace\dotnet\sdk\couchbase-net-client-scripted-build-pipeline\couchbase-net-client\Src\Couchbase\Core\VBucket.cs:line 40
   at Couchbase.Core.Buckets.CouchbaseRequestExecuter.SendWithRetry[T](IOperation`1 operation) in C:\Jenkins\workspace\dotnet\sdk\couchbase-net-client-scripted-build-pipeline\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseRequestExecuter.cs:line 657
   at Couchbase.CouchbaseBucket.Get[T](String key, TimeSpan timeout) in C:\Jenkins\workspace\dotnet\sdk\couchbase-net-client-scripted-build-pipeline\couchbase-net-client\Src\Couchbase\CouchbaseBucket.cs:line 978

In EnsureIPEndPointsAreLoaded() function

internal void EnsureIPEndPointsAreLoaded()
        {
            if (_ipEndPoints == null || !_ipEndPoints.Any())
            {
                lock (_syncObj)
                {
                    if (_ipEndPoints == null || !_ipEndPoints.Any())
                    {
                        _ipEndPoints = new List<IPEndPoint>();
                        foreach (var server in ServerList)
                        {
                            _ipEndPoints.Add(IPEndPointExtensions.GetEndPoint(server));
                        }
                    }
                }
            }
        }

The lock is acquired after Any() is called.
This could cause 1 thread to change the list _ipEndPoints while another thread is executing Any(). Causing the exception Collection was modified; enumeration operation may not execute.

To avoid lock contention, in version 2.7.12 the lock was moved after the null and Any() check.

But this has made the function, not thread-safe.

Would greatly appreciate it if this bug could be fixed!

1 Like