Net SDK 3 performance

Hello

On my tests, doing a very simple example (100.000 times storing and retrieving a string on the Couchbase server) i saw major performance difference from SDK 2.7.

The .Net SDK v3.0.7 is used against a Couchbase server 6.6
The .Net SDK v2.7 is used against a Couchbase server 6.0

The performance decrease was about 40%.

For the storing / retrieval of string using the .Net SDK v3 i used RawStringTranscoder which is, as the docs say, the fastest to store simple strings

Actually i did also other tests with other object types (for example dictionaries) with the same results, always performance degradation.

Is there any performance benchark done internally for the performance either of new SDK or new Couchbase server? A 40% decrease is quite huge.

How did you make RowStringTrascoder work in 3.0.7? For me it doesn’t

Did you check for getting an expected string from the server?

@jzissop -

Thanks for posting, we are looking into the performance difference between sdk2 and sdk3. The performance is related most definitely to the SDK and not CB server; I am guessing there are some internal knobs that need to be tweaked.

I opened a ticket for tracking: NCBC-2721

-Jeff

For me RawStringTranscoder working as expected Below my generic code. Look at SetCouchbaseString and GetCouchbaseString.

code.zip (841 Bytes)

Thank you for posting the code. Are you getting expected HasValue and HasException returned by GetCouchbaseString and a string returned by GetCouchbaseString is equal the string passed to SetCouchbaseString ?

@jzissop

What version of the .NET Framework are you using? They can potentially have very different performance profiles, as SDK v3 uses a lot of optimizations from .NET Core 2.1 and later.

Also, I see your sample code, and you’re using .GetAwaiter().GetResult() to get synchronous code to execute asynchronous code. Depending on various factors, this can have a very negative performance impact. SDK v3 is really intended to be used asynchronously all the way up the stack. Could you provide the code that is generating the load? That may help me determine if this is a factor or not.

1 Like

I get back the same value - so it works.

I use .net v4.8.

The reason i use .GetAwaiter().GetResult() is that i can;t change the whole app to use async methods, it’s a very big web app based on 4.8. If that has very negative performance i think you may have to implement also the equivalent synchronous methods for us we can’t go async all the way.

The code is a simple loop using the methods i sent (SetCouchbaseString , GetCouchbaseString )

Honestly, if you’re unable to refactor your application to be asynchronous, I’d consider using SDK 2.x. The problem with providing both synchronous and asynchronous overloads is it requires a massive amount of duplicate code (to do it properly without using GetAwaiter().GetResult() or similar inefficient hacks). This creates a lot of additional surface area for bugs and makes the SDK harder to maintain.

I’m not an SDK team member, just an individual contributor, but my understanding is that the decision was made with SDK v3 to focus on support for newer platforms and technology. Async/await has been available for many years, and is well established as the preferred approach in the .NET community. Therefore, SDK v3 focuses on supporting these modern approaches and dropped synchronous methods to make the SDK more maintainable going forward and to reduce developer confusion as to which method to use.

I know for some legacy applications this is non-ideal (including some of my applications where I work). But for these, we’re just sticking with SDK v2 or taking on the tech debt of the asynchronous refactor, on a case-by-case basis. However, if enough people strongly desire synchronous support, that may be something that sways the Couchbase SDK team into adding it back.

2 Likes

Started to work for me after 3.1.0 upgrade :slight_smile: . Thanks everyone

3 Likes

On my tests the big performance gap is on get operations. The set operations is about 10% slower, the get ones 50%+. So maybe its not only the async part but also something else on the implementation.

Also get operations are too slow when the value not already exists on the couchbase (5 times slower!). I suspect that is happening due to use of exceptions (maybe i’m wrong but just an idea)?

@jzissop

We are in the midst of reviewing and improving the performance of the SDK trying to find potential bottlenecks. So far, in experiments on my machine, I’ve gotten insert operation performance almost double what it is in 3.1.0.

However, please continue to provide feedback about where you’re seeing bottlenecks, I’ll make sure I include these spots in my analysis.

Would there be any performance impact of using the Couchbase Dependency Extensions (3.0.5.931) in a .NET Core web api? I believe we are using best practices of injecting the BucketProvider in the Web API controller, then using GetBucketAsync() in the controller methods. After migrating to the 3.1.0 and refactoring for async we are also seeing a considerable performance impact. Just wondering what the optimal usage pattern is in this case where web api calls can be in the hundreds per second.

@weezelboy

No, I would not expect any noticeable performance impact from using the DI infrastructure. It’s basically comparable to putting the ICluster in your DI system yourself and calling BucketAsync() on the ICluster. It’s a very thin wrapper. Of course, the DI system itself will incur some overhead compared to a global static variable. But since .NET Core is so built on DI anyway I wouldn’t be concerned.

Thanks @btburnett3, any timeframe on the new .NET SDK release?

@jmorris will have to speak to release time frames.

However, here is a list of the work completed so far for the 3.1.1 release (assuming no issues in final testing require a revert): https://issues.couchbase.com/issues/?jql=project%20%3D%20NCBC%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%203.1.1%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

@weezelboy -

Releases are scheduled the first Tuesday of each week; next (3.1.1) is planned for January 5th, 2021. In certain circumstances we release off-schedule, but its an internal decision. The release schedule can be found in the Couchbase .NET Client Library project in Jira.

Note that you can always pull the source from its Github repo and build it; the master branch is the current development branch and releases are tagged.

-Jeff

@jmorris It is Jan 8, and so far I don’t see 3.1.1 released yet, do you have an updated date in when it will be released to NuGet?

The link Couchbase .NET Client Library project in Jira.

Does not allow me access to see it :slight_smile:

Hi @Brian_Davis -

Due to the holidays 3.1.1 date was moved to January 12th 2021.

-Jeff

Hi Jeff,

Do you know what time today it will be released?

Thanks,

Brian Davis