Use System.Text.Json as DefaultSerializer?

.NET SDK
1

Has anyone configured their .NET SDK code to use Microsoft System.Text.Json for use as the DefaultSerializer rather than JSON.NET?

2

@justusweber

I have not, but the 3.x SDK had some design changes specifically with that in mind. For example, the ITypeSerializer.Deserialize method has an overload that accepts ReadOnlyMemory. I think it should be doable.

1 Like
3

I went ahead and created a custom serializer implementing ITypeSerializer and using System.Text.Json, and configured my GetAsync and UpsertAsync calls to use it. So far so good.

2 Likes
4

Hello,

I wanted to look into using System.Text.Json as the serializer for KV reads, and was able to implement as follows:

public class NewSerializer : ITypeSerializer
{
    .........
}

        ClusterOptions opts = new ClusterOptions()
            .WithSerializer(new NewSerializer ());

I noticed that when the GetAsync(…) call is performed on the collection, it calls through to the:
NewSerializer.Deserialize(ReadOnlyMemory buffer).

Given the above, is there any way to cause the async NewSerializer.DeserializeAsync(Stream stream, CancellationToken cancellationToken = default) to be called instead?

5

First, I’d like to point out that there are currently some limitations you may encounter trying to implement a custom serializer. For example, currently custom serializers are not used if you are using projections on get operations.

There is a change in the works to improve that, but it will require further tweaking of your custom serializer to support it: https://review.couchbase.org/c/couchbase-net-client/+/166110

We do have plans to implement a System.Text.Json serializer ourselves, but we’re trying to remove these hurdles first.

In regards to your actual question, no it will not call DeserializeAsync(Stream) for key/value operations. This is actually intentional due to the internal implementation. We always have the entire operation response in a buffer in memory by the time we deserialize it, so using an async stream to deserialize would actually be less performant.

Based on testing I did the other week, you should be able to implement Deserializer<T> pretty easily:

        public T Deserialize<T>(ReadOnlyMemory<byte> buffer)
        {
            return JsonSerializer.Deserialize<T>(buffer.Span, _options)!;
        }
6

Thanks @btburnett3 !

7

@obawin

I thought I’d let you know that the first work for official System.Text.Json support will be arriving soon in 3.2.6.

https://review.couchbase.org/c/couchbase-net-client/+/166750

Please note that this work is considered “volatile” and the API surface may change in future releases. Also, the support is incomplete and missing a few features available with Newtonsoft.Json (for now):

  • Deserializing dynamic objects is not supported
  • Queries emulate streaming results, they read the entire stream before returning the first result
  • GET operations with projections will fail

I expect the remaining features to arrive over the next couple of releases. I hope this helps.

2 Likes
8

Wow; thanks @btburnett3!

I did a quick comparison using System.Text.Json vs. the DefaultSerializer (i.e. Newtonsoft), and there is a performance improvement for sure (didn’t run long/in-depth, but it can upwards in the 10% range). This was on a .NET 6 service hosted in a dockerized Alpine image in the cloud (with Couchbase 6.6).

Other than the missing features above, are there any other limitations/concerns with this serializer? Would you recommend using it on a production system (after testing of course) if only KV queries are done on relatively straight-forward JSON? Or do you think it should be treated as experimental for now?

The API surface changing is ok as we would just need to refactor on subsequent upgrades of the CB SDK.

1 Like
9

Btw, in the future, is it possible to add support for Json Source Generation (.NET 6+)? With the proper input params, it could cause the Deserialize and Serialize calls to use the Json Source Generation overloads?

10

@obawin

The current SDK 3.2.6 does have support for that, in theory. It’s experimental for the moment, let me know if you have problems with it. The syntax to create a serializer from a context is like this:

var serializer = SystemTextJsonSerializer.Create(MyContext.Default);

Note that if you make this your global serializer on ClusterOptions then it must have every type you will be serializing/deserializing registered. If you want to use multiple contexts, you’ll need to pass a different transcoder/serializer to each operation in the XXXOptions object.

11

Sorry I missed this question before. My answer is that I think it’s fine to use in production so long as you’re okay with those limitations, given testing. For simple JSON it should be fine. I’m not currently aware of any other limitations, but please let me know if you encounter anything unexpected.

My only caveat is that the SDK internals (i.e. the Management APIs) are not currently compatible with it. In theory, they should be using an internal Newtonsoft serializer and not the serializer you configure on ClusterOptions, so it should be fine. But there is a chance some of the management APIs may be incorrectly using the serializer from ClusterOptions and therefore may have issues. One of the planned steps is to do an in-depth review of these and switch all the internals to using System.Text.Json. But I qualify this as low-risk and unlikely to be encountered by most applications.

1 Like
12

Great! Thanks!

I played around with binding (to a JsonSerializerOptions instance) a context containing serialization definitions for all possible JSON response DTO’s that could be read from Couchbase. Seems to improve performance by another 1-5% (fluctuates). Very neat!

1 Like