Home | Forums | Couchbase | Couchbase Server 2.0

Sometimes the value is not returned to the client although it does exist in the cluster

7 replies [Last post]
Sun, 02/10/2013 - 21:38
marinap
Offline
Joined: 01/15/2013
Groups: None

Hi guys, I'm using Couchbase 2.0 cluster and Enyim 2.12 client in custom session provider that maintains my site's web session.

For some weird reason some of my Get requests return NULL value.

This happens only when the requests are sent from IIS web server. When I'm calling the session provider code from a console application (under much higher load) - none of the requests are ever getting lost.

I'm running the console app on the same server where the console app is running, so I don't think it's the firewall related issue.

I even added several retries in case of the NULL value returned by Get method (thinking that the malfunction might be caused by some kind of network glitch), but those retries never help...

I know that those values DO exist in the memcache (getting the values for the failed keys from a console app right after the failure always return a value)

I'm really running out of ideas here and will appreciate a lot if someone could help me solving this mystery!

Thanks, Marina

Top
Mon, 02/11/2013 - 11:43
yaroslavs
Offline
Joined: 02/11/2013
Groups: None

Hi,

We tried to use SessionState Provider from ASP.NET 4.5 on Couchbase cluster that consists of 2 servers and had the same issues. At the same time we didn't have any problems when we used standalone Couchbase server (under RedHat 64-bit). We used Couchbase bucket with 1 replica.

We have ended rolling is back.

My guess would be that Moxi is not loading properly connections between servers so the request is not going to right server that have the right copy of active data. Probably some kind vBucket mapping issue.

I'm surprise that nobody responded yet because it's easy to reproduce.

Thanks,
Yaroslav

Top
Tue, 02/12/2013 - 00:00
marinap
Offline
Joined: 01/15/2013
Groups: None

Thanks Yaroslav.

Yesterday we finally managed to reproduce this issue from a console app, by running multiple threads getting and setting the same key (which simulates ajax requests for web session on the web site).

When testing the same scenario with a single node cluster, all the requests were handled correctly, which supports your assumption that the problem is the Moxi...

Does that mean that there's a serious problem with the Moxi and it shouldn't be used?

Although those GET failures happened also when we tried using Couchbase .NET client, which as far as I understand doesn't use Moxi..

So does anyone has a suggestion for how to make this work?

Thanks,
Marina

Top
Tue, 02/12/2013 - 07:34
tgrall
Offline
Joined: 09/05/2012
Groups: None

Hello Marina,

Most of the user that are working with Couchbase 2.0 are not using Moxi anymore but use the latest version of the SDKs.

I know that it does not answer your issue directly but can you test with the latest client API directly connected to the cluster directly (removing moxi from the picture) ?

In the same time I will check with the moxi team to see what could be the issue.

Regards

__________________

Tug
@tgrall

Top
Tue, 02/12/2013 - 07:40
yaroslavs
Offline
Joined: 02/11/2013
Groups: None

Hi,

I understand that is better to use the latest release version but since the Session State Provider was referencing Couchbase.dll version 1.1.6.0 we used it together.

Thanks,
Yaroslav

Top
Tue, 02/12/2013 - 20:45
marinap
Offline
Joined: 01/15/2013
Groups: None

Hi,

I tried using both the latest Enyim client and the latest Couchbase .NET client (v 1.2.1) and ended up having the same problem.

How can I know if Moxi is in the picture or not?
There's no option to turn it off on the Couchbase server...

Thanks,
Marina

Top
Wed, 02/13/2013 - 22:08
marinap
Offline
Joined: 01/15/2013
Groups: None

Ok, here's some more info.

Monitoring the traffic to and from the couchbase server in Wireshark saw that all my failing GET requests are followed by a response from the Couchbase machine with 'Not found' data...

Top
Sun, 02/17/2013 - 15:17
pgrigor
Offline
Joined: 01/08/2013
Groups: None

If it is the case that CB is returning "not found" when it actually does have the value stored then this is a *major* problem. I have a topic posted in these forums where I'm running into the same problem, however I assumed it was the PHP SDK that was somehow not returning an error code.

Top