First I started my local dev server as my demo site (the one we are looking at here) is non-functional at the moment. The dev server is basically one local VM with my application server and a database server and sync.gateway server.
And to my big surprise it reported the same problems!!! I have hardly used this environment for the last two months so I shouldn’t have been able to break the data myself
What has happened is that I have installed the latest JVM SDK before applying it to the other environments (to proof it didn’t break anything - and then deploy to the demo environment - and finally to the prod. environment).
Same goes for the sync. gateway, i.e. update the software to 2.6.0 in dev prior to the demo environment. The sync.gateway is not yet installed in the production environment as only the server side of the new version has been put into production (app not ready yet). So a potential cause could be version 2.6.0?
The only other thing I can think of is that I have code to generate indexes on the server. As it is a Community Edition I create say indexA via code. If there are more servers in the cluster (e.g. demo and prod) I create the indexes as indexA_1 and indexA_2 on their separate servers. This works fine…
Seeing that sync.gateway creates it’s own indexes, e.g. sg_access_x1, I decided to do the same for these indexes naming them sg_access_x (if one node) or sg_access_x_1 and sg_access_x_2 if there are two nodes. This way I would have a “complete” environment if one database node is down. However, sync.gateway checks on the names - so it just creates its “own” indexes again if they do not exist. So seeing this I have deleted these extra indexes again.
… just trying to sum up what changes I can identify since it worked.
Looking back at the release notes of the Java SDK I found that the Snappy aware code was introduced in 2.6.0. So I tried to install version 2.5.7 in my dev. environment - and now it works without failing!
AND it is in combination with Sync.Gateway 2.6.0 installed as I’m still running 2.7.9 in my prod. environment and don’t have the problem. But I think it is safest to downgrade that to 2.5.7 as well…
That would make sense, as the pre-Snappy SDK would not negotiate snappy support with the server during the initial HELLO, and so no documents will be sent with the snappy flag.
But as @ingenthr says above, a CE server should not offer snappy for negotiation anyway, regardless of the SDK version…
I think there’s enough evidence of something squiffy going on for someone (probably me) to try to replicate this here. I’ll try to find some time for it. In the mean-time, are you ok with the workaround for now of using a pre-2.6.0 client?
It works okay, yes. It may be possible to disable this negotiation at the 2.7 client (@graham.pople may know off the top of his head), but IIRC we just use the size tuneables.
If the environment or logs are still handy, it’d be useful to get the INFO level logging from the client after initial bootstrap. That may indicate what the server is saying in response to the client trying to negotiate compression. Even better, it’d be interesting to know if you still see this with the 6.5 Beta.
Also, even though it’s a bit early, I filed MB-36299 to track this and request the team see if there’s a likelihood the bit can be incorrectly set.
I have downgraded all my environments to use the Java SDK 2.5.7 - and it’s stable again.
I’ll be happy to take my dev. environment and reinstall the SDK 2.7.9 and provide you with the information that you need. However, I need a little help on exactly how to enable those settings and where to find the relevant output for you?
And are there any logs you want me to clear in advance to isolate the relevant info then please let me know as well.
I’m still a little new to Couchbase - but learning…
Hi @jda
Thanks for your patience while we investigated this. The issue was tracked down to MB-34879, please see that ticket for details and resolution. Thank you for raising this and for your detailed bug report.
