Data to persist loss when 1 of 3 nodes is restarted.
We have Version: 2.0.1 enterprise edition (build-170) , working on 3 nodes and 2 buckets with auto-failover setup.
There is persistent write activity to those 2 buckets (approx 5 records in second for each bucket) coming thru java couchbase client ver. 1.1.8
Everything is stable if all 3 nodes are up. But if some node is restarted - the loss of data begins, from 1000 expected records around 900 is saved.
This data looks lost forewer.
When java client is restarted next 1000 saves normally.
Also we have in logs a lot of warnings from java client, but no exceptions , looks like client trying to solve itself node loss problem, but the data is lost.
"2013-07-22 18:46:14.685 WARN com.couchbase.client.CouchbaseConnection: Node expected to receive data is inactive. This could be due to a failure within the cluster. Will check for updated configuration. Key without a configured node is: 6a982cf3-6753-4c9e-8fd7-5519bde7c098."
As i correctly understant client should write to other node, but looks like it still trying to write data to the offline node .
Is there any way to manually switch client to other , live , node.
Could you please advice.
sorry did see your reply.... try 2 replicas in you vBucket
What's your auto-failover timeout? Are you using replicas? Is auto-failover actually occurring? "There are a number of restrictions on automatic failover":