To start off, this has nothing to do with the stale parameter.
I had to take down a node a few days ago and then added it back today. Before starting CB on this server, I removed everything from the data directory and then added this node back to the cluster. During the rebalance, a different node failed. I restarted this node and started the rebalance process over again.
Now certain documents in state where they are not being considered during the indexing process. I’ve even created a new test design document and created a new view. It’s like these documents are not being passed to the map function.
Any ideas would be greatly appreciated.
(using 2.0.1 community edition)
I made a new view with a map function that emits the ids of all documents:
… but some documents aren’t included even though I can retrieve them with a get call and they show up in the admin page “Lookup ID” feature. I’m using the full data set and stale=false.
I decided to start up a fourth server with a fresh install of CB and replace what I though was the “bad” node with this new one, but the indexing problem persisted. I then started trying different combinations of these four servers as 3-node clusters with hopes of finding a culprit but to no success.
My last idea was to go down to 1 node and then work my way back up. But when rebalancing from 2 to 1 node, CB-server shutdown on the destination node. I restarted/rebalanced and then both nodes failed during the rebalance process. After about 5 tries of rebalancing, all of these attempts seeming to fail for different reasons, I finally got a 3-node cluster up and running that seems to index all of the documents.
Unfortunately, I can’t post back with a reasonable solution that could help anyone else with this situation, so hopefully someone else can post, a possible reason and/or solution for what might have happened here.