Couchbase indexing with Apache Storm
I am planning to use Couchbase with Apache Storm. Since Storm inserts data into Couchbase heavily(i.e. 2~3k updates or inserts per second), view's indexing also keeps updated and restarted endlessly with the default auto-indexing settings. So querying with stale=false is always stopped by the runtime exception saying "failed to access the view", and it seems like querying with stale=false is impossible in this case. Of course, with stale=ok, I can get a result but cannot make sure if the result includes the data I want to map/reduce or not since the indexing is not done yet.
So is there a way to get a correct result that is the same as the result when indexing is 100% in this case? Or a way to know what data was processed with stale=ok up to? For your information, Couchbase version is 2.2.0 community edition, and Java SDK version is 1.4.1
Thanks in advance!
Can you share the relevant codeparts? Also, it looks like the server is returning an error here. Were you doing rebalance/failover or the kind during the view query?
While it may be too slow with stale=false to suit your needs, it shouldn't give you "failed to access view". That seems unusual. Do you have any errors in the log at the cluster?
The best way perhaps is stale=update_after or to have the auto-index do your update and just cycle the query times.
Also, note that 2.2.0 is pretty dated at this stage and 2.5.1 would be better, if you can upgrade. 3.0 (currently under development) is expected to have some improvements here as well. Stay tuned for that in the near future.