(Please read update)
I’ve been using NOT_BOUND consistency for n1ql and it’s been working just fine. I just noticed that there was a minute delay with my index without a good reason. I’m very hesitant to use request_plus as we’ve observed slow performance with it. I think UPDATE_AFTER behavior from view should work for us. Is “at_plus” same as update_after from view consistency?
Is there a way to find out why there was delay in indexing?
We currently have 363,421,87+ documents and the size of the particular index is 1.5gb (disk size).
Update1: it turned out that there was at least 30 minutes delay (?).
What I’ve done exactly.
- Noticed that I could not find a document.
- Confirmed that the document exists
- Confirmed that the index exists and query was using the index
- Created the same index with a different name
- When the #4 index was being created (at 60%), I queried and it worked.
- After #4 was complete, I dropped it and query still worked.
@moon0326, which version are you on? if you have 4.5, we have built memory optimized indexes for exactly these types of situations where consistency and freshness of the index is critical. My guess is you have a large count of pending items on the index in question and it cannot drain them fast enough. The typical reason for indexing latency is either storage delay or resource limits on the local node the index is running on.
One option you have is to use memory optimized indexes. Another is to partition your index so each index can be placed independently in separate nodes to get access to more nodes for building the index. You can read a short piece here for partitioning your indexes here: http://developer.couchbase.com/documentation/server/4.5/indexes/gsi-for-n1ql.html
if you have questions on how to partition the index, we can chat more in detail based on your indexes and queries.
At_plus: At_plus is only available in 4.5. As opposed to using the “request time” for index consistency with request_plus, with at_plus, we use the last update timestamp. Even if that is only a short amount of time (10s or 100s of milliseconds), with systems that have large number of mutations in the background, at_plus can make a great deal of difference in latencies.
You can find details on RYOW and at_plus consistency level here: http://developer.couchbase.com/documentation/server/4.5/developer-guide/query-consistency.html
We are on 4.1. We will be upgrading to 4.5 once stable version gets released, but we really want to stay on 4.1 for now. The incident happened when we were migrating our legacy data to couchbase. We were basically hammering the server. I don’t expect we will have that much traffic once we go live.
I’m still not sure if it was issue with consistency at this point as it would not take 30+ minutes even with a slow machine. We had high read, low write situation when it occurred. I just thought it might be a possible bug as it started returning the data when I created a new index (with same def. though).
As I can’t replicate this anymore, I will report back if we get the same issue again.