10.3.121.104-8091-diag.txt.gz
10.3.121.105-8091-diag.txt.gz
[MB-6633] querying for views during rebalancing through node's REST api before it has any active vbucket return ""error":"no_active_vbuckets" Created: 12/Sep/12 Updated: 10/Jan/13 Resolved: 21/Sep/12 |
|
| Status: | Closed |
| Project: | Couchbase Server |
| Component/s: | ns_server, view-engine |
| Affects Version/s: | None |
| Fix Version/s: | 2.0-beta-2 |
| Security Level: | Public |
| Type: | Bug | Priority: | Blocker |
| Reporter: | Iryna Mironava | Assignee: | Aleksey Kondratenko |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | 2.0-beta-release-notes, regression | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
build 1717
centos 1 bucket, 200K items |
||
| Attachments: |
10.3.121.104-8091-diag.txt.gz
10.3.121.105-8091-diag.txt.gz
|
| Description |
|
test
-t viewquerytests.ViewQueryTests.test_employee_dataset_startkey_endkey_queries_rebalance_incrementaly,skip_rebalance=true steps to repro: 1. load data 2. start rebalance 1->2 3. wait some time, (in my case rebalance is going on 15 mins) try to see documents or query view from node which is adding to cluster no_active_vbuckets error appears iryna@ubuntu:~/couchbase/testrunner$ curl -v 'http://10.3.121.105:8092/default/_design/test_view-df0f1d8/_view/test_view-df0f1d8?debug=true&start_key=%5B2008%2C7%2Cnull%5D&connection_timeout=60000' * About to connect() to 10.3.121.105 port 8092 (#0) * Trying 10.3.121.105... * connected * Connected to 10.3.121.105 (10.3.121.105) port 8092 (#0) > GET /default/_design/test_view-df0f1d8/_view/test_view-df0f1d8?debug=true&start_key=%5B2008%2C7%2Cnull%5D&connection_timeout=60000 HTTP/1.1 > User-Agent: curl/7.26.0 > Host: 10.3.121.105:8092 > Accept: */* > < HTTP/1.1 404 Object Not Found < Server: MochiWeb/1.0 (Any of you quaids got a smint?) < Date: Wed, 12 Sep 2012 22:07:23 GMT < Content-Type: text/plain;charset=utf-8 < Content-Length: 106 < Cache-Control: must-revalidate < {"error":"no_active_vbuckets","reason":"Cannot execute view query since the node has no active vbuckets"} * Connection #0 to host 10.3.121.105 left intact * Closing connection #0 |
| Comments |
| Comment by Farshid Ghods [ 12/Sep/12 ] |
| looks like a blocker to me , is this regression on 1717 ? have you been able to reproduce ths on earlier builds ? |
| Comment by Iryna Mironava [ 12/Sep/12 ] |
| first seen on 1715 |
| Comment by Farshid Ghods [ 12/Sep/12 ] |
|
ok this could be related to ep-engine changes?
what do you see in diags ? can you do more investigations |
| Comment by Iryna Mironava [ 12/Sep/12 ] |
| it doesn't seem to be an ep-engine problem, we can query views from nodes which was in cluster successfully, the error appears only if we query node that we try to add. In logs i don't see something wrong, it is building replicas. |
| Comment by Farshid Ghods [ 12/Sep/12 ] |
|
Aliaksey,
I dont think this is a regression but the use cases is too simple ( doing view queries from a node that is being rebalanced in but its still doing replica building and does not have an active vbucket ) |
| Comment by Aleksey Kondratenko [ 12/Sep/12 ] |
|
I need 100% clarity here. Does that happen _after_ rebalance or _during_ rebalance. I'm asking because sometimes we need quite a lot of time to start moving any active vbuckets into new node. So if it's not after, then it's simple duplicate of |
| Comment by Farshid Ghods [ 13/Sep/12 ] |
|
|
| Comment by Iryna Mironava [ 13/Sep/12 ] |
| it happens during rebalance, but for much longer time window, 10-15 minutes i cannot see documents and query views. Is it a duplicate? |
| Comment by Aleksey Kondratenko [ 13/Sep/12 ] |
|
Yes it's duplicate.
It is reasonably simple to fix, _but_ it may have some quite bad effects on clients. |
| Comment by Aleksey Kondratenko [ 13/Sep/12 ] |
| BTW this is not AFAIK a regression. Do you want me to fix this ASAP? |
| Comment by Farshid Ghods [ 13/Sep/12 ] |
| moving this to beta refresh so once beta is shipped we will merge it |
| Comment by Aleksey Kondratenko [ 14/Sep/12 ] |
| Some fix is coming. But now I understand there's another cause of this. I think enabling consistent views will fix other and potentially more prolonged it time cause. |
| Comment by Karen Zeller [ 17/Sep/12 ] |
|
Add to beta release notes as : If a cluster is being rebalanced, and a node does not yet have any active
vBuckets, you will receive the error <literal>"error":"no_active_vbuckets"</literal> when you query a view via the REST API. |
| Comment by Aleksey Kondratenko [ 21/Sep/12 ] |
| merged fix |