http://www.couchbase.com/issues This file is an XML representation of an issue en-us 5.2.4 845 26-12-2012 http://www.couchbase.com/issues/browse/MB-6612 Couchbase Server test to reproduce: viewquerytests.ViewQueryTests.test_employee_dataset_startkey_endkey_queries_rebalance_in,num_nodes_to_add=1,skip_rebalance=truen <br/> Test uses employee data set: <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-documents are structured as {&quot;name&quot;: name&lt;string&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;join_yr&quot; : year&lt;int&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;join_mo&quot; : month&lt;int&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;join_day&quot; : day&lt;int&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;email&quot;: email&lt;string&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;job_title&quot; : title&lt;string&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;type&quot; : type&lt;string&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&quot;desc&quot; : desc&lt;tring&gt;} <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Steps to repro: <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Test is just started and created views, first 1-2 seconds after view creation queries for all 6 views returned errors <br/> <br/> 6 ddocs, 1 view per ddoc: <br/> test_view-217467b <br/> test_view-820faf6 <br/> test_view-6a68c94 <br/> test_view-d74d96e <br/> test_view-b943fd9 <br/> <br/> map fns: <br/> &#39;function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(&quot;^UI &quot;); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}&#39; <br/> &#39;function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(&quot;^System &quot;); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}&#39; <br/> &#39;function (doc) { if(doc.job_title !== undefined) { var myregexp = new RegExp(&quot;^Senior &quot;); if(doc.job_title.match(myregexp)){ emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}&#39; <br/> &#39;function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }&#39; <br/> &#39;function (doc) { if(doc.job_title !== undefined) emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] ); }&#39; + _count reduce fn <br/> &#39;function (doc, meta) { if(doc.job_title !== undefined) { var myregexp = new RegExp(&quot;^admin&quot;); if(meta.id.match(myregexp)) { emit([doc.join_yr, doc.join_mo, doc.join_day], [doc.name, doc.email] );}}}&#39; <br/> <br/> exact queries: <br/> &nbsp;<a href="http://10.2.2.60:8092/default/_design/test_view-6a68c94/_view/test_view-6a68c94?debug=true&start_key=%5B2008%2C7%2Cnull%5D&connection_timeout=60000">http://10.2.2.60:8092/default/_design/test_view-6a68c94/_view/test_view-6a68c94?debug=true&amp;start_key=%5B2008%2C7%2Cnull%5D&amp;connection_timeout=60000</a> <br/> <a href="http://10.2.2.60:8092/default/_design/test_view-b943fd9/_view/test_view-b943fd9?debug=true&start_key=%5B2008%2C7%2Cnull%5D&connection_timeout=60000">http://10.2.2.60:8092/default/_design/test_view-b943fd9/_view/test_view-b943fd9?debug=true&amp;start_key=%5B2008%2C7%2Cnull%5D&amp;connection_timeout=60000</a> <br/> <br/> error while querying returned: <br/> {{{badmatch,{not_found,no_db_file}}, <br/> &nbsp;&nbsp;[{couch_db_set,&#39;-handle_call/3-fun-1-&#39;,3}, <br/> &nbsp;&nbsp;&nbsp;{lists,foldl,3}, <br/> &nbsp;&nbsp;&nbsp;{couch_db_set,handle_call,3}, <br/> &nbsp;&nbsp;&nbsp;{gen_server,handle_msg,5}, <br/> &nbsp;&nbsp;&nbsp;{proc_lib,init_p_do_apply,3}]}, <br/> &nbsp;{gen_server,call, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[&lt;0.30044.20&gt;, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{add_partitions,[24,861,862,863,864,865,866,867,868,869,870,871, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;872,873,874,875,876,877,878,879,880,881,882, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;883,884,885,886,887,888,889,890,891,892,893, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;894,895,896,897,898,899,900,901,902,903,904, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;905,906,907,908,909,910,911,912,913,914,915, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;916,917,918,919,920,921,922,923,924,925,926, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;927,928,929,930,931,932,933,934,935,936,937, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;938,939,940,941,942,943,944,945,946,947,948, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;949,950,951,952,953,954,955,956,957,958,959, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;960,961,962,963,964,965,966,967,968,969,970, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;971,972,973,974,975,976,977,978,979,980,981, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;982,983,984,985,986,987,988,989,990,991,992, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;993,994,995,996,997,998,999,1000,1001,1002, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1003,1004,1005,1006,1007,1008,1009,1010,1011, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1012,1013,1014,1015,1016,1017,1018,1019,1020, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1021,1022,1023]}, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;infinity]}} {&quot;error&quot;:&quot;{{{badmatch,{not_found,no_db_file}},\n [{couch_db_set,&#39;-handle_call/3-fun-1-&#39;,3},\n {lists,foldl,3},\n {couch_db_set,handle_call,3},\n {gen_server,handle_msg,5},\n {proc_lib,init_p_do_apply,3}]},\n {gen_server,call,\n [&lt;0.30044.20&gt;,\n {add_partitions,[24,861,862,863,864,865,866,867,868,869,870,871,\n 872,873,874,875,876,877,878,879,880,881,882,\n 883,884,885,886,887,888,889,890,891,892,893,\n 894,895,896,897,898,899,900,901,902,903,904,\n 905,906,907,908,909,910,911,912,913,914,915,\n 916,917,918,919,920,921,922,923,924,925,926,\n 927,928,929,930,931,932,933,934,935,936,937,\n 938,939,940,941,942,943,944,945,946,947,948,\n 949,950,951,952,953,954,955,956,957,958,959,\n 960,961,962,963,964,965,966,967,968,969,970,\n 971,972,973,974,975,976,977,978,979,980,981,\n 982,983,984,985,986,987,988,989,990,991,992,\n 993,994,995,996,997,998,999,1000,1001,1002,\n 1003,1004,1005,1006,1007,1008,1009,1010,1011,\n 1012,1013,1014,1015,1016,1017,1018,1019,1020,\n 1021,1022,1023]},\n infinity]}}&quot;,&quot;reason&quot;:&quot;{gen_server,call,\n [&lt;0.30029.20&gt;,\n {set_view_group_req,update_after,true,\n [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,\n 18,19,20,21,22,23,24,25,26,27,28,29,30,31,\n 32,33,34,35,36,37,38,39,40,41,42,43,44,45,\n 46,47,48,49,50,51,52,53,54,55,56,57,58,59,\n 60,61,62,63,64,65,66,67,68,69,70,71,72,73,\n 74,75,76,77,78,79,80,81,82,83,84,85,86,87,\n 88,89,90,91,92,93,94,95,96,97,98,99,100,101,\n 102,103,104,105,106,107,108,109,110,111,112,\n 113,114,115,116,117,118,119,120,121,122,123,\n 124,125,126,127,128,129,130,131,132,133,134,\n 135,136,137,138,139,140,141,142,143,144,145,\n 146,147,148,149,150,151,152,153,154,155,156,\n 157,158,159,160,161,162,163,164,165,166,167,\n 168,169,170,171,172,173,174,175,176,177,178,\n 179,180,181,182,183,184,185,186,187,188,189,\n 190,191,192,193,194,195,196,197,198,199,200,\n 201,202,203,204,205,206,207,208,209,210,211,\n 212,213,214,215,216,217,218,219,220,221,222,\n 223,224,225,226,227,228,229,230,231,232,233,\n 234,235,236,237,238,239,240,241,242,243,244,\n 245,246,247,248,249,250,251,252,253,254,255,\n 256,257,258,259,260,261,262,263,264,265,266,\n 267,268,269,270,271,272,273,274,275,276,277,\n 278,279,280,281,282,283,284,285,286,287,288,\n 289,290,291,292,293,294,295,296,297,298,299,\n 300,301,302,303,304,305,306,307,308,309,310,\n 311,312,313,314,315,316,317,318,319,320,321,\n 322,323,324,325,326,327,328,329,330,331,332,\n 333,334,335,336,337,338,339,340,341,342,343,\n 344,345,346,347,348,349,350,351,352,353,354,\n 355,356,357,358,359,360,361,362,363,364,365,\n 366,367,368,369,370,371,372,373,374,375,376,\n 377,378,379,380,381,382,383,384,385,386,387,\n 388,389,390,391,392,393,394,395,396,397,398,\n 399,400,401,402,403,404,405,406,407,408,409,\n 410,411,412,413,414,415,416,417,418,419,420,\n 421,422,423,424,425,426,427,428,429,430,431,\n 432,433,434,435,436,437,438,439,440,441,442,\n 443,444,445,446,447,448,449,450,451,452,453,\n 454,455,456,457,458,459,460,461,462,463,464,\n 465,466,467,468,469,470,471,472,473,474,475,\n 476,477,478,479,480,481,482,483,484,485,486,\n 487,488,489,490,491,492,493,494,495,496,497,\n 498,499,500,501,502,503,504,505,506,507,508,\n 509,510,511,512,513,514,515,516,517,518,519,\n 520,521,522,523,524,525,526,527,528,529,530,\n 531,532,533,534,535,536,537,538,539,540,541,\n 542,543,544,545,546,547,548,549,550,551,552,\n 553,554,555,556,557,558,559,560,561,562,563,\n 564,565,566,567,568,569,570,571,572,573,574,\n 575,576,577,578,579,580,581,582,583,584,585,\n 586,587,588,589,590,591,592,593,594,595,596,\n 597,598,599,600,601,602,603,604,605,606,607,\n 608,609,610,611,612,613,614,615,616,617,618,\n 619,620,621,622,623,624,625,626,627,628,629,\n 630,631,632,633,634,635,636,637,638,639,640,\n 641,642,643,644,645,646,647,648,649,650,651,\n 652,653,654,655,656,657,658,659,660,661,662,\n 663,664,665,666,667,668,669,670,671,672,673,\n 674,675,676,677,678,679,680,681,682,683,684,\n 685,686,687,688,689,690,691,692,693,694,695,\n 696,697,698,699,700,701,702,703,704,705,706,\n 707,708,709,710,711,712,713,714,715,716,717,\n 718,719,720,721,722,723,724,725,726,727,728,\n 729,730,731,732,733,734,735,736,737,738,739,\n 740,741,742,743,744,745,746,747,748,749,750,\n 751,752,753,754,755,756,757,758,759,760,761,\n 762,763,764,765,766,767,768,769,770,771,772,\n 773,774,775,776,777,778,779,780,781,782,783,\n 784,785,786,787,788,789,790,791,792,793,794,\n 795,796,797,798,799,800,801,802,803,804,805,\n 806,807,808,809,810,811,812,813,814,815,816,\n 817,818,819,820,821,822,823,824,825,826,827,\n 828,829,830,831,832,833,834,835,836,837,838,\n 839,840,841,842,843,844,845,846,847,848,849,\n 850,851,852,853,854,855,856,857,858,859,860,\n 861,862,863,864,865,866,867,868,869,870,871,\n 872,873,874,875,876,877,878,879,880,881,882,\n 883,884,885,886,887,888,889,890,891,892,893,\n 894,895,896,897,898,899,900,901,902,903,904,\n 905,906,907,908,909,910,911,912,913,914,915,\n 916,917,918,919,920,921,922,923,924,925,926,\n 927,928,929,930,931,932,933,934,935,936,937,\n 938,939,940,941,942,943,944,945,946,947,948,\n 949,950,951,952,953,954,955,956,957,958,959,\n 960,961,962,963,964,965,966,967,968,969,970,\n 971,972,973,974,975,976,977,978,979,980,981,\n 982,983,984,985,986,987,988,989,990,991,992,\n 993,994,995,996,997,998,999,1000,1001,1002,\n 1003,1004,1005,1006,1007,1008,1009,1010,\n 1011,1012,1013,1014,1015,1016,1017,1018,\n 1019,1020,1021,1022,1023],\n true,main},\n infinity]}&quot;} centos, 1 node, 1 bucket <br/> build 1712 MB-6612

queries returns error {badmatch,{not_found,no_db_file}

Bug Blocker Closed Fixed Public Peter Wansch Iryna Mironava regression Tue, 11 Sep 2012 17:41:03 -0500 Thu, 10 Jan 2013 01:55:21 -0600 Tue, 25 Sep 2012 13:56:47 -0500 2.0-beta-2 ns_server 0 1 Potentially ns_server bug Cannot find any occurences of no_db_ in logs attached. Please, double check on your end See above there were no such messages in the log, just queries responses had it I see no trace of this pid either.<br/> <br/> Most likely logs have rotated past incident.<br/> <br/> May I have at least error level logs in case they still have some traces ? error logs attached logs of error level are attached Now I see multiple occurences of this problem. But sadly it was &#39;too long time ago&#39;. May I ask you to re-run your tests and capture diags at the middle ? logs are attached promoting this to blocker since this happens more frequenetly now and its easy to reproduce <br/> Found bug. Fixed and will soon upload to gerrit Fix is 3 commits ending here: <a href="http://review.couchbase.org/20813">http://review.couchbase.org/20813</a> Peter,<br/> <br/> given this issue is related to view engine can Filipe take a look ? Seeing this on system tests as well. build-1717 [error_logger:error,2012-09-14T2:08:03.441,<a href='mailto:ns_1@10.3.121.16'>ns_1@10.3.121.16</a>:error_logger:ale_error_logger_handler:log_report:72] <br/> =========================CRASH REPORT========================= <br/> &nbsp;&nbsp;crasher: <br/> &nbsp;&nbsp;&nbsp;&nbsp;initial call: couch_db_set:init/1 <br/> &nbsp;&nbsp;&nbsp;&nbsp;pid: &lt;0.17473.73&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;registered_name: [] <br/> &nbsp;&nbsp;&nbsp;&nbsp;exception exit: {{badmatch,{not_found,no_db_file}}, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[{couch_db_set,&#39;-handle_call/3-fun-1-&#39;,3}, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{lists,foldl,3}, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{couch_db_set,handle_call,3}, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{gen_server,handle_msg,5}, <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{proc_lib,init_p_do_apply,3}]} <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;in function gen_server:terminate/6 <br/> &nbsp;&nbsp;&nbsp;&nbsp;ancestors: [&lt;0.17468.73&gt;,&lt;0.17346.73&gt;,&lt;0.17345.73&gt;] <br/> &nbsp;&nbsp;&nbsp;&nbsp;messages: [] <br/> &nbsp;&nbsp;&nbsp;&nbsp;links: [&lt;0.17468.73&gt;] <br/> &nbsp;&nbsp;&nbsp;&nbsp;dictionary: [] <br/> &nbsp;&nbsp;&nbsp;&nbsp;trap_exit: false <br/> &nbsp;&nbsp;&nbsp;&nbsp;status: running <br/> &nbsp;&nbsp;&nbsp;&nbsp;heap_size: 2584 <br/> &nbsp;&nbsp;&nbsp;&nbsp;stack_size: 24 <br/> &nbsp;&nbsp;&nbsp;&nbsp;reductions: 38041 <br/> &nbsp;&nbsp;neighbours: <br/> fix is still sitting in gerrit Karan, <br/> <br/> was this during rebalancing ? <br/> did the error go away after retrying ? Integrated in github-couchdb-preview #507 (See [<a href="http://qa.hq.northscale.net/job/github-couchdb-preview/507/">http://qa.hq.northscale.net/job/github-couchdb-preview/507/</a>])<br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: stop maintaining open dbs lru (Revision 7587aa0cf6af53e49487771d1ab31eb425b29cee)<br/> <a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: open databases synchronously (Revision fa5b6fe30dbf84e03dec273c92ba6e17b2621ff7)<br/> <a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: just store pid in couch server ets tables (Revision 261348623337e84e38635c2470c95bec3b7aa757)<br/> <a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: exclude opening dbs from all_known_databases_with_prefix (Revision b94592131af1efb170f5710f65b142d4d786dc0f)<br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Result = SUCCESS<br/> pwansch : <br/> Files : <br/> * src/couchdb/couch_server.erl<br/> <br/> pwansch : <br/> Files : <br/> * src/couchdb/couch_server.erl<br/> <br/> pwansch : <br/> Files : <br/> * src/couchdb/couch_server.erl<br/> <br/> pwansch : <br/> Files : <br/> * src/couchdb/couch_server.erl<br/> Integrated in github-couchdb-preview #508 (See [<a href="http://qa.hq.northscale.net/job/github-couchdb-preview/508/">http://qa.hq.northscale.net/job/github-couchdb-preview/508/</a>])<br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-6736" title="Querying a view on a single node gives inconsistent_state error if bucket is empty"><strike>MB-6736</strike></a>: Revert &quot;<a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: exclude opening dbs from...&quot; (Revision 804ef001394269d3c02e3eb1ef05ac3a02f86c54)<br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Result = SUCCESS<br/> Farshid Ghods : <br/> Files : <br/> * src/couchdb/couch_server.erl<br/> RN text: Querying a view during rebalance had resulted in several errors due to the way database <br/> storage files were managed and named. We now exclude opening <br/> databases that are meant to be excluded from indexing and we now <br/> synchronously open databases <br/> for indexing to resolve the problem. Final relevant commit is this: <br/> <br/> commit 8ba83d2df2b35ebd8fbc621dcad193aec2255fe5 <br/> Author: Aliaksey Kandratsenka &lt;<a href='mailto:alk@tut.by'>alk@tut.by</a>&gt; <br/> Date: Mon Aug 27 17:44:43 2012 -0700 <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-6413" title="7 empty buckets is nearly killing server [was: Couchbase causes unreasonable high CPU utilization without any connected clients]"><strike>MB-6413</strike></a>: re-implemented bucket_databases using efficient API <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;As of <a href="http://review.couchbase.org/20223">http://review.couchbase.org/20223</a> couchdb supports listing all <br/> &nbsp;&nbsp;&nbsp;&nbsp;known vbuckets with given prefix. That allows us to only touch <br/> &nbsp;&nbsp;&nbsp;&nbsp;vbuckets of bucket we need. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;And it also opens all databases at startup. That allows us to avoid <br/> &nbsp;&nbsp;&nbsp;&nbsp;using slow all_databases call. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Change-Id: I656000e408af6b977cb27e981d216f6ea11ac0cb <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-on: <a href="http://review.couchbase.org/20224">http://review.couchbase.org/20224</a> <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-by: Damien Katz &lt;<a href='mailto:damien@couchbase.com'>damien@couchbase.com</a>&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Tested-by: Aliaksey Kandratsenka &lt;<a href='mailto:alkondratenko@gmail.com'>alkondratenko@gmail.com</a>&gt; <br/> <br/> So we first fixed <a href="http://www.couchbase.com/issues/browse/MB-6413" title="7 empty buckets is nearly killing server [was: Couchbase causes unreasonable high CPU utilization without any connected clients]"><strike>MB-6413</strike></a> where: <br/> <br/> Underlying issue was due to race in a way couch implemented all_databases that ns_server used to enumerate all physically present vbuckets. Particularly it used readdir which is known to be not atomic with respect to renames. And in practice it raced with final rename at the end of compaction causing some vbuckets to disappear. Which caused view queries to fail because we incorrectly removed valid vbucket from set of indexed vbuckets in all views. <br/> <br/> Fix was to stop doing slow and unreliable all_databases and just keep list of all open vbuckets in public ordered ets table. Thus we fixed race above and at the same time we&#39;ve got efficient way to list just vbuckets of some given bucket. <br/> <br/> In this bug (<a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>) we discovered that fix wasn&#39;t quite correctly implemented as pointed out in one of commits: <br/> <br/> commit fa5b6fe30dbf84e03dec273c92ba6e17b2621ff7 <br/> Author: Aliaksey Kandratsenka &lt;<a href='mailto:alk@tut.by'>alk@tut.by</a>&gt; <br/> Date: Wed Sep 12 18:30:52 2012 -0700 <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: open databases synchronously <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Problem that we see in <a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a> is that all_known_databases_with_prefix <br/> &nbsp;&nbsp;&nbsp;&nbsp;is scanning couch_dbs_by_name ets table. Given that previously we also <br/> &nbsp;&nbsp;&nbsp;&nbsp;stored dbs being opened that lead to condition where some vbuckets <br/> &nbsp;&nbsp;&nbsp;&nbsp;would be thought as present while in fact they are simply being tried <br/> &nbsp;&nbsp;&nbsp;&nbsp;to be opened and do not actually exist. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Given that we don&#39;t need async db opening in couchbase fork of couchdb <br/> &nbsp;&nbsp;&nbsp;&nbsp;for quite some time it seems logical to just get rid of async open and <br/> &nbsp;&nbsp;&nbsp;&nbsp;thus never have onopened dbs in ets table. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Change-Id: I5d1dad5f60c64d197e143cf4a7be1996a4fc4ea2 <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-on: <a href="http://review.couchbase.org/20812">http://review.couchbase.org/20812</a> <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-by: Damien Katz &lt;<a href='mailto:damien@couchbase.com'>damien@couchbase.com</a>&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Tested-by: Aliaksey Kandratsenka &lt;<a href='mailto:alkondratenko@gmail.com'>alkondratenko@gmail.com</a>&gt; <br/> <br/> As part of doing this work we also removed legacy couchdb behavior where some unused dbs were closed. I.e.: <br/> <br/> commit 261348623337e84e38635c2470c95bec3b7aa757 <br/> Author: Aliaksey Kandratsenka &lt;<a href='mailto:alk@tut.by'>alk@tut.by</a>&gt; <br/> Date: Wed Sep 12 18:34:04 2012 -0700 <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: just store pid in couch server ets tables <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Instead of {opened,..} tuple. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Change-Id: Ie88c1137fa55f55e8b9f1128c12f15750d4b36bd <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-on: <a href="http://review.couchbase.org/20813">http://review.couchbase.org/20813</a> <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-by: Damien Katz &lt;<a href='mailto:damien@couchbase.com'>damien@couchbase.com</a>&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Tested-by: Aliaksey Kandratsenka &lt;<a href='mailto:alkondratenko@gmail.com'>alkondratenko@gmail.com</a>&gt; <br/> <br/> commit 7587aa0cf6af53e49487771d1ab31eb425b29cee <br/> Author: Aliaksey Kandratsenka &lt;<a href='mailto:alk@tut.by'>alk@tut.by</a>&gt; <br/> Date: Wed Sep 12 18:14:32 2012 -0700 <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a>: stop maintaining open dbs lru <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Because that&#39;s long unused and simplifies things for actual <a href="http://www.couchbase.com/issues/browse/MB-6612" title="queries returns error {badmatch,{not_found,no_db_file}"><strike>MB-6612</strike></a> <br/> &nbsp;&nbsp;&nbsp;&nbsp;fix. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Change-Id: Ie8ac7e5f6d03e962f181d84461cc26ee53cb7309 <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-on: <a href="http://review.couchbase.org/20811">http://review.couchbase.org/20811</a> <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-by: Damien Katz &lt;<a href='mailto:damien@couchbase.com'>damien@couchbase.com</a>&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Tested-by: Aliaksey Kandratsenka &lt;<a href='mailto:alkondratenko@gmail.com'>alkondratenko@gmail.com</a>&gt; <br/> <br/> And after all that we found another issue in original implementation: <br/> <br/> commit 7a0f8699723d3557bcc71d1964464a0e1248c7cf <br/> Author: Aliaksey Artamonau &lt;<a href='mailto:aliaksiej.artamonau@gmail.com'>aliaksiej.artamonau@gmail.com</a>&gt; <br/> Date: Fri Oct 26 13:56:30 2012 -0700 <br/> <br/> &nbsp;&nbsp;&nbsp;&nbsp;<a href="http://www.couchbase.com/issues/browse/MB-7025" title="Not able to setup replication , getting a Server error during create replication {what,{badmatch,{not_found,no_db_file}}},"><strike>MB-7025</strike></a> Shutdown databases in couch_server:terminate correctly. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Since recently couch_dbs_by_name contains only PID and nothing <br/> &nbsp;&nbsp;&nbsp;&nbsp;more. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;It&#39;s not clear though if this actually fixes referred bug. <br/> &nbsp;&nbsp;&nbsp;&nbsp; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Change-Id: Ifb3a0bd9a9c011c4836e0513a4adbaba94c3c8c1 <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-on: <a href="http://review.couchbase.org/22021">http://review.couchbase.org/22021</a> <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-by: Aliaksey Kandratsenka &lt;<a href='mailto:alkondratenko@gmail.com'>alkondratenko@gmail.com</a>&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Tested-by: Aliaksey Kandratsenka &lt;<a href='mailto:alkondratenko@gmail.com'>alkondratenko@gmail.com</a>&gt; <br/> &nbsp;&nbsp;&nbsp;&nbsp;Reviewed-by: Filipe David Borba Manana &lt;<a href='mailto:fdmanana@gmail.com'>fdmanana@gmail.com</a>&gt; <br/> Date of First Response Tue, 11 Sep 2012 17:45:53 -0500 Rank 4002 Sprint Status Time In Status