views dont return any results ( couchdb fails to open the database doc )

Farshid Ghods added a comment - 18/Jun/12 4:38 PM   ./testrunner -i ../../ini/single-node-win-64.ini -t viewtests.ViewBasicTests.test_view_on_x_docs,num-docs=1000 [Test Input params] : {'num-docs': '1000', 'cluster_name': 'single-node-win-64', 'num_nodes': 1} test_view_on_x_docs (viewtests.ViewBasicTests) ... [2012-06-18 14:36:29,059] - [remote_util:111] INFO - connecting to 10.1.2.68 with username : Administrator password : Membase123 ssh_key: [2012-06-18 14:36:29,169] - [transport:1386] INFO - Connected (version 2.0, client OpenSSH_5.8) [2012-06-18 14:36:34,298] - [transport:1386] INFO - Authentication (password) successful! [2012-06-18 14:36:34,307] - [remote_util:129] INFO - Connected [2012-06-18 14:36:34,340] - [transport:1386] INFO - Secsh channel 1 opened. [2012-06-18 14:36:34,449] - [sftp:132] INFO - [chan 1] Opened sftp connection (server version 3) [2012-06-18 14:36:34,590] - [remote_util:1093] INFO - running command.raw systeminfo | grep 'System Type:' [2012-06-18 14:36:34,605] - [transport:1386] INFO - Secsh channel 2 opened. [2012-06-18 14:36:41,891] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:36:41,892] - [remote_util:1093] INFO - running command.raw systeminfo | grep 'OS Name: ' [2012-06-18 14:36:42,122] - [transport:1386] INFO - Secsh channel 3 opened. [2012-06-18 14:36:47,623] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:36:47,623] - [remote_util:1210] INFO - ['OS Name: Microsoft Windows Server 2008 R2 Enterprise '] [2012-06-18 14:36:47,653] - [transport:1386] INFO - Secsh channel 4 opened. [2012-06-18 14:36:47,747] - [sftp:132] INFO - [chan 4] Opened sftp connection (server version 3) [2012-06-18 14:36:47,934] - [sftp:132] INFO - [chan 4] sftp session closed. [2012-06-18 14:36:47,982] - [transport:1386] INFO - Secsh channel 5 opened. [2012-06-18 14:36:48,091] - [sftp:132] INFO - [chan 5] Opened sftp connection (server version 3) [2012-06-18 14:36:48,294] - [remote_util:1093] INFO - running command.raw systeminfo | grep 'System Type:' [2012-06-18 14:36:48,309] - [transport:1386] INFO - Secsh channel 6 opened. [2012-06-18 14:36:53,733] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:36:53,733] - [remote_util:1093] INFO - running command.raw systeminfo | grep 'OS Name: ' [2012-06-18 14:36:53,762] - [transport:1386] INFO - Secsh channel 7 opened. [2012-06-18 14:36:59,140] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:36:59,141] - [remote_util:1210] INFO - ['OS Name: Microsoft Windows Server 2008 R2 Enterprise '] [2012-06-18 14:36:59,357] - [transport:1386] INFO - Secsh channel 8 opened. [2012-06-18 14:36:59,419] - [sftp:132] INFO - [chan 8] Opened sftp connection (server version 3) [2012-06-18 14:36:59,513] - [remote_util:1093] INFO - running command.raw systeminfo | grep 'System Type:' [2012-06-18 14:36:59,528] - [transport:1386] INFO - Secsh channel 9 opened. [2012-06-18 14:37:04,921] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:37:04,923] - [remote_util:1093] INFO - running command.raw systeminfo | grep 'OS Name: ' [2012-06-18 14:37:05,154] - [transport:1386] INFO - Secsh channel 10 opened. [2012-06-18 14:37:10,515] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:37:10,516] - [remote_util:1210] INFO - ['OS Name: Microsoft Windows Server 2008 R2 Enterprise '] [2012-06-18 14:37:10,517] - [remote_util:1093] INFO - running command.raw net start couchbaseserver [2012-06-18 14:37:10,734] - [transport:1386] INFO - Secsh channel 11 opened. [2012-06-18 14:37:10,936] - [remote_util:1122] INFO - command executed successfully [2012-06-18 14:37:10,937] - [remote_util:1072] ERROR - The requested service has already been started. [2012-06-18 14:37:10,938] - [remote_util:1072] ERROR - [2012-06-18 14:37:10,938] - [remote_util:1072] ERROR - More help is available by typing NET HELPMSG 2182. [2012-06-18 14:37:10,938] - [remote_util:1072] ERROR - [2012-06-18 14:37:11,040] - [bucket_helper:123] INFO - deleting existing buckets on [ip:10.1.2.68 ssh_username:Administrator] [2012-06-18 14:37:11,069] - [cluster_helper:83] INFO - waiting for ns_server @ 10.1.2.68:8091 [2012-06-18 14:37:11,089] - [rest_client:516] INFO - settings/web params : username=Administrator&password=password&port=8091 [2012-06-18 14:37:11,094] - [rest_client:542] INFO - pools/default params : memoryQuota=2184&username=Administrator&password=password [2012-06-18 14:37:11,120] - [cluster_helper:83] INFO - waiting for ns_server @ 10.1.2.68:8091 [2012-06-18 14:37:11,128] - [bucket_helper:123] INFO - deleting existing buckets on [ip:10.1.2.68 ssh_username:Administrator] [2012-06-18 14:37:11,139] - [rest_client:95] INFO - existing buckets : [] [2012-06-18 14:37:11,147] - [rest_client:1098] INFO - http://10.1.2.68:8091//pools/default/buckets with param: proxyPort=11211&bucketType=membase&authType=sasl&replicaIndex=0&name=default&saslPassword=&replicaNumber=1&ramQuotaMB=1456 [2012-06-18 14:37:11,166] - [rest_client:1103] INFO - 0.0187711715698 seconds to create bucket default [2012-06-18 14:37:11,166] - [bucket_helper:215] INFO - waiting for memcached bucket : default in 10.1.2.68 to accept set ops [2012-06-18 14:37:23,825] - [data_helper:286] INFO - creating direct client 10.1.2.68:11210 default [2012-06-18 14:37:24,481] - [rest_client:95] INFO - existing buckets : [u'default'] [2012-06-18 14:37:24,481] - [rest_client:98] INFO - found bucket default [2012-06-18 14:37:24,492] - [viewtests:157] INFO - description : create a view on 1000 documents [2012-06-18 14:37:24,598] - [data_helper:327] INFO - creating proxy client 10.1.2.68:11211 default [2012-06-18 14:37:24,600] - [viewtests:169] INFO - inserting 1000 json objects [2012-06-18 14:37:25,490] - [viewtests:175] INFO - inserted 1000 json documents [2012-06-18 14:37:30,492] - [viewtests:459] INFO - Params {'connection_timeout': 60000, 'full_set': 'true', 'stale': 'update_after'} [2012-06-18 14:37:30,492] - [rest_client:233] INFO - index query url: http://10.1.2.68:8091/couchBase/default/_design/dev_test_view_on_1000_docs-766c51c/_view/dev_test_view_on_1000_docs-766c51c?connection_timeout=60000&full_set=true&stale=update_after [2012-06-18 14:37:30,556] - [viewtests:475] INFO - view returned empty results in 0.0641360282898 seconds, sleeping for 10 [2012-06-18 14:37:40,567] - [viewtests:459] INFO - Params {'connection_timeout': 60000, 'full_set': 'true', 'stale': 'update_after'} [2012-06-18 14:37:40,567] - [rest_client:233] INFO - index query url: http://10.1.2.68:8091/couchBase/default/_design/dev_test_view_on_1000_docs-766c51c/_view/dev_test_view_on_1000_docs-766c51c?connection_timeout=60000&full_set=true&stale=update_after [2012-06-18 14:37:40,580] - [viewtests:475] INFO - view returned empty results in 0.0131189823151 seconds, sleeping for 10 ^CTraceback (most recent call last):   File "./testrunner", line 258, in <module>     result = unittest.TextTestRunner(verbosity=2).run(suite)   File "/opt/python2.7/lib/python2.7/unittest/runner.py", line 151, in run     test(result)   File "/opt/python2.7/lib/python2.7/unittest/suite.py", line 70, in __call__     return self.run(*args, **kwds)   File "/opt/python2.7/lib/python2.7/unittest/suite.py", line 108, in run     test(result)   File "/opt/python2.7/lib/python2.7/unittest/case.py", line 391, in __call__     return self.run(*args, **kwds)   File "/opt/python2.7/lib/python2.7/unittest/case.py", line 327, in run     testMethod()   File "pytests/viewtests.py", line 977, in test_view_on_x_docs     ViewBaseTests._test_view_on_multiple_docs(self, self.num_docs)   File "pytests/viewtests.py", line 177, in _test_view_on_multiple_docs     results = ViewBaseTests._get_view_results(self, rest, bucket, view_name, len(doc_names), extra_params=params)   File "pytests/viewtests.py", line 476, in _get_view_results     time.sleep(timeout)

Farshid Ghods added a comment - 18/Jun/12 4:43 PM please look at the second diag file i attached.  i started from fresh install , created 1000 docs and ran view. couchdb is unable to load data from disk [couchdb:info] [2012-06-18 14:37:24] [ ns_1@10.1.2.68 :<0.11370.0>:couch_log:info:39] Updater reading changes from active partitions to update main set view group `_design/dev_test_view_on_1000_docs-766c51c` from set `default` [couchdb:info] [2012-06-18 14:37:24] [ ns_1@10.1.2.68 :<0.11370.0>:couch_log:error:42] Set view `default`, main group `_design/dev_test_view_on_1000_docs-766c51c`, doc loader error error: {badmatch,eof} stacktrace: [{couch_db,open_doc_int,3},              {couch_set_view_updater,load_doc,4},              {couch_set_view_updater,'-load_changes/7-fun-0-',6},              {couch_btree,stream_kv_node2,8},              {couch_btree,fold,4},              {couch_db,enum_docs_since,5},              {couch_db,fast_reads,2},              {couch_set_view_updater,'-load_changes/7-fun-2-',6}] [couchdb:info] [2012-06-18 14:37:24] [ ns_1@10.1.2.68 :<0.9823.0>:couch_log:error:42] Set view `default`, main group `_design/dev_test_view_on_1000_docs-766c51c`, received error from updater: {badmatch,                                                                                                            eof} [couchdb:info] [2012-06-18 14:37:34] [ ns_1@10.1.2.68 :<0.9823.0>:couch_log:info:39] Starting updater for set view `default`, main group `_design/dev_test_view_on_1000_docs-766c51c`

Farshid Ghods added a comment - 18/Jun/12 4:47 PM ok it seems like couchstore is not creating couch database files. as you can see here there are no files under default directory c:\Program Files\Couchbase\Server\var\lib\couchbase\data\default>dir  Volume in drive C has no label.  Volume Serial Number is 30CB-811B  Directory of c:\Program Files\Couchbase\Server\var\lib\couchbase\data\default 06/18/2012 02:44 PM <DIR> . 06/18/2012 02:44 PM <DIR> .. 06/18/2012 02:42 PM 16,463 master.couch.1                1 File(s) 16,463 bytes                2 Dir(s) 47,743,266,816 bytes free                            

Farshid Ghods added a comment - 18/Jun/12 4:47 PM to repro you can create a bucket and get the directory listing from c:\Program Files\Couchbase\Server\var\lib\couchbase\data\default

Filipe Manana added a comment - 18/Jun/12 5:05 PM Clearly a problem elsewhere outside the view engine (who only reads from databases).

Trond Norbye added a comment - 19/Jun/12 2:23 AM bucket_engine don't know about views/database files...

Filipe Manana added a comment - 19/Jun/12 7:07 AM Ah sorry, I meant couchbase bucket.

Peter Wansch added a comment - 20/Jun/12 1:15 PM Hi Jin, are you thinking of assigning this to Aaron for initial investigation on Windows?

Chiyoung Seo added a comment - 20/Jun/12 4:25 PM Aaron, Please do the initial investigation for this file open failure.

Filipe Manana added a comment - 25/Jun/12 12:14 PM Probably when opening the file on Windows, couch store needs to pass file share options, so that it gets POSIX-alike file semantics. We had this problem with older versions of Erlang/OTP and upstream CouchDB - it got fixed by passing the file share options to the windows file open library call. See the following OTP file driver source for pointers: https://github.com/erlang/otp/blob/maint/erts/emulator/drivers/win32/win_efile.c#L726

Filipe Manana added a comment - 25/Jun/12 12:23 PM More specifically, Jan contributed in past a patch that fixed all compaction file-switch issues on Windows: https://github.com/erlang/otp/commit/426a5e21ab8306517ecd6d73f253d8dc13e39925#erts/emulator/drivers/win32/win_efile.c It's thanks to this that upstream CouchDB works on Windows.

Farshid Ghods added a comment - 27/Jun/12 12:45 AM still seeing this issue with the latest build which has this commit commit f26e358f7112dcb28cda684a12e2999912d7a49c Author: Aaron Miller < apage43@ninjawhale.com > Date: Tue Jun 26 11:12:05 2012 -0700      MB-5601 Windows file access sharing fix          Change-Id: I209d98c33820db4372fa06a56d7a8f747ccd8cba     Reviewed-on: http://review.couchbase.org/17634     Reviewed-by: Damien Katz < damien@couchbase.com >     Tested-by: Aaron Miller < apage43@ninjawhale.com > will attach new diags in a few mins