[MB-12397] couchbase-cli tool Does not have a list of commands below. Created: 22/Oct/14  Updated: 22/Oct/14

Status: Open
Project: Couchbase Server
Component/s: documentation
Affects Version/s: 3.0
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Patrick Varley Assignee: Ruth Harris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://docs.couchbase.com/admin/admin/CLI/cbcli-intro.html


 Description   
The cbcli-intro (http://docs.couchbase.com/admin/admin/CLI/cbcli-intro.html) suggest that there is a list of commands below:

COMMAND is a command listed below.

The commands have been move to another page, we should link to them:

COMMAND is a command listed [here](http://docs.couchbase.com/admin/admin/CLI/CBcli/cbcli-commands.html)




[MB-12396] Projector continues to retry UPR Connection even after deletion of a bucket with secondary index(es) Created: 22/Oct/14  Updated: 22/Oct/14

Status: Open
Project: Couchbase Server
Component/s: secondary-index
Affects Version/s: sherlock
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Prathibha Bisarahalli Assignee: Sriram Melkote
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
Create a bucket and 2i in it.
Delete the bucket.

From projector logs, we can see that it is continuing to retry connection with the bucket -

2014/10/22 15:36:43 go-couchbase: Error connecting to upr feed of 127.0.0.1:12000: MCResponse status=0x20, opcode=SASL_AUTH, opaque=0, msg: Auth failure
2014/10/22 15:36:43 go-couchbase: Error connecting to upr feed of 127.0.0.1:12000: MCResponse status=0x20, opcode=SASL_AUTH, opaque=0, msg: Auth failure
2014/10/22 15:37:13 go-couchbase: UPR connection lost; reconnecting to bucket "test1" in 30s
2014/10/22 15:37:13 go-couchbase: UPR connection lost; reconnecting to bucket "test1" in 30s
2014/10/22 15:37:13 Unable to refresh bucket HTTP error 404 Object Not Found getting "http://127.0.0.1:9001/pools/default/buckets/test1?bucket_uuid=b47791fa38e35cda4e0ea4e7d4ad9ceb": Requested resource not found.
 
2014/10/22 15:37:13 Unable to refresh bucket HTTP error 404 Object Not Found getting "http://127.0.0.1:9001/pools/default/buckets/test1?bucket_uuid=b47791fa38e35cda4e0ea4e7d4ad9ceb": Requested resource not found.





Test Automation Advancements for Sherlock (MB-12223)

[MB-12395] Generate test reports with possible security weaknesses for Forest DB Created: 21/Oct/14  Updated: 21/Oct/14

Status: Open
Project: Couchbase Server
Component/s: test-execution
Affects Version/s: sherlock
Fix Version/s: sherlock
Security Level: Public

Type: Technical task Priority: Major
Reporter: Raju Suravarjjala Assignee: Tommie McAfee
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I have found this tool which might help us in finding possible security weakness in Forest db http://www.dwheeler.com/flawfinder/

If you know of a better tool, feel free to use that one instead




Test Automation Advancements for Sherlock (MB-12223)

[MB-12394] Code coverage report for secondary index tests Created: 21/Oct/14  Updated: 21/Oct/14

Status: Open
Project: Couchbase Server
Component/s: test-execution
Affects Version/s: sherlock
Fix Version/s: sherlock
Security Level: Public

Type: Technical task Priority: Major
Reporter: Raju Suravarjjala Assignee: Parag Agarwal
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[MB-12393] 'explain' stmt does not generated expected explain output Created: 21/Oct/14  Updated: 21/Oct/14

Status: Open
Project: Couchbase Server
Component/s: query
Affects Version/s: 2.5.0, 3.0, sherlock
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Ketaki Gangal Assignee: Gerald Sangudi
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: * longevity cluster.

Triage: Untriaged
Is this a Regression?: Yes

 Description   
1. Create bucket here http://172.23.107.102:8091
2. Couchbase version 2.5.X
3. pull latest code from github/query
4. Create Primary Index on bucket
4. Run stmt for EXPLAIN - I expect it to show IndexScan/ Fetch or any plan information on what was selected but the resultset does not contain any valuable information.

What is the expected output from EXPLAIN with n1ql?

cbq> EXPLAIN select * from `sCustomer`;
{
    "signature": "json",
    "results": [
    ],
    "state": "success",
    "metrics": {
        "elapsedTime": "696.153us",
        "executionTime": "633.554us",
        "resultCount": 0
    }
}







[MB-12392] Update current readme on github/query to include steps to setup the cbq-engine Created: 21/Oct/14  Updated: 21/Oct/14

Status: In Progress
Project: Couchbase Server
Component/s: query
Affects Version/s: 3.0, sherlock
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Major
Reporter: Ketaki Gangal Assignee: Gerald Sangudi
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently the readme does not contain setup steps for the cbq-engine https://github.com/couchbaselabs/query/blob/master/README.md

It would be nice to have something like https://github.com/couchbaselabs/tuqtng/blob/master/README.md

for detailed instructions on env variables, package dependencies during the setup.






[MB-12391] Cannot start indexer or projector due to dependency on forestdb:: : libforestdb.so: cannot open shared object file: Created: 21/Oct/14  Updated: 22/Oct/14

Status: Open
Project: Couchbase Server
Component/s: secondary-index
Affects Version/s: sherlock
Fix Version/s: sherlock
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Parag Agarwal Assignee: Chris Hillery
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: centos 6x, Version: 3.5.0 Enterprise Edition (build-18)


Triage: Untriaged
Is this a Regression?: Unknown

 Description   
Installed latest sherlock build and tried to run indexer and projector

[root@palm-10307 bin]# ./projector --help
./projector: error while loading shared libraries: libforestdb.so: cannot open shared object file: No such file or directory
[root@palm-10307 bin]# ./indexer --help
./indexer: error while loading shared libraries: libforestdb.so: cannot open shared object file: No such file or directory
[root@palm-10307 bin]# ./indexer
./indexer: error while loading shared libraries: libforestdb.so: cannot open shared object file: No such file or directory
[root@palm-10307 bin]# ./projector -debug localhost:9001 -kvaddrs="127.0.0.1:12002”
> -bash: unexpected EOF while looking for matching `"'
-bash: syntax error: unexpected end of file
[root@palm-10307 bin]# ./projector -debug localhost:9001 -kvaddrs="127.0.0.1:12002"
./projector: error while loading shared libraries: libforestdb.so: cannot open shared object file: No such file or directory
[root@palm-10307 bin]#

 Comments   
Comment by Sarath Lakshman [ 22/Oct/14 ]
We need to provide rpath while building indexer using cgo libraries.

Here is the corresponding change:
https://github.com/t3rm1n4l/indexing-build-scripts/commit/981ac7379f4bc58d5f885427f4a6a5e744be7378
Comment by Sarath Lakshman [ 22/Oct/14 ]
Ceej, it would be great if you could fix this problem in your toy build.




[MB-12390] select count(*) or select * fails for any bucket if no explicit primary index is defined Created: 21/Oct/14  Updated: 21/Oct/14

Status: In Progress
Project: Couchbase Server
Component/s: query
Affects Version/s: 3.0, sherlock
Fix Version/s: cbq-DP4
Security Level: Public

Type: Bug Priority: Critical
Reporter: Ketaki Gangal Assignee: Gerald Sangudi
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Centos 64-bit
Is this a Regression?: Unknown

 Description   
1.install couchbase 3.0 or 3.5
2. build.sh
3../server/main/cbq-engine -datastore http://10.1.3.176:8091
4. startup shell/cbq client

5. Run query like
cbq> select COUNT(*) from beer-sample;
syntax error

 select COUNT(*) from k0;
{
    "signature": {
        "$1": "number"
    },
    "results": [
        {
            "$1": 0
        }
    ],
    "errors": [
        {
            "caller": "couchbase:288",
            "code": 5000,
            "key": "Internal Error",
            "message": "No primary index found. Please create a primary index on bucket k0"
        }
    ],
    "state": "errors",
    "metrics": {
        "elapsedTime": "165.025808ms",
        "executionTime": "164.982529ms",
        "resultCount": 1,
        "errorCount": 1
    }
}

cbq> select COUNT(*) from b0;
{
    "signature": {
        "$1": "number"
    },
    "results": [
        {
            "$1": 1
        }
    ],
    "state": "success",
    "metrics": {
        "elapsedTime": "45.460007ms",
        "executionTime": "45.372988ms",
        "resultCount": 1
    }
}

Is Count not expected to work if primary indexes are not defined?


 Comments   
Comment by Gerald Sangudi [ 21/Oct/14 ]
select count(*) from `beer-sample`

should avoid the syntax error (hyphen in bucket name).

select count(*) should not require a primary key; will investigate.
Comment by Ketaki Gangal [ 21/Oct/14 ]
ok. Though now I get this error
cbq> select COUNT(*) from `beer-sample`;
{
    "signature": {
        "$1": "number"
    },
    "results": [
        {
            "$1": 0
        }
    ],
    "errors": [
        {
            "caller": "couchbase:288",
            "code": 5000,
            "key": "Internal Error",
            "message": "No primary index found. Please create a primary index on bucket beer-sample"
        }
    ],
    "state": "errors",
    "metrics": {
        "elapsedTime": "49.775952ms",
        "executionTime": "49.750593ms",
        "resultCount": 1,
        "errorCount": 1
    }
}





[MB-12389] Deletion of a field in a document does not result in secondary index being updated Created: 21/Oct/14  Updated: 21/Oct/14

Status: Open
Project: Couchbase Server
Component/s: secondary-index
Affects Version/s: sherlock
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Prathibha Bisarahalli Assignee: Deepkaran Salooja
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: No

 Description   
When there is a 2i created on a field, and the field is deleted from one or more documents, this mutation is not updated in the index. On querying for those field values, the documents are still returned by the query.

Repro:
1. Create a bucket with documents with field say, "State". Create 2i on State.
2. Delete the "State" field from one of the documents.
3. Query for the State with value that was deleted. The query still returns the document from which State field was removed.

In this case, the indexer just logs the error :

 [ERROR] Flusher::processUpsert Error Generating KeyFrom Mutation: [[]]. Skipped. Error: unexpected end of JSON input

and skips updating the index for the deleted field.

 Comments   
Comment by Deepkaran Salooja [ 21/Oct/14 ]
Collatejson is returning error for nil keys:
[ERROR] Flusher::processUpsert Error Generating KeyFrom Mutation: [[]]. Skipped. Error: unexpected end of JSON input

Will take this issue up when we revisit the key storage format in next sprint.




[MB-12388] Admin UI: Everywhere we should the release version, show the build number in close proximity Created: 20/Oct/14  Updated: 21/Oct/14

Status: Open
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 2.5.1
Fix Version/s: 2.5.1, 3.0.1, sherlock
Security Level: Public

Type: Task Priority: Critical
Reporter: Dave Finlay Assignee: Dave Finlay
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This was cloned from the parent issue: http://www.couchbase.com/issues/browse/MB-12387 to track the needed fixes in ns_server.

--------------------------

The build number is a critical piece of information to understand precisely what version of Couchbase server is running at the customer site.

E.g. For 2.5.1 we have released a maintenance pack. The MP has a different build number than vanilla 2.5.1 but the build number is not prominently displayed in close proximity to the release version, so customers and support who are working with the customer may incorrectly deduce the exact release version and build the customer is on.

To fix this, we propose to append the build number with a dash everywhere the release version shows.
E.g. instead of plain 2.5.1 we should show 2.5.1-1083 (for vanilla 2.5.1) and 2.5.1-1100 (for MP1.)


 




[MB-12387] Everywhere we should the release version, show the build number in close proximity Created: 20/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: None
Affects Version/s: 2.5.1
Fix Version/s: 2.5.1
Security Level: Public

Type: Task Priority: Critical
Reporter: Dave Finlay Assignee: Dave Finlay
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The build number is a critical piece of information to understand precisely what version of Couchbase server is running at the customer site.

E.g. For 2.5.1 we have released a maintenance pack. The MP has a different build number than vanilla 2.5.1 but the build number is not prominently displayed in close proximity to the release version, so customers and support who are working with the customer may incorrectly deduce the exact release version and build the customer is on.

To fix this, we propose to append the build number with a dash everywhere the release version shows.
E.g. instead of plain 2.5.1 we should show 2.5.1-1083 (for vanilla 2.5.1) and 2.5.1-1100 (for MP1.)

This issue is meant to be the uber task to which we'll attach individual issues to track the various places things need to be fixed.

ns_server: http://www.couchbase.com/issues/browse/MB-12388

 




[MB-12386] put 3.0.1 windows latest build in 3.0.1 section Created: 20/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: build
Affects Version/s: 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Thuan Nguyen Assignee: Chris Hillery
Resolution: Unresolved Votes: 0
Labels: windows
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: windows


 Description   
Please add one more url link for windows latest build in 3.0.1 section in
http://latestbuilds.hq.couchbase.com/




[MB-12385] unable to add back the node referenced by hostname Created: 20/Oct/14  Updated: 21/Oct/14  Resolved: 21/Oct/14

Status: Closed
Project: Couchbase Server
Component/s: None
Affects Version/s: 3.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Iryna Mironava Assignee: Iryna Mironava
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 3.0.1-1419

Triage: Untriaged
Operating System: Ubuntu 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump: https://s3.amazonaws.com/bugdb/jira/MB-12385/c488aa56/172.23.107.24-10202014-1059-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-12385/c488aa56/172.23.107.25-10202014-111-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-12385/c488aa56/172.23.107.26-10202014-110-diag.zip
Is this a Regression?: Yes

 Description   
-t hostname.hostnamemgmt_tests.HostnameMgmtTests.test_rename_failover_add_back

3 nodes refenrenced by hostname are in cluster
failover one of them and add back. Error appears
2014-10-20 10:58:49 | ERROR | MainProcess | test_thread | [rest_client.add_back_node] add_back_node ns_1@cherry-s10120.sc.couchbase.com error : {"otpNode":"invalid node name or node can't be used for delta recovery"}

attaching cbcollects

 Comments   
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
We've recently fixed MB-7226 and otpNode parameter is now validated. When previously it would just ignore errors.

So this is likely some previously undiscovered bug in testrunner. Check your code for exactly what is being POST-ed.
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
I'm not seeing anything suspicious in logs. And I'm certain that reAddNode works. I've just re-tested it myself.

So I'm certain it must be something in the test that wasn't previously spotted. Consider logging everything you're doing so that it's seen what exactly fails and why.




[MB-12384] append and prepend commands to support TTL Created: 20/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: memcached
Affects Version/s: 2.5.1
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Patrick Varley Assignee: Trond Norbye
Resolution: Unresolved Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
Currently the memcached binary protocol does not support the TTL field on append and prepend operations.

https://code.google.com/p/memcached/wiki/BinaryProtocolRevamped#Append,_Prepend

It would be useful to update the TTL at the same time as executing the command.




[MB-12383] on windows server 3.0, failures to flush Created: 20/Oct/14  Updated: 20/Oct/14  Resolved: 20/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 3.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Simon Baslé Assignee: Aleksey Kondratenko
Resolution: Duplicate Votes: 0
Labels: windows
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: dev workstation, windows 7-64bits, quad-core, 8GB RAM (including 700MB couchbase quota), Couchbase Server 3.0.1 Enterprise Edition (build-1323-rel)

Attachments: Zip Archive logs_flush_failure.zip    
Issue Links:
Duplicate
duplicates MB-6232 ep-engine needs 1.5 minutes to create... Reopened
Triage: Untriaged
Operating System: Windows 64-bit
Is this a Regression?: Unknown

 Description   
When trying to run tests with the Java SDK, I have some flush operations. The server returns {error,{flush_wait_failed,['ns_1@127.0.0.1'],[]}} ...

There is only the default bucket, which is empty and is assigned 256MB of the cluster's RAM quota of 700MB (this is a dev workstation and I have lots of ram consumed by other tools, hence the low quotas).

When trying to flush from webconsole, I also often get an error : "Save request failed because of timeout."

Logs from a clean start of the server with only previous two issues reproducted will be attached.

 Comments   
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
This is likely our old friend MB-6232.

Workaround should be same as on GNU/Linux.

Windows equivalent can be reached via drive settings. As can be seen on this screen-shot:

http://i.imgur.com/wHAx3BF.png

You should (at least just for test) check "turn off Window write-cache buffer flushing".

I'm not sure if it's going to be enough or not. Please try.
Comment by Simon Baslé [ 20/Oct/14 ]
seems like it works with the buffer turned off...
not really confortable doing it on my desktop (no battery power to prevent power outage failures, etc...) but I can activate it during tests and look into alternatives on where to run my test server for the future
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
Making it duplicate of MB-6232 then.

You can use VM where it's less risky to disable cache flushing. Also lowering number of vbuckets is known to help quite a lot. MB-6232 should have all instructions for that.




[MB-12382] Offline cluster upgrade failed 2.0.1-170-rel -> 3.0.1-1422-rel Created: 20/Oct/14  Updated: 20/Oct/14  Resolved: 20/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Sangharsh Agarwal Assignee: Sangharsh Agarwal
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Os: CentOS

Triage: Untriaged
Operating System: Centos 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump: 10.1.3.93 : https://s3.amazonaws.com/bugdb/jira/MB-12382/91882dce/10.1.3.93-diag.txt.gz
10.1.3.93 : https://s3.amazonaws.com/bugdb/jira/MB-12382/daf32985/10.1.3.93-10182014-1830-diag.zip
10.1.3.94 : https://s3.amazonaws.com/bugdb/jira/MB-12382/23591591/10.1.3.94-diag.txt.gz
10.1.3.94 : https://s3.amazonaws.com/bugdb/jira/MB-12382/753549fa/10.1.3.94-10182014-1831-diag.zip
Is this a Regression?: Yes

 Description   
Connection to Memcached is dropped after offline upgrade.

Offline upgrade 10.1.3.93 and 10.1.3.94 is failed.

[ns_server.babysitter.log]
[ns_server:info,2014-10-18T18:14:04.214,babysitter_of_ns_1@127.0.0.1:<0.78.0>:ns_port_server:log:181]memcached<0.78.0>: Invalid value specified for verbosity: []
memcached<0.78.0>:
memcached<0.78.0>: Terminating

[ns_server:info,2014-10-18T18:14:04.214,babysitter_of_ns_1@127.0.0.1:<0.77.0>:supervisor_cushion:handle_info:58]Cushion managed supervisor for memcached failed: {abnormal,1}
[ns_server:info,2014-10-18T18:14:04.214,babysitter_of_ns_1@127.0.0.1:<0.77.0>:supervisor_cushion:die_slowly:80]Service memcached exited on node 'babysitter_of_ns_1@127.0.0.1' in 0.08s

[error_logger:error,2014-10-18T18:14:04.215,babysitter_of_ns_1@127.0.0.1:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]** Generic server <0.78.0> terminating
** Last message in was {#Port<0.3398>,{exit_status,1}}
** When Server state == {state,#Port<0.3398>,memcached,
                               {["Terminating",[],
                                 "Invalid value specified for verbosity: []",
                                 empty],
                                [empty]},
                               {ok,{1413681244413,#Ref<0.0.0.280>}},
                               ["Terminating",[],
                                "Invalid value specified for verbosity: []"],
                               0,true}
** Reason for termination ==
** {abnormal,1}

[error_logger:error,2014-10-18T18:14:04.221,babysitter_of_ns_1@127.0.0.1:error_logger<0.6.0>:ale_error_logger_handler:do_log:203]
=========================CRASH REPORT=========================
  crasher:
    initial call: ns_port_server:init/1
    pid: <0.78.0>
    registered_name: []
    exception exit: {abnormal,1}
      in function gen_server:terminate/6 (gen_server.erl, line 744)
    ancestors: [<0.77.0>,ns_child_ports_sup,ns_babysitter_sup,<0.56.0>]
    messages: [{'EXIT',#Port<0.3398>,normal}]
    links: [<0.77.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 1025
  neighbours:


[Server Error]
[ns_server:info,2014-10-18T18:13:54.416,ns_1@10.1.3.93:ns_config<0.262.0>:ns_config_default:upgrade_config_from_2_0_to_2_2_0:623]Upgrading config from 2.0 to 2.2.0
[ns_server:info,2014-10-18T18:13:54.420,ns_1@10.1.3.93:ns_config<0.262.0>:ns_config_default:upgrade_config_from_2_2_0_to_2_3_0:630]Upgrading config from 2.2.0 to 2.3.0
[ns_server:info,2014-10-18T18:13:54.421,ns_1@10.1.3.93:ns_config<0.262.0>:ns_config_default:upgrade_config_from_2_3_0_to_3_0:644]Upgrading config from 2.3.0 to 3.0
[ns_server:info,2014-10-18T18:13:54.426,ns_1@10.1.3.93:ns_config<0.262.0>:ns_config_default:upgrade_config_from_3_0_to_3_0_1:680]Upgrading config from 3.0 to 3.0.1
[ns_server:info,2014-10-18T18:13:54.552,ns_1@10.1.3.93:ns_server_sup<0.271.0>:dir_size:start_link:49]Starting quick version of dir_size with program name: i386-linux-godu
[ns_server:warn,2014-10-18T18:13:54.596,ns_1@10.1.3.93:ns_config_isasl_sync<0.279.0>:ns_memcached:connect:1260]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[user:info,2014-10-18T18:13:55.602,ns_1@10.1.3.93:ns_cookie_manager<0.257.0>:ns_cookie_manager:do_cookie_sync:130]Node 'ns_1@10.1.3.93' synchronized otp cookie pcpsknplpzdwdovs from cluster
[ns_server:info,2014-10-18T18:13:55.632,ns_1@10.1.3.93:ns_config_rep<0.296.0>:ns_config_rep:do_pull:343]Pulling config from: 'ns_1@10.1.3.94'

[user:info,2014-10-18T18:14:04.109,ns_1@10.1.3.93:ns_server_sup<0.271.0>:menelaus_sup:start_link:44]Couchbase Server has started on web port 8091 on node 'ns_1@10.1.3.93'. Version: "3.0.1-1422-rel-enterprise".
[ns_server:info,2014-10-18T18:14:04.135,ns_1@10.1.3.93:<0.433.0>:ns_memcached_log_rotator:init:28]Starting log rotator on "/opt/couchbase/var/lib/couchbase/logs"/"memcached.log"* with an initial period of 39003ms
[user:info,2014-10-18T18:14:04.214,ns_1@10.1.3.93:<0.278.0>:ns_log:crash_consumption_loop:70]Port server memcached on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: Invalid value specified for verbosity: []

Terminating
[ns_server:warn,2014-10-18T18:14:04.293,ns_1@10.1.3.93:<0.488.0>:ns_memcached:connect:1260]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.
[ns_server:warn,2014-10-18T18:14:04.305,ns_1@10.1.3.93:<0.500.0>:ns_memcached:connect:1260]Unable to connect: {error,{badmatch,{error,econnrefused}}}, retrying.



[Test]
./testrunner -i INI_FILE.ini get-cbcollect-info=True,get-logs=False,stop-on-failure=False,get-coredumps=True,upgrade_version=${version_number},GROUP=ALL,upgrade_version=3.0.1-1422-rel -t xdcr.upgradeXDCR.UpgradeTests.offline_cluster_upgrade,initial_version=2.0.1-170-rel,sdata=False,bucket_topology=default:1>2;bucket0:1><2,upgrade_nodes=src,post-upgrade-actions=src-rebalanceout;dest-rebalanceout;dest-create_index


[Jenkins]
http://qa.hq.northscale.net/view/3.0%20By%20COMPONENT/job/centos_x64--01_03--XDCR_upgrade-P1/57/consoleFull


 Comments   
Comment by Sangharsh Agarwal [ 20/Oct/14 ]
All Offline upgrades from : 2.1.1-766-rel, 2.2.0-837-rel, 2.5.0-1059-rel, 2.5.1-1083-rel are failed with same issue.
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
Fixed be: http://review.couchbase.org/42299




[MB-12381] Memcached error #1 'Not found': Not found for vbucket Created: 20/Oct/14  Updated: 20/Oct/14  Resolved: 20/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Sangharsh Agarwal Assignee: Sangharsh Agarwal
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Build 3.0.1-1422

Triage: Untriaged
Operating System: Centos 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump: [Source]
10.1.3.93 : https://s3.amazonaws.com/bugdb/jira/MB-12381/964b1150/10.1.3.93-10182014-317-diag.zip
10.1.3.93 : https://s3.amazonaws.com/bugdb/jira/MB-12381/db68adac/10.1.3.93-diag.txt.gz
10.1.3.94 : https://s3.amazonaws.com/bugdb/jira/MB-12381/3cd88c1e/10.1.3.94-diag.txt.gz
10.1.3.94 : https://s3.amazonaws.com/bugdb/jira/MB-12381/fdf72095/10.1.3.94-10182014-321-diag.zip
10.1.3.95 : https://s3.amazonaws.com/bugdb/jira/MB-12381/547b84ae/10.1.3.95-diag.txt.gz
10.1.3.95 : https://s3.amazonaws.com/bugdb/jira/MB-12381/f41c4d6b/10.1.3.95-10182014-325-diag.zip

[Destination]
10.1.3.96 : https://s3.amazonaws.com/bugdb/jira/MB-12381/72200414/10.1.3.96-diag.txt.gz
10.1.3.96 : https://s3.amazonaws.com/bugdb/jira/MB-12381/b04de4c6/10.1.3.96-10182014-329-diag.zip
10.1.3.97 : https://s3.amazonaws.com/bugdb/jira/MB-12381/d280d046/10.1.3.97-diag.txt.gz
10.1.3.97 : https://s3.amazonaws.com/bugdb/jira/MB-12381/fb1b989e/10.1.3.97-10182014-333-diag.zip
10.1.3.99 : https://s3.amazonaws.com/bugdb/jira/MB-12381/daa3faca/10.1.3.99-10182014-342-diag.zip
10.1.3.99 : https://s3.amazonaws.com/bugdb/jira/MB-12381/ed19b721/10.1.3.99-diag.txt.gz
10.1.2.12 : https://s3.amazonaws.com/bugdb/jira/MB-12381/1d65e66c/10.1.2.12-10182014-338-diag.zip
10.1.2.12 : https://s3.amazonaws.com/bugdb/jira/MB-12381/c56b0cc3/10.1.2.12-diag.txt.gz
Is this a Regression?: Yes

 Description   
[Test Error]
[2014-10-18 03:16:24,101] - [task:640] ERROR - Memcached error #1 'Not found': Not found for vbucket :912 to mc 10.1.3.95:11210, key: loadOne15901 delete operation.


[Test Steps]
1. Setup -> Source : 3Nodes, Destination: 4 Nodes.
2. Buckets-> default, sasl_bucket_1
3. XDCR -> Mode: Xmem, Bi-directional.
4. Load 20 Items on each node on Source (keys: loadOne*) and Destination (loadTwo*) on each bucket.
5. Disable compaction on "default" bucket.

[2014-10-18 03:14:44,352] - [rest_client:1940] INFO - 'default' bucket's settings will be changed with parameters: autoCompactionDefined=true&parallelDBAndViewCompaction=false&ramQuotaMB=1053


5. Update/Delete items on Source cluster and Create deleted items. -> Received MemcachedError on Source during Delete Operation itself.


[2014-10-18 03:16:24,101] - [task:640] ERROR - Memcached error #1 'Not found': Not found for vbucket :912 to mc 10.1.3.95:11210, key: loadOne15901 delete operation.

[2014-10-18 03:16:31,062] - [task:640] ERROR - Memcached error #1 'Not found': Not found for vbucket :144 to mc 10.1.3.93:11210, key: loadOne15227 delete operation.


[Notes]
- Test is passed with XDCR + SSL. Though It is not sure if it is related to XDCR directly.

 Comments   
Comment by Sangharsh Agarwal [ 20/Oct/14 ]
Ran this test separately, issue not reproduced.




[MB-12380] 27% regression in 80th percentile query latency with 1 bucket, 20M docs and stale=false parameter Created: 19/Oct/14  Updated: 19/Oct/14

Status: Open
Project: Couchbase Server
Component/s: view-engine
Affects Version/s: 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Venu Uppalapati Assignee: Sarath Lakshman
Resolution: Unresolved Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Windows 64-bit
Is this a Regression?: Yes

 Description   
Test description:
80th percentile query latency (ms), 1 bucket x 20M x 2KB, non-DGM, 4 x 1 views, 500 mutations/sec/node, 400 queries/sec, stale=false

Observation:
80th percentile latency increase from 5457 ms to 6954 ms

links:
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_301-1330_686_access
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_251-1083_6c5_access

logs:
http://ci.sc.couchbase.com/job/zeus-64/1222/artifact/172.23.96.25.zip
http://ci.sc.couchbase.com/job/zeus-64/1222/artifact/172.23.96.26.zip
http://ci.sc.couchbase.com/job/zeus-64/1222/artifact/172.23.96.27.zip
http://ci.sc.couchbase.com/job/zeus-64/1222/artifact/172.23.96.28.zip
http://ci.sc.couchbase.com/job/zeus-64/1222/artifact/web_log_172.23.96.25.json




[MB-12379] Default and Maximum lock time described incorrectly in Developer Guide Created: 19/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: documentation
Affects Version/s: 2.5.1, 3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Chris Malarky Assignee: Amy Kurtzman
Resolution: Unresolved Votes: 0
Labels: documentation
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to NCBC-691 Maximum and Default Locking times are... Open
relates to PYCBC-260 Default and Maximum lock described in... Open
relates to RCBC-186 Description of locking potentially mi... Open
relates to JCBC-596 Default and Maximum Lock Time needs c... New
Triage: Untriaged
Is this a Regression?: Unknown

 Description   
The maximum lock in 2.5.1 and 3.0.0 is 30 secs, the default is 15 seconds. Any attempt to set a lock higher than the maximum results in the default being used. The Developer Guide needs updating to match this behaviour.




[MB-12378] [ui] inclusive_end is off by default (was: Change in behaviour between 2.5 and 3.0 for inclusive_end default) Created: 18/Oct/14  Updated: 20/Oct/14  Resolved: 20/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: UI
Affects Version/s: 3.0
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Minor
Reporter: marcusr Assignee: Aleksey Kondratenko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Ubuntu 64-bit
Is this a Regression?: Unknown

 Description   
When using the web console to run queries against views, on 2.5 the default query string is:

?stale=false&connection_timeout=60000&limit=10&skip=0

even when not set to true in the filter results dialog.

On version 3.0 inclusive_end is set to false

?stale=false&inclusive_end=false&connection_timeout=60000&limit=10&skip=0

which can catch you out if you are running a query that returns a single document. This is not necessarily a regression but is a change which caught me out.



 Comments   
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
it's back to defaulting to true now. Will be part of 3.0.1

Fixed by this commit: http://review.couchbase.org/#/c/42301/




[MB-12377] SAMPLE clause with N1QL for queries to target only smaller subset of data for a quicker but inprecise answer to query Created: 18/Oct/14  Updated: 18/Oct/14

Status: Open
Project: Couchbase Server
Component/s: None
Affects Version/s: cbq-DP3
Fix Version/s: sherlock
Security Level: Public

Type: Improvement Priority: Major
Reporter: Cihan Biyikoglu Assignee: Gerald Sangudi
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
SAMPLE clause can be useful to limit the index or bucket scan we need to do to answer a query. we could also specify a special flag to only query data already resident in memory.




[MB-12376] Should add http_access.log to the list of log files in documentation Created: 17/Oct/14  Updated: 18/Oct/14

Status: Open
Project: Couchbase Server
Component/s: documentation
Affects Version/s: 3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Don Pinto Assignee: Ruth Harris
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
The admin access log feature records server requests (including admin logins) coming through the REST or Couchbase admin console.

See MB-10020




[MB-12375] 315% regression in 80th percentile query latency with 1 Bucket, 20M docs, non-DGM scenario with 4x1 views Created: 17/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: view-engine
Affects Version/s: 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Venu Uppalapati Assignee: Sarath Lakshman
Resolution: Unresolved Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Windows 64-bit
Is this a Regression?: Yes

 Description   
Test description:
80th percentile query latency (ms), 1 bucket x 20M x 2KB, non-DGM, 4 x 1 views, 500 mutations/sec/node, 400 queries/sec

Observation:
80th percentile latency increased from 13ms to 54ms

links:
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_301-1330_2c1_access
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_251-1083_eba_access

logs:
http://ci.sc.couchbase.com/job/zeus-64/1221/artifact/172.23.96.25.zip
http://ci.sc.couchbase.com/job/zeus-64/1221/artifact/172.23.96.26.zip
http://ci.sc.couchbase.com/job/zeus-64/1221/artifact/172.23.96.27.zip
http://ci.sc.couchbase.com/job/zeus-64/1221/artifact/172.23.96.28.zip
http://ci.sc.couchbase.com/job/zeus-64/1221/artifact/web_log_172.23.96.25.json


 Comments   
Comment by Volker Mische [ 20/Oct/14 ]
I've added the label "Windows" as it really makes a difference. As the performance looks right on Linux.
Comment by Volker Mische [ 20/Oct/14 ]
Removed the "windows" label again as I just found out that there's also a way to set the "Operating System".




[MB-12374] 115% regression in 80th percentile query latency with 1 bucket, 100M documents and 4x1 views. Created: 17/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: view-engine
Affects Version/s: 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Venu Uppalapati Assignee: Sarath Lakshman
Resolution: Unresolved Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Windows 64-bit
Is this a Regression?: Yes

 Description   
Test description:
80th percentile query latency (ms), 1 bucket x 100M x 2KB, DGM, 4 x 1 views, 500 mutations/sec/node, 400 queries/sec

Observation:
80th percentile latency increased from 13ms to 28ms.

links:
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_301-1330_df0_access
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_251-1083_c2e_access

logs:
http://ci.sc.couchbase.com/job/zeus-64/1223/artifact/172.23.96.25.zip
http://ci.sc.couchbase.com/job/zeus-64/1223/artifact/172.23.96.26.zip
http://ci.sc.couchbase.com/job/zeus-64/1223/artifact/172.23.96.27.zip
http://ci.sc.couchbase.com/job/zeus-64/1223/artifact/172.23.96.28.zip
http://ci.sc.couchbase.com/job/zeus-64/1223/artifact/web_log_172.23.96.25.json



 Comments   
Comment by Volker Mische [ 20/Oct/14 ]
I've added the label "Windows" as it really makes a difference. As the performance looks right on Linux.
Comment by Volker Mische [ 20/Oct/14 ]
Removed the "windows" label again as I just found out that there's also a way to set the "Operating System"




[MB-12373] Tracking Item: Sync GW moving to DCP Created: 17/Oct/14  Updated: 17/Oct/14

Status: Open
Project: Couchbase Server
Component/s: DCP
Affects Version/s: 3.0
Fix Version/s: sherlock
Security Level: Public

Type: Improvement Priority: Major
Reporter: Cihan Biyikoglu Assignee: Adam Fraser
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
tracking the work in syncGW - https://github.com/couchbase/sync_gateway/issues/409

 Comments   
Comment by Cihan Biyikoglu [ 17/Oct/14 ]
created for tracking the item on syncGW and any asks that may come back.




[MB-12372] Fix the typo on the first Title Created: 17/Oct/14  Updated: 17/Oct/14  Resolved: 17/Oct/14

Status: Closed
Project: Couchbase Server
Component/s: documentation
Affects Version/s: 3.0
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Anil Kumar Assignee: Ruth Harris
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
http://docs.couchbase.com/developer/dev-guide-3.0/cb-vs-rdbms.html - "Comparing Couchbase and traditional RDMS"

Fix RDMS => RDBMS

 Comments   
Comment by Ruth Harris [ 17/Oct/14 ]
Fixed and pushed to Github.




[MB-12371] Incremental CBBackup shouldn't consume additional space with no delta Created: 17/Oct/14  Updated: 17/Oct/14  Resolved: 17/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: tools
Affects Version/s: 3.0, 3.0-Beta
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Anil Kumar Assignee: Bin Cui
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Debian 6
CentOS 6.5
Couchbase Server 3.0.0

Issue Links:
Relates to
Triage: Triaged
Is this a Regression?: Unknown

 Description   
Customer is reporting that cbbackup unexpectedly consumes additional storage in the following scenario:

1. Ensure that no operations can be performed against the Couchbase Server cluster
2. Perform a full backup of all buckets/nodes with `-m full`
3. Note disk usage of backup target directory
4. Perform a cumulative backup of all buckets/nodes with `-m accu`
5. Note disk usage of backup target has increased approximately 50%

I was able to setup a basic test on a cluster doing no work after an initial cbworkloadgen of 50k items and observed the same behavior as can be seen here:

[root@brian4 opt]# du -sh couchbase_backups/
4.0K couchbase_backups/

[root@brian4 opt]# /opt/couchbase/bin/cbbackup http://brian4.local:8091 /opt/couchbase_backups/ -m full -u Administrator -p couchbase
  [####################] 100.0% (50000/estimated 50000 msgs)
bucket: makura, msgs transferred...
       : total | last | per sec
 byte : 15773320 | 15773320 | 805225.6
done

[root@brian4 opt]# du -sh couchbase_backups/
26M couchbase_backups/

[root@brian4 opt]# /opt/couchbase/bin/cbbackup http://brian4.local:8091 /opt/couchbase_backups/ -m accu -u Administrator -p couchbase
  [############# ] 66.4% (33224/estimated 50000 msgs)
bucket: makura, msgs transferred...
       : total | last | per sec
 byte : 10485649 | 10485649 | 703111.0
done

[root@brian4 opt]# du -sh couchbase_backups/
42M couchbase_backups/

Why is there additional storage being consumed if nothing has changed on the cluster between backup invocations?

Thanks!


 Comments   
Comment by Bin Cui [ 17/Oct/14 ]
http://review.couchbase.org/#/c/42225/




[MB-12370] No reloc.sh in opt/couchbase/bin/install/reloc.sh Created: 17/Oct/14  Updated: 17/Oct/14

Status: Open
Project: Couchbase Server
Component/s: None
Affects Version/s: 3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Juan Luis Pérez Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Ubuntu 64-bit
Is this a Regression?: No

 Description   
I downloaded the couchbase from the downloads page. Couchbase 3.0.0 for ubuntu 64bits and there isn't the reloc.sh file to install as non-root




[MB-12367] cbq-engine crashes after first query run after building cbq-engine Created: 16/Oct/14  Updated: 16/Oct/14

Status: Open
Project: Couchbase Server
Component/s: query
Affects Version/s: cbq-DP4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Iryna Mironava Assignee: Gerald Sangudi
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
last commit: 4eb2b16415cfd771190a9bbca3a211a31220a603
1) build cbq-engine
2) run cbq-engine

it crashes:
_time="2014-10-16T09:34:38-07:00" _level="INFO" _msg="New site created with url http://10.1.3.176:8091"
_time="2014-10-16T09:34:38-07:00" _level="INFO" _msg="cbq-engine started" version="0.7.0" datastore="http://10.1.3.176:8091"
_time="2014-10-16T09:35:39-07:00" _level="INFO" _msg="Refreshing pool default"
2014/10/16 09:35:50 http: multiple response.WriteHeader calls
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x5dff9c]

goroutine 17 [running]:
net/http.(*switchWriter).Write(0xc20018ec60, 0xc20066b000, 0xc, 0x800, 0x3300000000, ...)
/usr/local/go/src/pkg/net/http/chunked.go:0 +0x5c
bufio.(*Writer).Flush(0xc200197640, 0xc2001be7e0, 0x2ac0d239cc98)
/usr/local/go/src/pkg/bufio/bufio.go:465 +0xb9
net/http.(*response).Flush(0xc2001be7e0)
/usr/local/go/src/pkg/net/http/server.go:952 +0x4a
github.com/couchbaselabs/query/server/http.(*httpRequest).Flush(0xc2001ca0f0)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/http/http_response.go:241 +0x67
github.com/couchbaselabs/query/server/http.(*httpRequest).writeString(0xc2001ca0f0, 0xc2006d6530, 0xc, 0xa)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/http/http_response.go:135 +0x8d
github.com/couchbaselabs/query/server/http.(*httpRequest).Fail(0xc2001ca0f0, 0xc200631640, 0xc2006da550)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/http/http_response.go:33 +0xd2
github.com/couchbaselabs/query/server.(*Server).serviceRequest(0xc2001f7460, 0xc200174e60, 0xc2001ca0f0)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:107 +0x24d
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:93 +0x9a
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 1 [IO wait]:
net.runtime_pollWait(0x2ac0d238ddc0, 0x72, 0x0)
/usr/local/go/src/pkg/runtime/znetpoll_linux_amd64.c:118 +0x82
net.(*pollDesc).WaitRead(0xc2001bf980, 0xb, 0xc2001c1990)
/usr/local/go/src/pkg/net/fd_poll_runtime.go:75 +0x31
net.(*netFD).accept(0xc2001bf900, 0xb0da38, 0x0, 0xc2001c1990, 0xb, ...)
/usr/local/go/src/pkg/net/fd_unix.go:385 +0x2c1
net.(*TCPListener).AcceptTCP(0xc20022b1a8, 0x5d1f46, 0x2ac0d0e7ac90, 0x5d1f46)
/usr/local/go/src/pkg/net/tcpsock_posix.go:229 +0x45
net.(*TCPListener).Accept(0xc20022b1a8, 0xc2001a81e0, 0xc200665f98, 0xc2001bfd80, 0x0, ...)
/usr/local/go/src/pkg/net/tcpsock_posix.go:239 +0x25
net/http.(*Server).Serve(0xc2001a8bb0, 0xc2001ff700, 0xc20022b1a8, 0x0, 0x0, ...)
/usr/local/go/src/pkg/net/http/server.go:1542 +0x85
net/http.(*Server).ListenAndServe(0xc2001a8bb0, 0xc2001ecb40, 0xc2001a8ba0)
/usr/local/go/src/pkg/net/http/server.go:1532 +0x9e
github.com/couchbaselabs/query/server/http.(*HttpEndpoint).ListenAndServe(0xc2001a8ba0, 0x1, 0xa0d820)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/http/http_endpoint.go:36 +0x2a
main.main()
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/main/main.go:96 +0x739

goroutine 2 [syscall]:

goroutine 4 [IO wait]:
net.runtime_pollWait(0x2ac0d238df00, 0x72, 0x0)
/usr/local/go/src/pkg/runtime/znetpoll_linux_amd64.c:118 +0x82
net.(*pollDesc).WaitRead(0xc2001bf080, 0xb, 0xc2001c1990)
/usr/local/go/src/pkg/net/fd_poll_runtime.go:75 +0x31
net.(*netFD).accept(0xc2001bf000, 0xb0da58, 0x0, 0xc2001c1990, 0xb, ...)
/usr/local/go/src/pkg/net/fd_unix.go:385 +0x2c1
net.(*UnixListener).AcceptUnix(0xc200196700, 0x0, 0xc2001a5898, 0x77c32a)
/usr/local/go/src/pkg/net/unixsock_posix.go:286 +0x49
net.(*UnixListener).Accept(0xc200196700, 0x0, 0x0, 0x0, 0x0, ...)
/usr/local/go/src/pkg/net/unixsock_posix.go:297 +0x49
github.com/couchbaselabs/retriever/logger.doHandleConnections(0xc2001bd780, 0xa3d810, 0xa)
/root/tuq/gocode/src/github.com/couchbaselabs/retriever/logger/accept.go:50 +0x32f
github.com/couchbaselabs/retriever/logger.handleConnections(0xc2001bd780, 0xa3d810, 0xa)
/root/tuq/gocode/src/github.com/couchbaselabs/retriever/logger/accept.go:24 +0x39
created by github.com/couchbaselabs/retriever/logger.NewLogger
/root/tuq/gocode/src/github.com/couchbaselabs/retriever/logger/logger.go:136 +0x26f

goroutine 279 [semacquire]:
sync.runtime_Semacquire(0xc2006b6020)
/usr/local/go/src/pkg/runtime/zsema_linux_amd64.c:165 +0x2e
sync.(*Cond).Wait(0xc2001bd9f0)
/usr/local/go/src/pkg/sync/cond.go:74 +0x95
io.(*pipe).read(0xc2001bd9c0, 0xc200669000, 0x1000, 0x1000, 0x0, ...)
/usr/local/go/src/pkg/io/pipe.go:52 +0x1f2
io.(*PipeReader).Read(0xc2006b6008, 0xc200669000, 0x1000, 0x1000, 0xc4, ...)
/usr/local/go/src/pkg/io/pipe.go:130 +0x5d
net/http.(*liveSwitchReader).Read(0xc2001bfda8, 0xc200669000, 0x1000, 0x1000, 0xc4, ...)
/usr/local/go/src/pkg/net/http/server.go:205 +0x91
io.(*LimitedReader).Read(0xc200668e20, 0xc200669000, 0x1000, 0x1000, 0xc4, ...)
/usr/local/go/src/pkg/io/io.go:394 +0xc0
net/http.(*switchReader).Read(0xc20018eb90, 0xc200669000, 0x1000, 0x1000, 0x0, ...)
/usr/local/go/src/pkg/net/http/chunked.go:0 +0x62
bufio.(*Reader).fill(0xc2001a8cc0)
/usr/local/go/src/pkg/bufio/bufio.go:79 +0x10c
bufio.(*Reader).ReadSlice(0xc2001a8cc0, 0x40be0a, 0x0, 0x0, 0x0, ...)
/usr/local/go/src/pkg/bufio/bufio.go:262 +0x202
bufio.(*Reader).ReadLine(0xc2001a8cc0, 0x0, 0x0, 0x0, 0x2ac0d0e86c00, ...)
/usr/local/go/src/pkg/bufio/bufio.go:293 +0x61
net/textproto.(*Reader).readLineSlice(0xc2001ecc30, 0xe005c872b, 0x42016f, 0xc2006475b0, 0x9b27a0, ...)
/usr/local/go/src/pkg/net/textproto/reader.go:55 +0x51
net/textproto.(*Reader).ReadLine(0xc2001ecc30, 0xc2006475b0, 0x1000, 0x5, 0x0, ...)
/usr/local/go/src/pkg/net/textproto/reader.go:36 +0x25
net/http.ReadRequest(0xc2001a8cc0, 0xc2006475b0, 0x0, 0x0)
/usr/local/go/src/pkg/net/http/request.go:510 +0x86
net/http.(*conn).readRequest(0xc2001bfd80, 0x0, 0x0, 0x0)
/usr/local/go/src/pkg/net/http/server.go:547 +0x1bc
net/http.(*conn).serve(0xc2001bfd80)
/usr/local/go/src/pkg/net/http/server.go:1052 +0x398
created by net/http.(*Server).Serve
/usr/local/go/src/pkg/net/http/server.go:1564 +0x266

goroutine 14 [chan receive]:
github.com/couchbaselabs/query/datastore/couchbase.keepPoolFresh(0xc2001a7b00)
/root/tuq/gocode/src/github.com/couchbaselabs/query/datastore/couchbase/couchbase.go:226 +0x5f
created by github.com/couchbaselabs/query/datastore/couchbase.loadNamespace
/root/tuq/gocode/src/github.com/couchbaselabs/query/datastore/couchbase/couchbase.go:131 +0x5de

goroutine 281 [chan receive]:
github.com/couchbaselabs/query/server/http.func·001()
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/http/http_request.go:93 +0x43
created by github.com/couchbaselabs/query/server/http.newHttpRequest
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/http/http_request.go:95 +0x6e8

goroutine 18 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 19 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 20 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 21 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 22 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 23 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 24 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 25 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 26 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 27 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 28 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 29 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 30 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 31 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 32 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 33 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 34 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 35 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 36 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 37 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 38 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 39 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 40 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 41 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 42 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 43 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 44 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 45 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 46 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 47 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 48 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 49 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 50 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 51 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 52 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 53 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 54 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 55 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 56 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 57 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 58 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 59 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 60 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 61 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 62 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 63 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 64 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 65 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 66 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 67 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 68 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 69 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 70 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 71 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 72 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 73 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 74 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 75 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 76 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 77 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 78 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 79 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 80 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 81 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 82 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 83 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 84 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 85 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 86 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 87 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 88 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 89 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 90 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 91 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 92 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 93 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 94 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 95 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 96 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 97 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 98 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 99 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 100 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 101 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 102 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 103 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 104 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 105 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 106 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 107 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 108 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 109 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 110 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 111 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 112 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 113 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 114 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 115 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 116 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 117 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 118 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 119 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 120 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 121 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 122 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 123 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 124 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 125 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 126 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 127 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 128 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 129 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 130 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 131 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 132 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 133 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 134 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 135 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 136 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 137 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 138 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 139 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 140 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 141 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 142 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 143 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 144 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 145 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 146 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 147 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 148 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 149 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 150 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 151 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 152 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 153 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 154 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 155 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 156 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 157 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 158 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 159 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 160 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 161 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 162 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 163 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 164 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 165 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 166 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 167 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 168 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 169 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 170 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 171 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 172 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 173 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 174 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 175 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 176 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 177 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 178 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 179 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 180 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 181 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 182 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 183 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 184 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 185 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 186 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 187 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 188 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 189 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 190 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 191 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 192 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 193 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 194 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 195 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 196 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 197 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 198 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 199 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 200 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 201 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 202 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 203 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 204 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 205 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 206 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 207 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 208 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 209 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 210 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 211 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 212 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 213 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 214 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 215 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 216 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 217 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 218 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 219 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 220 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 221 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 222 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 223 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 224 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 225 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 226 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 227 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 228 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 229 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 230 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 231 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 232 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 233 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 234 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 235 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 236 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 237 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 238 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 239 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 240 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 241 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 242 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 243 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 244 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 245 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 246 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 247 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 248 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 249 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 250 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 251 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 252 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 253 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 254 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 255 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 256 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 257 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 258 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 259 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 260 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 261 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 262 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 263 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 264 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 265 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 266 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 267 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 268 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 269 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 270 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 271 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 272 [chan receive]:
github.com/couchbaselabs/query/server.(*Server).doServe(0xc2001f7460)
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:92 +0x3c
created by github.com/couchbaselabs/query/server.func·002
/root/tuq/gocode/src/github.com/couchbaselabs/query/server/server.go:86 +0x58

goroutine 280 [runnable]:
net.runtime_pollWait(0x2ac0d238de60, 0x72, 0x0)
/usr/local/go/src/pkg/runtime/znetpoll_linux_amd64.c:118 +0x82
net.(*pollDesc).WaitRead(0xc2001bfd70, 0xb, 0xc2001c1990)
/usr/local/go/src/pkg/net/fd_poll_runtime.go:75 +0x31
net.(*netFD).Read(0xc2001bfcf0, 0xc2006b8000, 0x8000, 0x8000, 0x0, ...)
/usr/local/go/src/pkg/net/fd_unix.go:195 +0x2b3
net.(*conn).Read(0xc200665f98, 0xc2006b8000, 0x8000, 0x8000, 0x8000, ...)
/usr/local/go/src/pkg/net/net.go:123 +0xc3
io.Copy(0xc2006b7000, 0xc2006b6010, 0xc2001c1b10, 0xc200665f98, 0x0, ...)
/usr/local/go/src/pkg/io/io.go:348 +0x1c6
net/http.func·004()
/usr/local/go/src/pkg/net/http/server.go:162 +0x66
created by net/http.(*conn).closeNotify
/usr/local/go/src/pkg/net/http/server.go:168 +0x1c6




[MB-12366] unable to build cbq-engine (undefined: couchbase.PoolServices) Created: 16/Oct/14  Updated: 16/Oct/14  Resolved: 16/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: query
Affects Version/s: cbq-DP4
Fix Version/s: cbq-DP4
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Iryna Mironava Assignee: Gerald Sangudi
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
[root@grape-001 query]# ./build.sh
cd parser/n1ql
nex...
goyacc...
go build...
cd server/main
go build -o cbq-engine...
# github.com/couchbaselabs/query/clustering/couchbase
../../clustering/couchbase/clustering_cb.go:93: undefined: couchbase.PoolServices
../../clustering/couchbase/clustering_cb.go:94: c.cbConn.GetPoolServices undefined (type *couchbase.Client has no field or method GetPoolServices)
cd shell
go build -o cbq...
cd tutorial
go build


 Comments   
Comment by Gerald Sangudi [ 16/Oct/14 ]
Fixed. Please do:

git pull
build.sh

Thanks.




[MB-12365] ep_testsuite.cc:6288 Test failed: `Expected reading the value back in to update the read counter' Created: 16/Oct/14  Updated: 21/Oct/14  Resolved: 21/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: couchbase-bucket
Affects Version/s: sherlock
Fix Version/s: sherlock
Security Level: Public

Type: Bug Priority: Major
Reporter: Dave Rigby Assignee: Mike Wiederhold
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: MacOSX 64-bit
Is this a Regression?: Unknown

 Description   
Running the ep_testsuite from top-of-trunk (ep_engine: GIT:2473b19) I see the following error *after* reverting the recent-ish change to add memory hooks to mock_server[1]:

Running [0078/0293]: io stats (couchstore).../Users/dave/repos/couchbase/server/source/ep-engine/tests/ep_testsuite.cc:6288 Test failed: `Expected reading the value back in to update the read counter' (get_int_stat(h, h1, "ro_0:io_num_read", "kvstore") == 1 && get_int_stat(h, h1, "ro_0:io_read_bytes", "kvstore") == 4)

To reproduce:

1. Apply http://review.couchbase.org/#/c/41647/ (to fix the mem_used errors).
2. cd build/ep-engine && make test

(or specific failing test: ./engine_testapp -E ep.so -T ep_testsuite.so -e "flushall_enabled=true;ht_size=13;ht_locks=7")

As an aside, do we not have a CI build testing this?

[1]: http://review.couchbase.org/#/c/41647/

 Comments   
Comment by Chiyoung Seo [ 16/Oct/14 ]
Mike,

Seems like a regression from the code refactoring and backfill changes in CouchKVStore
Comment by Mike Wiederhold [ 16/Oct/14 ]
I just want to note that doesn't appear be a regression. From what it sounds like Dave is saying when I make a certain change which does not exist in the master branch this test fails. I'll take a look at it, but I have not seen this test case fail anywhere.
Comment by Mike Wiederhold [ 16/Oct/14 ]
Dave,

It appears that one of the merges into the master branch was done incorrectly and now master requires a c++ 11 compiler. As a result I can not currently build the master branch on any of the machines I use.
Comment by Dave Rigby [ 16/Oct/14 ]
@MIke: No idea about that - all I found is that my change (3 weeks ago! to memcached) - caused some ep_engine tests to fail. Even if I back out that change - see http://review.couchbase.org/#/c/41647/ - ep_engine still fails (as per this MB).

It baffles me that ep_engine's unit tests have been broken for (at least) 3 weeks and no-one has noticed. Surely we should have a CI build in place to check for such things...

On the topic of C++11, all I know is that Trond has starting using C++11 features in memcached.
 
Comment by Dave Rigby [ 16/Oct/14 ]
(additionally - if it's any help I can build master on OS X (Mavericks) and Ubuntu 14.04 no problem).
Comment by Mike Wiederhold [ 16/Oct/14 ]
I think my memcached master branch wasn't being checked out properly when ran my tests so I was still using the 3.0.1 branch. I can see in the timings.cc file there are reference to c11 atomics that Trond removed in the 3.0.1 branch, but it appears those changes are not in the master branch. I'll see if I can fix it right now.
Comment by Mike Wiederhold [ 16/Oct/14 ]
Please see the memcached changes to fix the c++11 issues.

http://review.couchbase.org/#/c/42219/
http://review.couchbase.org/#/c/42224/
Comment by Mike Wiederhold [ 21/Oct/14 ]
Cannot reproduce this on the latest code in the branch-master.xml manifest.
Comment by Mike Wiederhold [ 21/Oct/14 ]
I ran this on the wrong branch. The fix is below.

http://review.couchbase.org/#/c/42334/




[MB-12364] Improve logging on auto_failover:reset_auto_failover_count Created: 16/Oct/14  Updated: 16/Oct/14  Resolved: 16/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 2.5.1, 3.0
Fix Version/s: 3.0.1
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Chris Malarky Assignee: Aleksey Kondratenko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Would be very useful to see the reset_auto_failover_count from auto_failover.erl in the same logs as node failover, so user_log() rather than log_debug() ?

 Comments   
Comment by Aleksey Kondratenko [ 16/Oct/14 ]
http://review.couchbase.org/42215




[MB-12363] [ElasticSearch] Remote cluster's IP (of ES endpoint) is mangled from localhost to wlan0's address Created: 16/Oct/14  Updated: 20/Oct/14

Status: Open
Project: Couchbase Server
Component/s: cross-datacenter-replication
Affects Version/s: 3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Egmont Koblinger Assignee: Marty Schoch
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Ubuntu 14.10-beta, 64 bit, CB-community-3.0.0, ElasticSearch 1.3.0, transport 2.0.0

Triage: Untriaged
Operating System: Ubuntu 64-bit
Is this a Regression?: Unknown

 Description   
Under Couchbase Console's XDCR tab, I set up a cluster reference to "127.0.0.1:9091" (or "localhost:9091") where my ElasticSearch is running. Then I set up a proper replication (using version 1).

At this moment, the cluster reference's IP address or hostname gets rewritten to my development laptop's wlan0 IP address, e.g. "192.168.0.10:9091".

The problem with this is that, depending on which network (e.g. work vs. home) I open the laptop, I get different IP address. That is, a previously set cluster reference becomes unusable, I have to destroy and create it again every time.

Expected behavior: Whatever hostname or IP address I specify there should always remain the same, CB should not change it. If I say "localhost", it should stay "localhost".

 Comments   
Comment by Aleksey Kondratenko [ 16/Oct/14 ]
This "renaming" happens because node's name appears to be "192.168.0.10".

In order to confirm I'll need collectinfo(s) from your cluster.
Comment by Egmont Koblinger [ 17/Oct/14 ]
Which node are you referring to, CB or ES? Is there a way to get that node name?

I started CB configuration from scratch (because the collectinfo is damn large otherwise, and anyways contains the ES password which is really not nice). I wish I could attach the 1.5MB file but I keep getting "Cannot attach file collectinfo-2014-10-17T115716-ns_1@127.0.0.1.zip: Unable to communicate with JIRA"

On the opening setup screen of CB there's a field "Configure Server Hostname", I left the default 127.0.0.1 there. I created a new bucket without replication, set up XDCR cluster reference followed by an actual replication to an already existing ES index.
Comment by Aleksey Kondratenko [ 17/Oct/14 ]
Ah I think I misunderstood.

Here's what happens at couchbase server side:

* when you set up cluster reference it goes to specified endpoint and fetches remote cluster info

* It will then rewrite reference if remote cluster info does not have host with same name.

So apparently elasticsearch plugin that's servicing :9091 is sending out node cluster info in this way.
Comment by Aleksey Kondratenko [ 17/Oct/14 ]
Passing this to ES plugin maintainer.
Comment by Marty Schoch [ 17/Oct/14 ]
Elasticsearch allows you to control which network address is used to advertise itself. Our plugin honors this configuration.

The user should find the line in their config file:

#network.publish_host: 192.168.0.1

Uncomment it, and change the publish_host value to the address they want. In this case 127.0.0.1
Comment by Egmont Koblinger [ 18/Oct/14 ]
Thanks guys for your kind responses, indeed setting network.publist_host works.

That being said, I don't understand the rationale behind this method of advertising the IP address and forcing the client to overwrite it. This fails in my particular use case, and might fail in other cases too (e.g. complicated firewall rules, or a host with two interfaces and no IP forwarding that requires that all incoming packets come with the destination address belonging to the incoming interface).

If I specify an IP address on the client (CB) and the client is able to reach the server (ES) then that IP address is working, why overwrite it to something else which might not work in tricky setups?

Looks to me like a step that overcomplicates things for no reason. What am I missing here? :)
Comment by Marty Schoch [ 18/Oct/14 ]
There are really two separate questions.

1. Why does Elasticsearch advertise its own address in this way?
and
2. Why does Couchbase update the XDCR configuration based on the information returned by Elasticsearch?

I can only answer #1.

The Couchbase Elasticsearch plugin was designed to operate within the existing Elasticsearch framework. Part of that includes behaving like the other Elasticsearch components and even sharing their configuration when it makes sense. Now, Elasticsearch, like many other distributed networking services needs to advertise to other services how it can be reached, sometimes this information is broadcast, or recorded in one place for consumption by someone else. In these cases you cannot rely on the idea of "just reach me again on the same address you did to get here approach" because the information is flowing the other direction. In these cases Elasticsearch will first attempt to "guess" which address it should use. Sometimes it guesses wrong and does not choose the most appropriate address. That is why they have the publish.host setting to allow you to override it.

In the cases where a client does initiate an incoming request why doesn't Elasticsearch always respond with that address? First, sometimes that isn't what the administrator wants. And second, I think when binding to a wildcard address (0.0.0.0) it isn't easy to determine from which interface the request came. (My recollection is that you separately bind to each address it is easier) I would say though that if Elasticsearch had a way to do this easily I would offer it as an option.

Perhaps Aleksey may provide insight into #2.
Comment by Aleksey Kondratenko [ 20/Oct/14 ]
ns_server is behaving the way it behaves because target cluster members may change over time. So it has to track all cluster members and rewrite cluster reference when previous reference points to node that's not part of cluster anymore.

In your case there's no way to be sure that 127.0.0.1 is same (and will remain same) as your wlan ip that's advertised by ES.
Comment by Egmont Koblinger [ 20/Oct/14 ]
Thanks again guys for your responses. I'm new to this world of databases/replicas/etc, apparently there's a lot for me to learn about the standard practices here.




[MB-12361] Backport to 2.5.1 - implement access.log like functionality of port 8091 Created: 15/Oct/14  Updated: 21/Oct/14  Resolved: 15/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 2.2.0
Fix Version/s: 2.5.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Dave Finlay Assignee: Aleksey Kondratenko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged

 Description   
This ticket tracks the backporting of http://www.couchbase.com/issues/browse/MB-10020 to 2.5.1 as a hot fix.

 Comments   
Comment by Aleksey Kondratenko [ 15/Oct/14 ]
merged a bunch of commits ending here: http://review.couchbase.org/42181
Comment by Chris Hillery [ 15/Oct/14 ]
Updated rel-2.5.1.xml manifest (and cleaned up some 2.5.1.1-related stuff) here: http://review.couchbase.org/#/c/42189/

Build 2.5.1-1102 underway.




[MB-12359] Remove support for SSL v3 in memcached SSL server sockets to mitigate against the POODLE attack Created: 15/Oct/14  Updated: 16/Oct/14  Resolved: 16/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: memcached
Affects Version/s: 2.5.1, 3.0
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Dave Finlay Assignee: Trond Norbye
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to MB-12358 Remove support for SSL v3 in ns_serve... Resolved
Triage: Untriaged
Flagged:
Release Note
Is this a Regression?: No

 Description   
Poodle attack described here: http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html

More user friendly description here: http://blog.cryptographyengineering.com/2014/10/attack-of-week-poodle.html

My thinking is we currently fix this for 3.0.1 and then create an MB to backport to 2.5.2 whenever that ships.

Suggested fix: remove SSL v3 support in the versions the server SSL socket supports.

 Comments   
Comment by Trond Norbye [ 16/Oct/14 ]
http://review.couchbase.org/#/c/42199/
Comment by Dave Rigby [ 16/Oct/14 ]
Flagging as release note - we should highlight high-profile security fixes such as this.
Comment by Trond Norbye [ 16/Oct/14 ]
Fixed in 3.0.1




[MB-12358] Remove support for SSL v3 in ns_server SSL server sockets to mitigate against the POODLE attack Created: 15/Oct/14  Updated: 16/Oct/14  Resolved: 16/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 2.5.1, 3.0.1
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Dave Finlay Assignee: Aleksey Kondratenko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to MB-12359 Remove support for SSL v3 in memcache... Resolved
Triage: Untriaged
Is this a Regression?: Unknown

 Description   
Poodle attack described here: http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html

More user friendly description here: http://blog.cryptographyengineering.com/2014/10/attack-of-week-poodle.html

My thinking is we currently fix this for 3.0.1 and then create an MB to backport to 2.5.2 whenever that ships.

Suggested fix: remove SSL v3 support in the versions the server SSL socket supports.

 Comments   
Comment by Aleksey Kondratenko [ 15/Oct/14 ]
http://review.couchbase.org/42188
Comment by Aleksey Kondratenko [ 15/Oct/14 ]
Actually forgot to do same for downstream listener of ssl proxy.
Comment by Aleksey Kondratenko [ 16/Oct/14 ]
ssl proxy is covered by: http://review.couchbase.org/42209




[MB-12357] Custom error message from updater Created: 15/Oct/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase Server
Component/s: None
Affects Version/s: 3.0
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Volker Mische Assignee: Volker Mische
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
Make it possible to have a custom error message from the updater.




[MB-12356] unusual high number of gets per sec (20k) opening "Documents" view Created: 15/Oct/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase Server
Component/s: UI
Affects Version/s: 3.0.1
Fix Version/s: bug-backlog
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Aleksey Kondratenko
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Windows 64-bit
Is this a Regression?: Unknown

 Description   
Reported by a user here: https://forums.couchbase.com/t/what-is-causing-high-number-of-gets-per-sec-20k-opening-documents-view-3-0-1/1746

Every time I click on "Documents" to view documents in a Bucket, the page hangs for a little bit while it loads. If I switch over to the General Bucket Analysis graph, I see some 20k gets/sec. I am currently able to produce this behavior consistently. There are no Views for this bucket.

 Comments   
Comment by rdev5 [ 15/Oct/14 ]
Please see https://github.com/rdev5/cachewarmer/tree/master for instructions on how I was able to reproduce this on a new 3.0.0/1 cluster.
Comment by Matt Ingenthron [ 15/Oct/14 ]
Thanks so much for the sample rdev5! That's hugely helpful in finding/addressing the issue.
Comment by rdev5 [ 15/Oct/14 ]
Sure thing. Hope you guys are able to track it down. Let me know if you need anything else.
Comment by rdev5 [ 15/Oct/14 ]
Just an added observation - It appears that flushing the bucket purges it of this behavior. Running the test project again brings it back (even after items have been expired).
Comment by Aleksey Kondratenko [ 15/Oct/14 ]
We have tracked it down.

It's interesting combination of our fundamental lack of support of range ops and the way how expirations work.

Specifically:

* every time you open Documents section of UI it makes "get me 10 docs with lexicographically smallest keys".

* our in-memory layer is unable to handle it directly. So couchbase actually has to go to specific .couch files. This is where problem number one exists: it's unable to see docs that are created but are not persisted yet.

* getting 10 smallest keys from specific per-vbucket .couch file is efficient. But getting list of 10 smallest keys across all 1024 vbuckets is far less efficient. It's quick if your by-id btree of db files fit in page cache, but once you get to larger data sizes expect it to be very slow

* after we've got 10 smallest keys from underlying .couch files we fetch their bodies from caching layer. This is where second issue kicks in.

* It happens because actual expirations are lazy in couchbase (same as upstream memcached in fact). When document gets expired it does not get deleted immediately. Instead, when you GET some key, caching layer checks if document is expired and if it is, then it'll delete it and return "missing" response. There is however periodic task that periodically scans all keys and deletes expired documents.

* so this causes problem for views (both for DCP-based views of 3.0.0 and .couch-file based views of 2.x). And it causes similar "problem" for documents ui.

* so when underlying Documents UI REST API implementation gets list of keys and does GETs, it will in your case discover that those docs are actually gone already. It will try larger window and larger until it finds 10 "live" docs.

* and that's what is causing slowness for you


In general, Documents UI works for development environments (albeit with issues mentioned above). But don't expect Documents UI to be work ok for serious production deployments. It's just not ready.

Also, as far as I know there are no plans to fix it. There are some internal discussions about primary index related to niql work. But I'm not aware of any plans at all w.r.t. expirations problem. So I have no idea when this 2 issues that you're dealing with will be fixed.


Comment by rdev5 [ 15/Oct/14 ]
Two quick questions:

So basically avoid using the Documents UI in production altogether since it could possibly impact server performance for clients in production?

Are older versions like 2.2 and 2.5 also affected by this?
Comment by Aleksey Kondratenko [ 15/Oct/14 ]
>> So basically avoid using the Documents UI in production altogether since it could possibly impact server performance for clients in production?

Yes.

>> Are older versions like 2.2 and 2.5 also affected by this?

Yes. Even more than 3.0, because 3.0's implementation is more optimized.




[MB-12355] Inconsistent placement of Delete button on XDCR page Created: 15/Oct/14  Updated: 16/Oct/14  Resolved: 16/Oct/14

Status: Resolved
Project: Couchbase Server
Component/s: UI
Affects Version/s: 3.0
Fix Version/s: 3.0.1
Security Level: Public

Type: Bug Priority: Trivial
Reporter: Egmont Koblinger Assignee: Aleksey Kondratenko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Ubuntu 64-bit
Is this a Regression?: Unknown

 Description   
The web console's XDCR page looks like:

<one particular remote cluster> [Delete] [Edit]
<one particular ongoing replication> [Settings] [Delete]

I find the inconsistent placement of the Delete button quite confusing, I think it should be either the first button at both places, or the last button at both.

(In the mean time I'm wondering why Edit and Settings are different words, both buttons take you to a dialog where you can edit the settings – albeit they're different, one allows you to edit all the properties, the other one only allows to edit the advanced parameters. So this is just an FYI.)

 Comments   
Comment by Aleksey Kondratenko [ 16/Oct/14 ]
Thanks for feedback.




[MB-12354] Log timestamps lag behind after a laptop suspend Created: 15/Oct/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase Server
Component/s: None
Affects Version/s: 3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Minor
Reporter: Egmont Koblinger Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Laptop with Ubuntu 14.10-beta, 64-bit; CB 3.0 community edition

Triage: Untriaged
Operating System: Ubuntu 64-bit
Is this a Regression?: Unknown

 Description   
I use my laptop for developing to-be-production CB-based software. If I suspend and later wake it up, the timestamps in the log files (e.g. /opt/couchbase/var/lib/couchbase/logs/couchdb.log) will lag behind by the amount my laptop spent suspended.

E.g. if I suspend it for 10 hours overnight, records logged at 2014-10-15T09:00:00 will have the timestamp 2014-10-14T23:00:00 or so. This makes timestamps in the log files effectively useless.

Expected behavior: all the timestamps should reflect the actual time at that point, as correctly known by the operating system, and as returned by the time()/gettimeofday()/localtime()/strftime()/etc. kernel/glibc calls.

(BTW this bug also makes me wonder whether DST changes are handled correctly. I haven't tested that.)




[MB-12353] [Windows]32% regression from 2.5.1 in Rebalance out with views and query workload. Created: 15/Oct/14  Updated: 16/Oct/14

Status: Open
Project: Couchbase Server
Component/s: ns_server, view-engine
Affects Version/s: 3.0.1
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Venu Uppalapati Assignee: Sarath Lakshman
Resolution: Unresolved Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Triage: Untriaged
Operating System: Windows 64-bit
Is this a Regression?: Yes

 Description   
Rebalance out time increased from 242.7 to 322.3 minutes from 2.5.1 to 3.0.1.

Test details:
Rebalance-out (min), 4 -> 3, 1 bucket x 100M x 2KB, 1 x 1 views, 10K ops/sec, 400 queries/sec

Observations (2.5.1 vs 3.0.1):

12x increase in bg_fetch latency from 250us to 3000us.
23x decrease in avg disk update time from max of 70us to 3us.
~4x increase in 90 percentile query latency from 13ms to 46ms.
cache miss rate ~2% in both runs.
active resident ratio 20-25% in both runs.

machine details:
RAID 10 HDD
24 vCPU, 64 GB
links:
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_301-1330_606_rebalance
http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=zeus_251-1083_20f_rebalance

logs:
http://ci.sc.couchbase.com/job/zeus-64/1218/artifact/172.23.96.25.zip
http://ci.sc.couchbase.com/job/zeus-64/1218/artifact/172.23.96.26.zip
http://ci.sc.couchbase.com/job/zeus-64/1218/artifact/172.23.96.27.zip
http://ci.sc.couchbase.com/job/zeus-64/1218/artifact/172.23.96.28.zip

 Comments   
Comment by Venu Uppalapati [ 16/Oct/14 ]
Assigning to view team for initial investigation since rebalance out for KV case has lesser regression of 12.5%.




[MB-12352] Remove unnesscary couchstore open/reads from backfill Created: 15/Oct/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase Server
Component/s: couchbase-bucket
Affects Version/s: 3.0
Fix Version/s: sherlock
Security Level: Public

Type: Bug Priority: Major
Reporter: Jim Walker Assignee: Jim Walker
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Observed on many.

Triage: Untriaged
Is this a Regression?: Unknown

 Description   
Whilst investigating rebalance performance (time, disk and net I/O) I was curious as to why a 100% resident rebalance was reading from disk (Linux disk caches dropped prior to rebalance).

Found DCP backfill performs a call to retrieve the number of items in a couchstore data file followed by a "dump" of the file. These are two open/closes, the first one looks to be removable.

Given that the dump of seqno -> end is going to execute the CacheCallback for each K, we can use the CacheCallback to also increment "backfill-remaining"

This appears a safe thing todo, I can't see that the dump/scan requires the backfill count? If it does then we can scrap this simple fix and i'll move along to the next set of tests and investigations :)

With the patch it was hard to see if there's a perf improvement, but we do reduce the amount of data read from disk. The clearest stat was measuring how many pread64's a node performed during rebalance. Average of 2 runs shows the following.

1. current master - 28,902 calls to pread64
2. With patch - 24,170 calls to pread64

That's 16% less. It's gotta be useful?



 Comments   
Comment by Jim Walker [ 15/Oct/14 ]
http://review.couchbase.org/42156
Comment by Jim Walker [ 15/Oct/14 ]
When I say hard to measure perf improvement, this refers to real-time of the rebalance. The rebalance without this fix had a variation of around 20% and no obvious/dramatic difference after the patch, only directly measuring pread was any "benefit" observed.

With bigger and bigger vbuckets, this may be more useful. I only had 4.5 million items (vbucket disk size around 1.7MB)




[MB-12351] Faulty ISO 8601 timestamp format in log files (one-digit hour) Created: 15/Oct/14  Updated: 15/Oct/14

Status: Open
Project: Couchbase Server
Component/s: None
Affects Version/s: 3.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Trivial
Reporter: Egmont Koblinger Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: 5m
Time Spent: Not Specified
Original Estimate: 5m
Environment: Ubuntu 14.04 and 14.10-beta, 64-bit; CB 3.0.0 community edition

Triage: Untriaged
Operating System: Ubuntu 64-bit
Is this a Regression?: Unknown

 Description   
The log files under /opt/couchbase/var/lib/couchbase/logs contain timestamps along the lines of
2014-10-15T9:59:50.272

Even though the ISO 8601 standard itself is not available for free so I can't verify that for sure, but extracts of that (such as http://en.wikipedia.org/wiki/ISO_8601 and http://www.w3.org/TR/NOTE-datetime) state that the hour should be HH, that is, two digits, zero-padded if necessary.





Generated at Wed Oct 22 05:52:39 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.