[MB-6423] Allow users to set index_path post upgrade to 2.0 Created: 25/Aug/12  Updated: 09/Jan/13  Resolved: 08/Oct/12

Status: Closed
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 2.0-beta
Fix Version/s: 2.0
Security Level: Public

Type: Bug Priority: Major
Reporter: Karan Kumar (Inactive) Assignee: Aliaksey Artamonau
Resolution: Fixed Votes: 0
Labels: system-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 1639

Use case:-
1) Users upgrade to 20 from 181
2) They decide to separate out partition for indexes. Currently there is no way to configure just index_path without wiping out data_path or reconfiguring the cluster.
 Currently, issuing the data_path command will wipe out the database files.

One can configure the data path
curl -d path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

This does not seem to work...
curl -d index_path=/data2 http://localhost:8091/nodes/self/controller/settings

Comment by Karan Kumar (Inactive) [ 25/Aug/12 ]
I guess right now, we would need separate rest api to configure just indexes.
Not a bug, but a feature i guess.

Following seems to work fine.
curl -d "path=/data&index_path=/indexes" http://localhost:8091/nodes/self/controller/settings
Comment by Farshid Ghods [ 25/Aug/12 ]
yes for our testing we want to try and put each index on a seperate disk. not sure if there is an api that allows us to do so
Comment by Aleksey Kondratenko [ 27/Aug/12 ]
We don't plan to have such API. You can manually replace files with symlinks. Another option is raid 0
Comment by Karan Kumar (Inactive) [ 07/Sep/12 ]
From capacity planning point of view,

if customer upgrade to 20 and later plan to create views,

by default, views would get created on the same partition as the database files.

There is no separate rest_api to setup path for just indexes.
Comment by Aleksey Kondratenko [ 07/Sep/12 ]
I see _no point at all_ having _separate_ API for setting up just index path.

We _do_ have API to setup separate index path.

The only thing we dont have is api to set up per-index path.
Comment by Aleksey Kondratenko [ 07/Sep/12 ]
And my performance recommendation would be:

instead of have disk spindle per index, just raid 0 all spindles you have. That should be as good as or even better than spindle per index.
Comment by Karan Kumar (Inactive) [ 07/Sep/12 ]
Sure. I can create a separate ticket to test out raid 0 and see if it provides better performance.
Comment by Karan Kumar (Inactive) [ 07/Sep/12 ]
Will try to convey this in a better way:-

I can configure the data_path as following
curl -d path=/data http://localhost:8091/nodes/self/controller/settings

BUT, I cant configure the index_path as following (not per index, but for all indexes)
curl -d index_path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

curl -vvv -d "index_path=/data" http://Administrator:password@localhost:8091/nodes/self/controller/settings
* About to connect() to localhost port 8091 (#0)
* Trying ::1... Connection refused
* Trying connected
* Connected to localhost ( port 8091 (#0)
* Server auth using Basic with user 'Administrator'
> POST /nodes/self/controller/settings HTTP/1.1
> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==
> User-Agent: curl/7.19.7 (x86_64-unknown-linux-gnu) libcurl/7.19.7 NSS/ zlib/1.2.3 libidn/1.18 libssh2/1.2.2
> Host: localhost:8091
> Accept: */*
> Content-Length: 16
> Content-Type: application/x-www-form-urlencoded
< HTTP/1.1 400 Bad Request
< Server: Couchbase Server 2.0.0-1695-rel-enterprise
< Pragma: no-cache
< Date: Fri, 07 Sep 2012 19:10:36 GMT
< Content-Type: application/json
< Content-Length: 47
< Cache-Control: no-cache
* Connection #0 to host localhost left intact
* Closing connection #0
["The database or index path cannot be empty."]

The only way for me to configure index_path is by specfying both index_path and data_path
as following:-
curl -d "path=/data&index_path=/indexes" http://localhost:8091/nodes/self/controller/settings

The above command nukes the bucket and looses deletes all the data.

Comment by Karan Kumar (Inactive) [ 10/Sep/12 ]
Well here, I am not trying to point out a performance issue. I am trying to point out a functional bug.

I will take this offline with Alk
Comment by Farshid Ghods [ 19/Sep/12 ]
Any updates ?

do you need to discuss this with Dipti ?
Comment by Aleksey Kondratenko [ 25/Sep/12 ]
Looks like simply implementing new API or extending old in some way is the way to go here.
Comment by Farshid Ghods [ 08/Oct/12 ]
Comment by Andrei Baranouski [ 09/Oct/12 ]
build 1820

verification steps:
1. 1 node with default bucket and 1 item in it
2. create 1 view
3. change index path
curl --user Administrator:password -d index_path=/tmp/bbbbb/ http://localhost:8091/nodes/self/controller/settings
observation: view works after indexing, data index location was changed
4. change index path in second time
curl --user Administrator:password -d index_path=/tmp/aaaa/ http://localhost:8091/nodes/self/controller/settings
observation: view works after indexing, data index location was changed on new
5. change data path
curl --user Administrator:password -d path=/tmp/data_folder/ http://localhost:8091/nodes/self/controller/settings

result: view is left on UI for some time but with error on request {"error":"not_found","reason":"Design document _design/dev_a not found"}
after a couple minutes ddoc/view is deleted from ui ( browser cash was expired)
Comment by Aleksey Kondratenko [ 09/Oct/12 ]
Andrei so ?

BTW changing data path after the fact is a bad idea. We don't necessarily handle it well.
Comment by kzeller [ 26/Oct/12 ]
In RN: This enables users to change the disk location of an index without destroying
persisted data. You can now set index_path and it will delete an existing index only
and create a new disk location for use.
Generated at Thu Apr 17 01:20:45 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.