Details
-
Type:
Bug
-
Status:
Resolved
-
Priority:
Blocker
-
Resolution: Fixed
-
Affects Version/s: 2.0
-
Fix Version/s: 2.0
-
Component/s: documentation
-
Security Level: Public
-
Labels:None
Description
Thankfully this didn't happen on a live system, but running the following command against a node of a cluster with buckets already configured deleted them all:
/opt/couchbase/bin/couchbase-cli node-init -c 10.145.129.217:8091 -u Administrator -p administrator --node-init-data-path=/disk2/couchbase/data/
Let me know if you need me to reproduce this, so far has happened twice (and I expect to continue as I go through the rest of the nodes)
Perry
/opt/couchbase/bin/couchbase-cli node-init -c 10.145.129.217:8091 -u Administrator -p administrator --node-init-data-path=/disk2/couchbase/data/
Let me know if you need me to reproduce this, so far has happened twice (and I expect to continue as I go through the rest of the nodes)
Perry
-
Hide
- logs.zip
- 04/Dec/12 6:22 AM
- 12.78 MB
- Perry Krug
-
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/couchbase.log 761 kB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.xdcr.log 9 kB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.couchdb.log 1.58 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/stats.log 1.29 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.stats.log 120.76 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.error.log 30.62 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.views.log 8.33 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.info.log 51.55 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.xdcr_errors.log 0.2 kB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.mapreduce_errors.log 0.2 kB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/diag.log 8.30 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ns_server.debug.log 119.44 MB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/ddocs.log 0.3 kB
- cbcollect_info_ns_1@127.0.0.1_20121204-121640/memcached.log 9 kB
Activity
- All
- Comments
- Work Log
- History
- Activity
- Gerrit Reviews
Hide
Permalink
Perry Krug
added a comment -
As a note, changing the data path of 1 node in a cluster removes the buckets across the ENTIRE cluster. This command should only affect the node it is being run against...
Show
Perry Krug
added a comment - As a note, changing the data path of 1 node in a cluster removes the buckets across the ENTIRE cluster. This command should only affect the node it is being run against...
Hide
Perry Krug
added a comment -
Adding to documentation since there should be an extremely large warning about doing this on a live cluster until this bug is fixed. Note this is a regression from 1.x
Show
Perry Krug
added a comment - Adding to documentation since there should be an extremely large warning about doing this on a live cluster until this bug is fixed. Note this is a regression from 1.x
Hide
Farshid Ghods
added a comment -
similar bug here :
http://www.couchbase.com/issues/browse/MB-7257
given that this is a regression and it actually impacts the entire cluster i recommend fixing this in 2.0
http://www.couchbase.com/issues/browse/MB-7257
given that this is a regression and it actually impacts the entire cluster i recommend fixing this in 2.0
Show
Farshid Ghods
added a comment - similar bug here :
http://www.couchbase.com/issues/browse/MB-7257
given that this is a regression and it actually impacts the entire cluster i recommend fixing this in 2.0
Show
Aleksey Kondratenko
added a comment - Not similar but exactly same
Hide
Aleksey Kondratenko
added a comment -
Decision was made to disallow procedure described in docs. As soon as cluster is provisioned, there's no way to change data path.
Show
Aleksey Kondratenko
added a comment - Decision was made to disallow procedure described in docs. As soon as cluster is provisioned, there's no way to change data path.
Hide
Perry Krug
added a comment -
So we need a doc update on how to change the data path. It makes enough sense that you need to remove the node. Has the code been changed to prevent this from happening?
Show
Perry Krug
added a comment - So we need a doc update on how to change the data path. It makes enough sense that you need to remove the node. Has the code been changed to prevent this from happening?
Hide
Aleksey Kondratenko
added a comment -
Commit to prevent change of data path of provisioned cluster nodes is merged: http://review.couchbase.org/#/c/23113/
Show
Aleksey Kondratenko
added a comment - Commit to prevent change of data path of provisioned cluster nodes is merged: http://review.couchbase.org/#/c/23113/
Hide
Farshid Ghods
added a comment -
Hi Aliaksey,
as part this change(http://review.membase.org/#/c/23113/) we made sure that if the node is provisioned the user will not be able to change the data path settings even though there are no buckets on the node. my question is what is the procedure now for a user who would like to change the data path of a node :
1- should they re-install couchbase and then go through the wizard for modifying such path
Yes
2- is there another REST endpoint where they can invoke to un-provision a node
Yes, somewhat.
wget -O- -vvd --user=Administrator --password=asdasd --post-data="otpNode=zzzzForce" http://lh:9000/controller/ejectNode
as part this change(http://review.membase.org/#/c/23113/) we made sure that if the node is provisioned the user will not be able to change the data path settings even though there are no buckets on the node. my question is what is the procedure now for a user who would like to change the data path of a node :
1- should they re-install couchbase and then go through the wizard for modifying such path
Yes
2- is there another REST endpoint where they can invoke to un-provision a node
Yes, somewhat.
wget -O- -vvd --user=Administrator --password=asdasd --post-data="otpNode=zzzzForce" http://lh:9000/controller/ejectNode
Show
Farshid Ghods
added a comment - Hi Aliaksey,
as part this change( http://review.membase.org/#/c/23113/ ) we made sure that if the node is provisioned the user will not be able to change the data path settings even though there are no buckets on the node. my question is what is the procedure now for a user who would like to change the data path of a node :
1- should they re-install couchbase and then go through the wizard for modifying such path
Yes
2- is there another REST endpoint where they can invoke to un-provision a node
Yes, somewhat.
wget -O- -vvd --user=Administrator --password=asdasd --post-data="otpNode=zzzzForce" http://lh:9000/controller/ejectNode
Hide
Farshid Ghods
added a comment -
#2 > reset the node if node is still in a provisioned state after removing from the cluster. to reset the node you can use ejectNode api or reinstall couchbase
#3. update the data path
#4. re-add the node
#3. update the data path
#4. re-add the node
Show
Farshid Ghods
added a comment - #2 > reset the node if node is still in a provisioned state after removing from the cluster. to reset the node you can use ejectNode api or reinstall couchbase
#3. update the data path
#4. re-add the node
Hide
Perry Krug
added a comment -
FYI, you don't "need" to update the path via the REST API. The first page of the setup wizard lets you set the data path. So I would say the documentation should be:
If you want to change the data path of an existing node, you should first rebalance it out of the cluster (making sure you follow these best practices so as not to reduce the capacity of your cluster and impact performance...in here we should have that the best practice is to always swap it with a new node, do we have that somewhere yet?) and then change the data path either using the web UI, the CLI or the REST API and add it back to the cluster with a rebalance.
If you want to change the data path of an existing node, you should first rebalance it out of the cluster (making sure you follow these best practices so as not to reduce the capacity of your cluster and impact performance...in here we should have that the best practice is to always swap it with a new node, do we have that somewhere yet?) and then change the data path either using the web UI, the CLI or the REST API and add it back to the cluster with a rebalance.
Show
Perry Krug
added a comment - FYI, you don't "need" to update the path via the REST API. The first page of the setup wizard lets you set the data path. So I would say the documentation should be:
If you want to change the data path of an existing node, you should first rebalance it out of the cluster (making sure you follow these best practices so as not to reduce the capacity of your cluster and impact performance...in here we should have that the best practice is to always swap it with a new node, do we have that somewhere yet?) and then change the data path either using the web UI, the CLI or the REST API and add it back to the cluster with a rebalance.
Hide
MC Brown
added a comment -
Documentation has been updated with a new section with the recommended process:
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-admin-tasks-changepath.html
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-admin-tasks-changepath.html