[MB-4564] Don't delete database files when node leaves cluster. Created: 21/Dec/11  Updated: 13/May/12  Resolved: 09/Apr/12

Status: Resolved
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 1.7.2
Fix Version/s: 1.8.1
Security Level: Public

Type: Improvement Priority: Blocker
Reporter: Perry Krug Assignee: Aliaksey Artamonau
Resolution: Fixed Votes: 0
Labels: 1.8.1-release-notes, customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A recent bug brought up a concern around the process of deleting the on-disk data files when a node is failed over or removed from the cluster.

I think it would be a better approach to preserve this data in the event that it is needed. Perhaps moving them out of the way and messaging to the user that they can clean them up manually when needed.

 Comments   
Comment by Aleksey Kondratenko [ 30/Jan/12 ]
Perry, you want files preserved for eternity or we can/should get rid of them when node is re-initialized or joins some cluster ?
Comment by Perry Krug [ 30/Jan/12 ]
I think the join cluster could be a fine situation to clean them up. What do you mean by re-initialized? This is really only needed as a fail-safe situation in case we need to recover the data from the node that was failed over or removed.
Comment by Aleksey Kondratenko [ 30/Jan/12 ]
that's when/if node is initialized via setup wizard as standalone cluster
Comment by Perry Krug [ 30/Jan/12 ]
Oh, yes...then that's a good time as well too. My only concern would be that the current code path re-initializes the node when it gets failed over or removed so we'd have to change that behavior.
Comment by Steve Yen [ 30/Mar/12 ]
Aliaksey still needs final requirements on how this needs to behave.
Comment by Steve Yen [ 03/Apr/12 ]
Please document decision here.
Comment by Aleksey Kondratenko [ 07/Apr/12 ]
Aliaksey took this.

Decision was to don't touch files when node is ejected/failed over. But rid of them either in the middle of wizard or when rebalance back in starts.
Comment by Perry Krug [ 09/Apr/12 ]
Could we also add a task to notify the user when the files are about to be deleted (either in the wizard or when adding back / rebalancing)?
Comment by Aleksey Kondratenko [ 09/Apr/12 ]
how you want user to be notifed? We can use user-visible log message or we can have alert-like thing.
Comment by Perry Krug [ 09/Apr/12 ]
I'd specifically like something that makes the user confirm that they understand that proceeding with whatever action will result in the deleting of files they may want to save. A log message after the fact doesn't give them the ability to recover that data.
Comment by Steve Yen [ 09/Apr/12 ]
alk predicts this evening ETA
Comment by Aleksey Kondratenko [ 09/Apr/12 ]
http://review.couchbase.org/14628
Comment by Thuan Nguyen [ 20/Apr/12 ]
Integrated in github-ns-server-2-0 #333 (See [http://qa.hq.northscale.net/job/github-ns-server-2-0/333/])
    MB-4564 Don't delete database files when node leaves cluster. (Revision ac961c9839396f64ef6c0ef56e5f298095a0da63)

     Result = SUCCESS
Aliaksey Kandratsenka :
Files :
* src/ns_memcached.erl
* src/ns_cluster.erl
* src/menelaus_web_buckets.erl
* src/menelaus_web.erl
* src/ns_storage_conf.erl
* src/ns_rebalancer.erl
Generated at Wed Jul 30 16:42:57 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.