Couchbase Server

[RN 2.0.2 + Doc]2.0.0 cluster restore with data and index

Details

Description

In our document showing how to restore from one cluster to another identical cluster (http://www.couchbase.com/docs/couchbase-manual-1.8/couchbase-backup-restore-restore.html at session 5.2.2.2. Restoring to a different cluster),
it shows only restore data.
In 2.0.0, we have @indexes directory to store index. What is the instruction to do cluster restore (data and indexes) from one cluster to another cluster.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Aleksey Kondratenko added a comment -
AFAIK there's no need to do anything. Indexes will be automatically rebuilt. Or at least I think so. No sure we should care about that however.
Show
Aleksey Kondratenko added a comment - AFAIK there's no need to do anything. Indexes will be automatically rebuilt. Or at least I think so. No sure we should care about that however.
Hide
Permalink
Aleksey Kondratenko added a comment -
Pinging Farshid
Show
Aleksey Kondratenko added a comment - Pinging Farshid
Hide
Permalink
Aleksey Kondratenko added a comment -
And Yaseen
Show
Aleksey Kondratenko added a comment - And Yaseen
Hide
Permalink
Thuan Nguyen added a comment -
In large cluster with more than 30 million items, rebuild index may take more than a day to complete. So in cluster restore in 2.0.0, do we need to copy both data and indexes files to new cluster to skip rebuild initial index?
Show
Thuan Nguyen added a comment - In large cluster with more than 30 million items, rebuild index may take more than a day to complete. So in cluster restore in 2.0.0, do we need to copy both data and indexes files to new cluster to skip rebuild initial index?
Hide
Permalink
Farshid Ghods added a comment -
this is a use case which we need to discuss for 2.1 or 3.0 release.
as part of cluster restore which is meant for large deployments we want to allow them to also copy their index files as well as their data files

currently cbbackup/cbrestore indeed restore the data files but also definition of design documents .

the cluster restore feature on the other hand is now designed to work for data files and not indexes.

Tony'
it is clear that indexes are not restored but can you confirm whether design docs are created or not after restore ?
Show
Farshid Ghods added a comment - this is a use case which we need to discuss for 2.1 or 3.0 release. as part of cluster restore which is meant for large deployments we want to allow them to also copy their index files as well as their data files currently cbbackup/cbrestore indeed restore the data files but also definition of design documents . the cluster restore feature on the other hand is now designed to work for data files and not indexes. Tony' it is clear that indexes are not restored but can you confirm whether design docs are created or not after restore ?
Hide
Permalink
Farshid Ghods added a comment -
please reassign to Yaseen after comemtning
Show
Farshid Ghods added a comment - please reassign to Yaseen after comemtning
Hide
Permalink
Steve Yen added a comment -
looks like the instructions may need a scrub, too.
Show
Steve Yen added a comment - looks like the instructions may need a scrub, too.
Hide
Permalink
Steve Yen added a comment -
bug-scrub: since this is docs, just need alk to evaluate docs (he believes cluster-restore works)
Show
Steve Yen added a comment - bug-scrub: since this is docs, just need alk to evaluate docs (he believes cluster-restore works)
Hide
Permalink
Aleksey Kondratenko added a comment -
Tried manually. Everything works as expected. master couch database is restored as part of everything and indexes are back. Just being rebuilt after restore
Show
Aleksey Kondratenko added a comment - Tried manually. Everything works as expected. master couch database is restored as part of everything and indexes are back. Just being rebuilt after restore
Hide
Permalink
Aleksey Kondratenko added a comment -
The only thing that may be somewhat problematic is lack of _exact_ instructions of what data files are here (quoting from documentation):

The necessary steps for migrating data using this method are as follows:

Take a backup of the data files of all nodes, using the above procedure. Alternately, shut down the couchbase-server on all nodes and copy the DB files.

Install Couchbase Server (of at least version 1.7.1) on new nodes and cluster together. If using the web console to setup your cluster, a 'default' bucket will be created. Please delete this bucket before proceeding.

Show
Aleksey Kondratenko added a comment - The only thing that may be somewhat problematic is lack of _exact_ instructions of what data files are here (quoting from documentation): The necessary steps for migrating data using this method are as follows: Take a backup of the data files of all nodes, using the above procedure. Alternately, shut down the couchbase-server on all nodes and copy the DB files. Install Couchbase Server (of at least version 1.7.1) on new nodes and cluster together. If using the web console to setup your cluster, a 'default' bucket will be created. Please delete this bucket before proceeding.
Hide
Permalink
Farshid Ghods added a comment -
per bug scrub.

the cluster restore feature on 2.0 is going to only restore data files. the user can use cbbackup/cbrestore tool to backup the ddoc definitions and restore it to the new cluster.

and then they need to wait for index to build on the new cluster
Show
Farshid Ghods added a comment - per bug scrub. the cluster restore feature on 2.0 is going to only restore data files. the user can use cbbackup/cbrestore tool to backup the ddoc definitions and restore it to the new cluster. and then they need to wait for index to build on the new cluster
Hide
Permalink
Karen Zeller added a comment -
Removing documentation as component until we confirm this item for 1) release notes, or 2) required new content.
Show
Karen Zeller added a comment - Removing documentation as component until we confirm this item for 1) release notes, or 2) required new content.
Hide
Permalink
Karen Zeller added a comment -
Is this something that needs to be documented? If so, what is the point we want to make?
Show
Karen Zeller added a comment - Is this something that needs to be documented? If so, what is the point we want to make?
Hide
Permalink
Thuan Nguyen added a comment -
I think we need to add more info for restore in 2.0.0 and later if user has design doc and view. After restore data to cluster, just re-create doc, view and indexes will be rebuild.
Show
Thuan Nguyen added a comment - I think we need to add more info for restore in 2.0.0 and later if user has design doc and view. After restore data to cluster, just re-create doc, view and indexes will be rebuild.
Hide
Permalink
Maria McDuff added a comment -
abhinav, pls work with Karen for her to doc additional restore functionality/expected behavior.
Show
Maria McDuff added a comment - abhinav, pls work with Karen for her to doc additional restore functionality/expected behavior.
Hide
Permalink
Maria McDuff added a comment -
tony, before closing this bug, verify if there's any doc for karen to do. thanks.
Show
Maria McDuff added a comment - tony, before closing this bug, verify if there's any doc for karen to do. thanks.
Hide
Permalink
Wayne Siu added a comment -
Updating the fix version to 2.0.2. Please review the current instruction, and provide addition/new steps if needed.
Show
Wayne Siu added a comment - Updating the fix version to 2.0.2. Please review the current instruction, and provide addition/new steps if needed.
Hide
Permalink
Anil Kumar added a comment -
Tony, can you provide additional info and assign this to Karen for documentation.
Show
Anil Kumar added a comment - Tony, can you provide additional info and assign this to Karen for documentation.
Hide
Permalink
Thuan Nguyen added a comment -
Just talk to Aleksey, all we need to backup is database files and restore them to new cluster.
In document for backup restore in 2.0,http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-backup-restore-restore.html
 I don't see how to restore docs and view in that page. We need to add how to restore views into that page.
To restore docs and views, all user need to do is to re-create docs and views after restore data to new cluster. Then index will rebuild automatically.
Show
Thuan Nguyen added a comment - Just talk to Aleksey, all we need to backup is database files and restore them to new cluster. In document for backup restore in 2.0, http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-backup-restore-restore.html  I don't see how to restore docs and view in that page. We need to add how to restore views into that page. To restore docs and views, all user need to do is to re-create docs and views after restore data to new cluster. Then index will rebuild automatically.
Hide
Permalink
Maria McDuff added a comment -
karen, pls work with tony to get this doc'd for 2.0.2.
thanks.
Show
Maria McDuff added a comment - karen, pls work with tony to get this doc'd for 2.0.2. thanks.
Hide
Permalink
Karen Zeller added a comment -
so when I read through this comment thread it appears this ticket is resolved by the new content on cbrestore of design documents for MB-7300. This has been sent out for review.

Consolidating.
Show
Karen Zeller added a comment - so when I read through this comment thread it appears this ticket is resolved by the new content on cbrestore of design documents for MB-7300 . This has been sent out for review. Consolidating.
Hide
Permalink
Karen Zeller added a comment -
so when I read through this comment thread it appears this ticket is resolved by the new content on cbrestore of design documents for MB-7300. This has been sent out for review.

Consolidating.
Show
Karen Zeller added a comment - so when I read through this comment thread it appears this ticket is resolved by the new content on cbrestore of design documents for MB-7300 . This has been sent out for review. Consolidating.

People

  • Assignee:
    Karen Zeller
    Reporter:
    Thuan Nguyen
Vote (0)
Watch (5)

Dates

  • Created:
    Updated:
    Resolved: