Database Migration issues to delevloper preview 3

There's no official way to upgrade from one version to the next, but there is no change to the file format so you should be able to:
1) Back up (or move out of the way) the data files
2) Remove the package
3) Install the new package
4) Configure the new node with the same buckets
5) Shutdown
6) Drop in the data files

That's single node only, obviously. It might work without deleting the data files and just updating the package, but this hasn't really been tested.

It *may* also be possible to rebalance from one release to the next. It hasn't been tested (we focus testing on from the previous release, not previous dev preview), but I don't know if there's an incompatibility.

It does NOT seem to be possible to rebalance from the previous dev preview to the current one (dev preview 3).

I tried removing a node, upgrading that node (cannot upgrade in place, must uninstall old, then install new), then joining an rebalancing. Did not work as expected.

Am experiencing the same problem, where old nodes can 'see' the new node, but the new nodes get 'lost connection to server at localhost: 8091', and thus it cannot 'see' the other nodes. Same goes after even following this process with two nodes.

-ChssAddct

You may have to remove all nodes but one, rebalance, then upgrade in place, then rebalance back out. Sorry there is not a better option here, since this is not fully released yet, there may be changes that are not compatible between previews.

Hi,
and Happy new year...

Thanks for the tipp. As i'm back from holidays i will try your suggestion with a clone of the cluster and report back when i got some results.

Best regards,

Andreas

tried your suggestuins

i created a clone of the cluster (with has only 2 nodes)
* create two blank nodes (with the same version of Couchbase 2.0.0r-1-ge4c8742.deb as on the original nodes)
* join them in a cluster with replikation level n-1 (whch is 2 -1 = 1 in this)
* rebalance
* create the bucket(s) with same name and config as on original cluster
* stop all nodes
* restore backup of the original node CouchDB backups (i do only the /opt/couchbase/var/lib/membase/data/XYZ dirs only) to every new node
* restart all nodes
* wait for them to catch up with the new data. (Seems no rebalance to be neccessary)
* check data in the cluster....

After that i did a failover of the second (new) node so i have only one node in the cluster with all the data.

First Try:
Install the preview3 (couchbase-server-community_x86_64_2.0.0-dev-preview-3.deb) over the existing one (couchbase-server-community_x86_64_2.0.0r-1-ge4c8742.deb).

And....
all buckets are empty. :(

Second Try:
Get an updated node to join a cluster with an old version full-data node also fails. there are some reports in the logs about some error repeating very often so my guess is its some differences in the versions protocoll that don't let both versions talk with each other...
So the rebalancing hangs forever and all data remains on the old version node and the cluster is unresponsive to queries.

Third try:
doing a full json dump of the old cluster (how?), creating a new cluster with new version, play back the json data to the new cluster.
But i don't seem to get the dump of all documents in all buckets... Any tips?

Thanks in advance,

Andreas

I believe you mean to address your comments to ingenthr. I simply stated my experience (problems) with updating from earlier developer preview to later. I didn't offer suggestions.

Good luck!

-ChssAddct

i need to clarify my last post:
"empty" means all documents are empty (that is in the WebGUI and Futon), i got a list of documents that looks complete though.

i saw that my backup has only 16 *.couch files (and in a different dir too) and the created bucket got 256 couch files.
so i tried:
* deleting all files in the created database dir and having only the backed up 16 files in there and
* "merging" the backup files with the created 256 (empty) files.
* only copying the numberd file 0.couch to 15.couch in the database dir.

I all case i got the same result: A semmeingly complete List of the documents but they all are completly empty (Furton says "missing").

Some changes in the underlying CouchDB storage perhaps?

Hi again,

after some trial and (mostly) error i reverted to a php script that reads a dump (made with a REST call to "/_all_docs?include_docs=true") and put the contents in it into a couchbase (via the memcache interface).
And an extra script to upload the views. (all_docs seems to exclude them).
The script work with an older and preview3 version of couchbase on ubuntu64.
Seems to work, just not very elegant...

Best Regards,

Andreas

Sorry it was such trouble, but I'm glad you were able to migrate the data. Any chance you'd be willing to post the script somewhere so others can make use of it too?

Note that one minor improvement might be to use _all_docs to get the document _id, but actually fetch the doc over memcached protocol. If you use our PHP client, with include_docs, I believe that's what it will automagically do for you.

Actually i tried to attach this script here, but there is no Attach Button here.

In the present state the script reads form stdin, eg. an former dump so no need to have access to the old couchbase installation (firewall or already shut down or ...). Which i think is a good idea, for now...
I will clean up the script and make it more reliable and presentable and then post a url here.

Here is the core of a small php script, removed all "comfort extras" like error and optopn handling and declarations ($cb ist the couchbase php object for accessing Couchbase)

The php script:

$cb = new Couchbase();
// in content is the text of the dump or bulk document
$array = json_decode($content, true);
 
foreach ($rows as $row) {
// check json type 
// Bulk has an array fo docs named docs
// dump has an array rows with multiple key and on of them is "doc" containing the document.
    if ($isBulk) $doc = $row;
    if ($isDump) $doc = $row["doc"];
 
    // _rev is bad if on a new server...
    if (array_key_exists("_rev", $doc)) unset($doc['_rev']);
    if (array_key_exists("_id", $doc)) {
        $id = $doc['_id'];
        if (strpos($id, "_design") !== false) {
            // store design documents / Views with HTTP REST API
            $result = _saveDesign($doc);
        } else {
            // content over memcache interface
            unset($doc['_id']); 
            $result = $cb->set($id, json_encode($doc));
        }
    }
}
 
function _saveDesign($doc)
{
    global $cb; // YUCK
    // get _rev
    $matches = array();
    if (false === preg_match('/^_design\/(.*)/', $doc["_id"], $matches) ) {
        return;
    }
    $name = $matches[1];
    $old = json_decode($cb->couchdb->open("_design/$name"));
    if (!isset($old->error)) {
        $doc["_rev"] = $old->_rev;
    }
    $result = $cb->couchdb->saveDoc(json_encode($doc));
    $json = json_decode($result);
    return $result;
}

Lets hope its useful for someone...

For now i have a different Problem:
Couchbase hangs on my updated Ubunutu Oneiric.
I don't get any abnswers and a stop hangs indefinitly until i kill all couchbase or erlang processes manually.

But that's an other story...

On Ubuntu and Debain Squeeze i can't get the preview3 to run stable.
Looks like a conflict in some libs, for now we keep the previous version.
But i'll need to update in the near future, so i keep trying, especially when DP4 is out..

DP4 was released last week. See couchbase.com/download

We are currently in the process of migrating to dp4. :)

Found one minor issue right now... :-/

Thanks..