[MB-3922] rpm -U does not restore the data after upgrade for 1.6.0,1.6.1,1.6.3 Created: 30/May/11  Updated: 09/Jan/13  Resolved: 17/Jun/11

Status: Closed
Project: Couchbase Server
Component/s: ns_server
Affects Version/s: 1.7 beta
Fix Version/s: 1.7 GA
Security Level: Public

Type: Bug Priority: Major
Reporter: Farshid Ghods Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: 1.7.0-release-notes, 1.7.1-release-notes
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: basestar-350

Attachments: GZip Archive ns-diag-20110531103911.txt.gz    
Issue Links:
Duplicate
duplicates MB-3921 rpm -U does not restore the data aft... Closed
Flagged:
Release Note

 Description   
I will post more logs later.

rpm -U reports that it has moved over 850 items but after upgrading to 1.7.0 default bucket does not have any data.


[2011-05-30 11:03:13,426] - [remote_util] [47264785109232] - INFO - running command rpm -U /tmp/membase-server-enterprise_x86_64_basestar-351-g7f6cf4d.rpm
[2011-05-30 11:03:25,169] - [remote_util] [47264785109232] - INFO - command executed successfully
[2011-05-30 11:03:25,169] - [remote_util] [47264785109232] - ERROR - warning: /etc/opt/membase/1.6.1/ns_1/config.dat saved as /etc/opt/membase/1.6.1/ns_1/config.dat.rpmsave
[2011-05-30 11:03:25,169] - [remote_util] [47264785109232] - INFO - Stopping membase ...
[2011-05-30 11:03:25,169] - [remote_util] [47264785109232] - INFO - Stopping Membase server[ OK ]
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO - Upgrading membase ...
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO - /opt/membase/bin/mbupgrade -c /etc/opt/membase/1.6* -a yes
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO - src = multiDB@/opt/membase/1.6.1/data/ns_1/default
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO - dest = multiMTVBDB@/opt/membase/1.6.1/data/ns_1/default-data/default
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO - Each . represents 10000 items moved.
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,170] - [remote_util] [47264785109232] - INFO - Moved 850 items.
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO - Source version from "/opt/membase/1.6.1/data/ns_1/default" is 0
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO - Automatic mode: running without interactive questions or confirmations.
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO - Upgrading your Membase Server to basestar-351-g7f6cf4d.
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO - The upgrade process might take awhile.
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO - Analysing...
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO - Target node: ns_1@127.0.0.1
[2011-05-30 11:03:25,171] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO - Membase should not be running.
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO - Please use: /etc/init.d/membase-server stop
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO - Database dir: /opt/membase/1.6.1/data/ns_1
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO - Buckets to upgrade: default
[2011-05-30 11:03:25,172] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO - Checking disk space available for buckets in directory:
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO - /opt/membase/1.6.1/data/ns_1
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO - Free disk bucket space wanted: 1433600.0
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO - Free disk bucket space available: 7786844160
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO - Free disk space factor: 2.0
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO - Ok.
[2011-05-30 11:03:25,173] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO - Analysis complete.
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO - Copying /etc/opt/membase/1.6.1/ns_1/config.dat
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO - cp /etc/opt/membase/1.6.1/ns_1/config.dat /opt/membase/var/lib/membase/config/config.dat
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO - Ensuring bucket data directories.
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO - Ensuring bucket data directory: /opt/membase/1.6.1/data/ns_1/default-data
[2011-05-30 11:03:25,174] - [remote_util] [47264785109232] - INFO - mkdir -p /opt/membase/1.6.1/data/ns_1/default-data
[2011-05-30 11:03:25,175] - [remote_util] [47264785109232] - INFO - Ensuring dbdir owner/group: /opt/membase/1.6.1/data/ns_1
[2011-05-30 11:03:25,175] - [remote_util] [47264785109232] - INFO - chown -R membase:membase /opt/membase/1.6.1/data/ns_1
[2011-05-30 11:03:25,175] - [remote_util] [47264785109232] - INFO - Upgrading buckets.
[2011-05-30 11:03:25,175] - [remote_util] [47264785109232] - INFO - Upgrading bucket: default
[2011-05-30 11:03:25,176] - [remote_util] [47264785109232] - INFO - /opt/membase/bin/mbdbupgrade /opt/membase/1.6.1/data/ns_1/default /opt/membase/1.6.1/data/ns_1/default-data
[2011-05-30 11:03:25,176] - [remote_util] [47264785109232] - INFO - mbdbupgrade pid: 14228
[2011-05-30 11:03:25,176] - [remote_util] [47264785109232] - INFO - mbdbupgrade err: 0
[2011-05-30 11:03:25,176] - [remote_util] [47264785109232] - INFO - Ensuring bucket owner/group: /opt/membase/1.6.1/data/ns_1/default-data
[2011-05-30 11:03:25,176] - [remote_util] [47264785109232] - INFO - chown -R membase:membase /opt/membase/1.6.1/data/ns_1/default-data
[2011-05-30 11:03:25,177] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,177] - [remote_util] [47264785109232] - INFO - Done.
[2011-05-30 11:03:25,177] - [remote_util] [47264785109232] - INFO - Starting Membase server[ OK ]
[2011-05-30 11:03:25,177] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,177] - [remote_util] [47264785109232] - INFO - You have successfully installed Membase Server.
[2011-05-30 11:03:25,178] - [remote_util] [47264785109232] - INFO - Please browse to http://localhost.localdomain:8091/ to configure your server.
[2011-05-30 11:03:25,178] - [remote_util] [47264785109232] - INFO - Please refer to http://couchbase.com for additional resources.
[2011-05-30 11:03:25,178] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,178] - [remote_util] [47264785109232] - INFO - Please note that you have to update your firewall configuration to
[2011-05-30 11:03:25,178] - [remote_util] [47264785109232] - INFO - allow connections to the following ports: 11211, 11210, 4369, 8091
[2011-05-30 11:03:25,179] - [remote_util] [47264785109232] - INFO - and from 21100 to 21299.
[2011-05-30 11:03:25,179] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,179] - [remote_util] [47264785109232] - INFO - By using this software you agree to the End User License Agreement.
[2011-05-30 11:03:25,179] - [remote_util] [47264785109232] - INFO - See /opt/membase/LICENSE.txt.
[2011-05-30 11:03:25,179] - [remote_util] [47264785109232] - INFO -
[2011-05-30 11:03:25,180] - [remote_util] [47264785109232] - INFO - Stopping Membase server[FAILED]
[2011-05-30 11:03:25,285] - [rest_client] [47264785109232] - INFO - settings/web params : username=Ad

 Comments   
Comment by Farshid Ghods [ 30/May/11 ]
i will run this test manually and confirm the bug later.
Comment by Farshid Ghods [ 30/May/11 ]
the test case inserts 850 items into the default bucket and then waits for 2 minutes for the data to be persisted in the disk
then we run rpm -U and check whether those keys exists in the database or not

rpm -U restores the default bucket but does not restore the data back to the bucket...
Comment by Steve Yen [ 31/May/11 ]
Reopening this to focus on the versions before 1.6.4.
Comment by Steve Yen [ 31/May/11 ]
I tested with 1.6.1, and upgrade to basestar failed apparently due to ns_server configuration issues. (This is a different failure than the 1.6.4 upgrade failure.) diag attached.

One interesting diag entry...

ns_1@127.0.0.1:<0.270.0>:ns_memcached:506: Unable to get config for bucket "default": {error,
                                                                                       {badmatch,
                                                                                        {error,
                                                                                         read_file_info}},
                                                                                       [{ns_bucket,
                                                                                         config_string,
                                                                                         1},
                                                                                        {ns_memcached,
                                                                                         ensure_bucket,
                                                                                         2},
                                                                                        {ns_memcached,
                                                                                         init,
                                                                                         1},
                                                                                        {gen_server,
                                                                                         init_it,
                                                                                         6},
                                                                                        {proc_lib,
                                                                                         init_p_do_apply,
                                                                                         3}]}
Comment by Steve Yen [ 31/May/11 ]
The workaround for this is after upgrading, to manually make a symlink...

    /etc/init.d/membase-server stop
    mkdir -p /opt/membase/1.6.1
    ln -s /var/opt/membase/1.6.1/data /opt/membase/1.6.1/data
    /etc/init.d/membase-server start
Comment by Steve Yen [ 31/May/11 ]
btw, if the user was using a custom data directory (aka, "dbdir") the upgrade works fine.
Comment by Steve Yen [ 17/Jun/11 ]
Users should use the documented workaround.
Generated at Wed Apr 16 21:08:36 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.