[MB-4295] Upgrade from 1.6.5.4 ->1.7.0 -> 1.7.1.1 is not supported Created: 20/Sep/11  Updated: 23/Jul/12  Resolved: 28/Dec/11

Status: Closed
Project: Couchbase Server
Component/s: installer
Affects Version/s: 1.7.1.1, 1.7.2
Fix Version/s: 1.7.2
Security Level: Public

Type: Bug Priority: Major
Reporter: Karan Kumar (Inactive) Assignee: Karan Kumar (Inactive)
Resolution: Won't Fix Votes: 0
Labels: 1.7.1-release-notes
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
ERROR MESSAGES:-
[2011-09-20 10:17:49,023] - [remote_util] [47550853327296] - ERROR - Would overwrite the following files:
[2011-09-20 10:17:49,023] - [remote_util] [47550853327296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default
[2011-09-20 10:17:49,024] - [remote_util] [47550853327296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-0.mb
[2011-09-20 10:17:49,024] - [remote_util] [47550853327296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-3.mb
[2011-09-20 10:17:49,024] - [remote_util] [47550853327296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-2.mb
[2011-09-20 10:17:49,024] - [remote_util] [47550853327296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-1.mb
[2011-09-20 10:17:49,025] - [remote_util] [47550853327296] - ERROR - ERROR: upgrade failed for bucket: default
[2011-09-20 10:17:49,025] - [remote_util] [47550853327296] - ERROR - dpkg: error processing membase-server (--install):
[2011-09-20 10:17:49,025] - [remote_util] [47550853327296] - ERROR - subprocess installed post-installation script returned error exit status 1
[2011-09-20 10:17:49,025] - [remote_util] [47550853327296] - ERROR - Errors were encountered while processing:
[2011-09-20 10:17:49,026] - [remote_util] [47550853327296] - ERROR - membase-server
[2011-09-20 10:17:49,026] - [remote_util] [47550853327296] - INFO - (Reading database ... 53283 files and directories currently installed.)

[2011-09-20 10:17:49,037] - [remote_util] [47550853327296] - INFO - /opt/membase/bin/mbdbupgrade /var/opt/membase/1.6.5.4/data/ns_1/default /var/opt/membase/1.6.5.4/data/ns_1/default-data
[2011-09-20 10:17:49,037] - [remote_util] [47550853327296] - INFO - mbdbupgrade pid: 22871
[2011-09-20 10:17:49,037] - [remote_util] [47550853327296] - INFO - mbdbupgrade err: 256
[2011-09-20 10:17:49,037] - [remote_util] [47550853327296] - INFO - Processing triggers for ureadahead ...

 Comments   
Comment by Farshid Ghods (Inactive) [ 20/Sep/11 ]
seems like a path issue
Comment by Bin Cui [ 20/Sep/11 ]
On windows, we extract db path from configuration file. We might need to do similar thing on ubuntu.
Comment by Steve Yen [ 21/Sep/11 ]
Even on the first part, installing 1.6.5.4, shutting down, then upgrading from 1.6.5.4 to 1.7.1.1, the installer reports an onminous error. (After starting up, however, all my data is there in the 1.7.1.1 server.)


root@ubuntu1004-64:~# dpkg -i membase-server-community_x86_64_1.7.1.1.deb
(Reading database ... 56904 files and directories currently installed.)
Preparing to replace membase-server 1.6.5.4 (using membase-server-community_x86_64_1.7.1.1.deb) ...
 * Stopped Membase server
Upgrading previous membase ... (1.6.5.4)
Stopping previous membase ... (1.6.5.4)
 * Stopped Membase server
Saving previous membase config.dat ...
Cleaning symlinks ...
Unpacking replacement membase-server ...
dpkg: warning: unable to delete old directory '/etc/opt/membase/1.6.5.4/ns_1': Directory not empty
dpkg: warning: unable to delete old directory '/etc/opt/membase/1.6.5.4': Directory not empty
dpkg: warning: unable to delete old directory '/etc/opt/membase': Directory not empty
dpkg: warning: unable to delete old directory '/etc/opt': Directory not empty
dpkg: warning: unable to delete old directory '/var/opt/membase/1.6.5.4/data': Directory not empty
dpkg: warning: unable to delete old directory '/var/opt/membase/1.6.5.4/logs': Directory not empty
dpkg: warning: unable to delete old directory '/var/opt/membase/1.6.5.4/mnesia': Directory not empty
dpkg: warning: unable to delete old directory '/var/opt/membase/1.6.5.4': Directory not empty
dpkg: warning: unable to delete old directory '/var/opt/membase': Directory not empty
Setting up membase-server (1.7.1.1) ...
Upgrading from 1.6.5.4, directory /etc/opt/membase/1.6.5.4 ...
  /opt/membase/bin/mbupgrade -c /etc/opt/membase/1.6.5.4 -a yes
Automatic mode: running without interactive questions or confirmations.
Upgrading your Membase Server to 1.7.1.1.
The upgrade process might take awhile.
Analysing...
Previous config.dat file is /etc/opt/membase/1.6.5.4/ns_1/config.dat.debsave
Target node: ns_1@127.0.0.1

Membase should not be running.
  Please use: /etc/init.d/membase-server stop

Database dir: /var/opt/membase/1.6.5.4/data/ns_1

Buckets to upgrade: default

Checking disk space available for buckets in directory:
  /var/opt/membase/1.6.5.4/data/ns_1
  Free disk bucket space wanted: 10240.0
  Free disk bucket space available: 11378458624
  Free disk space factor: 2.0
  Ok.

Analysis complete.

Copying /etc/opt/membase/1.6.5.4/ns_1/config.dat.debsave
    cp /etc/opt/membase/1.6.5.4/ns_1/config.dat.debsave /opt/membase/var/lib/membase/config/config.dat
Copying /opt/membase/var/lib/membase/ip
    cp /opt/membase/var/lib/membase/ip /opt/membase/var/lib/membase/ip
Ensuring bucket data directories.
Ensuring bucket data directory: /var/opt/membase/1.6.5.4/data/ns_1/default-data
    mkdir -p /var/opt/membase/1.6.5.4/data/ns_1/default-data
Ensuring dbdir owner/group: /var/opt/membase/1.6.5.4/data/ns_1
    chown -R membase:membase /var/opt/membase/1.6.5.4/data/ns_1
Upgrading buckets.
Upgrading bucket: default
    /opt/membase/bin/mbdbupgrade /var/opt/membase/1.6.5.4/data/ns_1/default /var/opt/membase/1.6.5.4/data/ns_1/default-data
mbdbupgrade pid: 23250
Would overwrite the following files:
/var/opt/membase/1.6.5.4/data/ns_1/default-data/default
/var/opt/membase/1.6.5.4/data/ns_1/default-data/default-0.mb
/var/opt/membase/1.6.5.4/data/ns_1/default-data/default-3.mb
/var/opt/membase/1.6.5.4/data/ns_1/default-data/default-2.mb
/var/opt/membase/1.6.5.4/data/ns_1/default-data/default-1.mb
mbdbupgrade err: 256
ERROR: upgrade failed for bucket: default
dpkg: error processing membase-server (--install):
 subprocess installed post-installation script returned error exit status 1
Processing triggers for ureadahead ...
Errors were encountered while processing:
 membase-server
root@ubuntu1004-64:~#
Comment by Steve Yen [ 21/Sep/11 ]
Ignore that last comment - there was an old 1.6.5.3.1 on that box that wasn't fully cleaned.
Comment by Steve Yen [ 21/Sep/11 ]
The workaround at the moment is after upgrading to 1.7.1.1 to cleanup the data directory by moving or removing the old 1.6.5.x database files out of the way.

For example...

# mkdir -p /tmp/somplace-else
# mv /var/opt/membase/1.6.5.4/data/ns_1/default /tmp/somplace-else
# mv /var/opt/membase/1.6.5.4/data/ns_1/default-shm /tmp/somplace-else
# mv /var/opt/membase/1.6.5.4/data/ns_1/default-wal /tmp/somplace-else
# mv /var/opt/membase/1.6.5.4/data/ns_1/default-*.mb* /tmp/somplace-else

After you move them out of the way, your data directory should look like...

# ls -al /var/opt/membase/1.6.5.4/data/ns_1
total 20
drwxr-xr-x 3 membase membase 4096 2011-09-21 15:28 .
drwxr-xr-x 3 membase membase 4096 2011-09-21 15:25 ..
drwxr-xr-x 2 membase membase 4096 2011-09-21 15:27 default-data
-rw-r--r-- 1 membase membase 23 2011-09-21 15:25 isasl.pw
-rw-r--r-- 1 membase membase 323 2011-09-21 15:25 ns_log

After that, the upgrade to 1.7.2 should proceed ok...

# dpkg -i membase-server-community_x86_64_1.7.2r-9-gd41b35f.deb

Comment by Steve Yen [ 21/Sep/11 ]
http://review.couchbase.org/9709

This fix will affect all upgrades, so retesting the full upgrade matrix is required (1.6.x to 1.7.2; 1.7.1.x to 1.7.2; 1.6.x to 1.7.1 to 1.7.2; all platforms; clustered and single-node)
Comment by Karan Kumar (Inactive) [ 22/Sep/11 ]
1.6.5.4->1.7.0->1.7.1.1 (failed)
Comment by Karan Kumar (Inactive) [ 22/Sep/11 ]
Install 1654
Upgrade to 170

dpkg -i /tmp/membase-server-enterprise_x86_64_1.7.1.1.deb

[2011-09-22 10:11:46,820] - [remote_util] [47760455887296] - ERROR - Would overwrite the following files:
[2011-09-22 10:11:46,820] - [remote_util] [47760455887296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default
[2011-09-22 10:11:46,820] - [remote_util] [47760455887296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-0.mb
[2011-09-22 10:11:46,821] - [remote_util] [47760455887296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-3.mb
[2011-09-22 10:11:46,821] - [remote_util] [47760455887296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-2.mb
[2011-09-22 10:11:46,821] - [remote_util] [47760455887296] - ERROR - /var/opt/membase/1.6.5.4/data/ns_1/default-data/default-1.mb
[2011-09-22 10:11:46,821] - [remote_util] [47760455887296] - ERROR - ERROR: upgrade failed for bucket: default
[2011-09-22 10:11:46,821] - [remote_util] [47760455887296] - ERROR - dpkg: error processing membase-server (--install):
[2011-09-22 10:11:46,822] - [remote_util] [47760455887296] - ERROR - subprocess installed post-installation script returned error exit status 1
[2011-09-22 10:11:46,822] - [remote_util] [47760455887296] - ERROR - Errors were encountered while processing:
[2011-09-22 10:11:46,822] - [remote_util] [47760455887296] - ERROR - membase-server
[2011-09-22 10:11:46,822] - [remote_util] [47760455887296] - INFO - (Reading database ... 53273 files and directories currently installed.)
[2011-09-22 10:11:46,822] - [remote_util] [47760455887296] - INFO - Preparing to replace membase-server 1.7.0 (using .../membase-server-enterprise_x86_64_1.7.1.1.deb) ...
[2011-09-22 10:11:46,823] - [remote_util] [47760455887296] - INFO - * Stopped Membase server
[2011-09-22 10:11:46,823] - [remote_util] [47760455887296] - INFO - Upgrading previous membase ... (1.7.0)
[2011-09-22 10:11:46,823] - [remote_util] [47760455887296] - INFO - Stopping previous membase ... (1.7.0)
[2011-09-22 10:11:46,823] - [remote_util] [47760455887296] - INFO - * Stopped Membase server
[2011-09-22 10:11:46,823] - [remote_util] [47760455887296] - INFO - Saving previous membase config.dat ...
[2011-09-22 10:11:46,824] - [remote_util] [47760455887296] - INFO - Cleaning symlinks ...
[2011-09-22 10:11:46,824] - [remote_util] [47760455887296] - INFO - Unpacking replacement membase-server ...
[2011-09-22 10:11:46,824] - [remote_util] [47760455887296] - INFO - Setting up membase-server (1.7.1.1) ...
[2011-09-22 10:11:46,824] - [remote_util] [47760455887296] - INFO - Upgrading from 1.7.0, directory /opt/membase/var/lib/membase/config ...
[2011-09-22 10:11:46,824] - [remote_util] [47760455887296] - INFO - /opt/membase/bin/mbupgrade -c /opt/membase/var/lib/membase/config -a yes
[2011-09-22 10:11:46,825] - [remote_util] [47760455887296] - INFO - Automatic mode: running without interactive questions or confirmations.
[2011-09-22 10:11:46,825] - [remote_util] [47760455887296] - INFO - Upgrading your Membase Server to 1.7.1.1.
[2011-09-22 10:11:46,825] - [remote_util] [47760455887296] - INFO - The upgrade process might take awhile.
[2011-09-22 10:11:46,825] - [remote_util] [47760455887296] - INFO - Analysing...
[2011-09-22 10:11:46,826] - [remote_util] [47760455887296] - INFO - Previous config.dat file is /opt/membase/var/lib/membase/config/config.dat.debsave
[2011-09-22 10:11:46,826] - [remote_util] [47760455887296] - INFO - Target node: ns_1@10.1.4.82
[2011-09-22 10:11:46,826] - [remote_util] [47760455887296] - INFO -
[2011-09-22 10:11:46,826] - [remote_util] [47760455887296] - INFO - Membase should not be running.
[2011-09-22 10:11:46,826] - [remote_util] [47760455887296] - INFO - Please use: /etc/init.d/membase-server stop
[2011-09-22 10:11:46,827] - [remote_util] [47760455887296] - INFO -
[2011-09-22 10:11:46,827] - [remote_util] [47760455887296] - INFO - Database dir: /var/opt/membase/1.6.5.4/data/ns_1
[2011-09-22 10:11:46,827] - [remote_util] [47760455887296] - INFO -
[2011-09-22 10:11:46,827] - [remote_util] [47760455887296] - INFO - Buckets to upgrade: default
[2011-09-22 10:11:46,827] - [remote_util] [47760455887296] - INFO -
[2011-09-22 10:11:46,828] - [remote_util] [47760455887296] - INFO - Checking disk space available for buckets in directory:
[2011-09-22 10:11:46,828] - [remote_util] [47760455887296] - INFO - /var/opt/membase/1.6.5.4/data/ns_1
[2011-09-22 10:11:46,828] - [remote_util] [47760455887296] - INFO - Free disk bucket space wanted: 36515840.0
[2011-09-22 10:11:46,828] - [remote_util] [47760455887296] - INFO - Free disk bucket space available: 11640205312
[2011-09-22 10:11:46,828] - [remote_util] [47760455887296] - INFO - Free disk space factor: 2.0
[2011-09-22 10:11:46,829] - [remote_util] [47760455887296] - INFO - Ok.
[2011-09-22 10:11:46,829] - [remote_util] [47760455887296] - INFO -
[2011-09-22 10:11:46,829] - [remote_util] [47760455887296] - INFO - Analysis complete.
[2011-09-22 10:11:46,829] - [remote_util] [47760455887296] - INFO -
[2011-09-22 10:11:46,829] - [remote_util] [47760455887296] - INFO - Copying /opt/membase/var/lib/membase/config/config.dat.debsave
[2011-09-22 10:11:46,830] - [remote_util] [47760455887296] - INFO - cp /opt/membase/var/lib/membase/config/config.dat.debsave /opt/membase/var/lib/membase/config/config.dat
[2011-09-22 10:11:46,831] - [remote_util] [47760455887296] - INFO - mkdir -p /var/opt/membase/1.6.5.4/data/ns_1/default-data
[2011-09-22 10:11:46,831] - [remote_util] [47760455887296] - INFO - Ensuring dbdir owner/group: /var/opt/membase/1.6.5.4/data/ns_1
[2011-09-22 10:11:46,831] - [remote_util] [47760455887296] - INFO - chown -R membase:membase /var/opt/membase/1.6.5.4/data/ns_1
[2011-09-22 10:11:46,831] - [remote_util] [47760455887296] - INFO - Upgrading buckets.
[2011-09-22 10:11:46,831] - [remote_util] [47760455887296] - INFO - Upgrading bucket: default
[2011-09-22 10:11:46,832] - [remote_util] [47760455887296] - INFO - /opt/membase/bin/mbdbupgrade /var/opt/membase/1.6.5.4/data/ns_1/default /var/opt/membase/1.6.5.4/data/ns_1/default-data
[2011-09-22 10:11:46,832] - [remote_util] [47760455887296] - INFO - mbdbupgrade pid: 23247
[2011-09-22 10:11:46,832] - [remote_util] [47760455887296] - INFO - mbdbupgrade err: 256
Comment by Steve Yen [ 22/Sep/11 ]
Double-checking, as 1.7.1.1 shipped already.

Did you mean instead that the following upgrade pathway failed? 1.6.5.4->1.7.0->1.7.1.1->1.7.2r-LATEST
Comment by Steve Yen [ 22/Sep/11 ]
Hi,
I just talked to Karan, and reconfirmed that he found that 1.7.1.1 has the same upgrade bug as 1.7.2r-9. That is, the following does not work: 1.6.5.4->1.7.0->1.7.1.1

However, he has confirmed that the following does work: 1.6.5.4->1.7.1.1->1.7.2r-LATEST

Please reopen this bug if the following (or other cases against 1.7.2r-LATEST doesn't work): 1.6.5.4->1.7.x->1.7.2r-LATEST
Comment by Karan Kumar (Inactive) [ 22/Sep/11 ]
For the sake of completeness 1654 -> 170 -> 172 works fine on Ubuntu.
Comment by Karan Kumar (Inactive) [ 27/Dec/11 ]
1.6.5.4->1.7.0->1.7.1.1 NOT SUpported.
Generated at Fri Sep 19 00:56:43 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.