Hmmmm… I was sure that I disabled THP at the time of installation (and reviewed the deployment guide). But I see that you are right:
[root@db2 log]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
[root@db2 log]# cat /sys/kernel/mm/transparent_hugepage/defrag
[always] madvise never
Not sure if any subsequent OS updates could have changed this… In our test environment (originally set up exactly the same way) it is marked as [never]. Ok, but I’ll have to fix that - and look at the other settings also.
I suppose I could just remove the 6.6.0 package - but not sure if I should ignore dependencies and run uninstall scripts?
We did have issues with time sync, until the middle of last week. The VMs had been set to sync. time with the host - but they shouldn’t as I control that from the VM itself. Therefore, anything in the logs about time sync. should be older than 4 August… But as I couldn’t clear the logs before sending it to you there may be some “expected” noise there… Time is running fine since then. I thought I mentioned it earlier - sorry if I forgot that.
I really appreciate your input - and am not too worried about you being out of the comfort zone here. I’m further out - as for a starter I don’t know the “good values” on Couchbase (yet)
Argh… the /etc/init.d/disable-thp is in place - but apparently the chmod... command to make it executable has never been run (at least correctly) . Thanks for spotting!
… and similarly vm.swappiness had been set to 1 on the test servers - but for some reason not in production. Not good… changed and will reboot servers soon when deploying the “readonly” change to queries.
So this is the error that yum reported on the update:
yum check
Loaded plugins: fastestmirror
couchbase-server-community-7.0.0-5302.x86_64 is a duplicate with couchbase-server-community-6.6.0-7909.x86_64
Error: check all
Apparently something crashed during the yum update - and left things “undone”
The resolution was to run yum-complete-transaction (in case someone else searches for this issue):
yum-complete-transaction
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.fysik.dtu.dk
* epel: ftp.lysator.liu.se
* extras: mirror.fysik.dtu.dk
* updates: mirror.fysik.dtu.dk
There are 2 outstanding transactions to complete. Finishing the most recent one
The remaining transaction had 2 elements left to run
Removing couchbase-server-community.x86_64 0:6.6.0-7909 - ud from the transaction
Removing couchbase-server-community.x86_64 0:7.0.0-5302 - u from the transaction
--> Running transaction check
---> Package couchbase-server-community.x86_64 0:6.6.0-7909 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================================================================================================================
Removing:
couchbase-server-community x86_64 6.6.0-7909 @couchbase-server-community 516 M
Transaction Summary
============================================================================================================================================================================================================================================================
Remove 1 Package
Installed size: 516 M
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Erasing : couchbase-server-community-6.6.0-7909.x86_64 1/1
Verifying : couchbase-server-community-6.6.0-7909.x86_64 1/1
Removed:
couchbase-server-community.x86_64 0:6.6.0-7909
Complete!
Cleaning up completed transaction file
. Thanks for spotting!