Automated installation on Linux and a few other questions
Hi all,
I am new to Membase.
Here is some background on what I am after:
I intend to use it as a local storage for a cache server I wrote(which also fetches data from central servers).
currently my cache server has its own memory cache component, and a persistent storage component (based on jbosscache and mysql).
my plan is to replace the persistent storage part with a membase instance which will run locally on each of my cache nodes.
I have a few problems at the moment :
1 . Since I plan to run membase on servers which are configured automatically using puppet, I will need to automate the installation (interacting with the web console is nice for development, but in production I want to be able to configure membase automatically).
2. I want to benchmark membase performance when it stores and fetches data from the disk.
I have created a simple benchmark client that just stores many blobs of data (33kb each) in membase (using xmemcached java client). However, I did not see the size of the data files at /opt/membase/1.6.5.4/data/ grow even after storing 200,000 records of 33k each. (thats around 6.5GB of data, and I only configured membase to use 256mb).
I suspect membase decided to evict my data before it got stored, is that possible?
3. is it possible to reconfigure membase to use a different amount of memory once it's been configured?
Thanks.
Hi Perry,
There are several reasons:
1. I have some applicative logic in the cache server, specifically - if data is missing in the local cache it goes to the central databases and brings it to the local cache (my cache layer is designed to take load of the central databases by bringing data to the local node).
2. I wrote it a couple of years ago, before I was aware of membase (which in truth, has yet to prove itself as a viable alternative).
As to your answers:
1. cool. I`ll give it a shot.
2. I am using a membase bucket, the data dir shown in the web cnsole is /var/opt/membase/1.6.5.4/data/ns_1, and I don't see that its using more space as I set values.
3. cool. if I call it with the parameters already in the servers its not supposed to change anything it lose data, right?
btw: the link you pointed to has an incorrect command:
/opt/membase/bin/membase cluster-init -c ...
should be bin/cli, no?
Thanks Omry, that makes sense. I would be a bit cautious about having multiple cache layers because you need to deal with cache invalidation and potential consistency issues. I trust that you're taking that into consideration...
For the data directory, do you see ANY size increase (over 4k or so) on those files? If you're doing deletes (or expiration is happening) it will create "holes" in the sqlite files which will then be filled by new data coming in so you may not see the files increase in size. Also, if you're not adding new data, the keys will be updated "in-place" in the files and so not result in a size increase. It would be good to do some basic spot checking to ensure that data is actually getting into the files...
You're correct that changing the RAM quotas will not cause any data loss. If you are DECREASING the quota, you may cause data to be ejected to disk but it won't actually be lost. Increasing the quota will not have any effect other than allowing the process to use more RAM.
My apologies, you are correct about the path typo. In 1.7 it has changed to what I originally posted, in previous versions your path is correct.
Perry
May I ask why you need your own cache server if Membase provides both a memcached caching layer and a persistent backend?
To answer your questions:
1 - You can easily automate the configuration through our CLI (http://techzone.couchbase.com/wiki/display/membase/Membase+Command-line+...)
2 - Make sure that you are using a Membase bucket and not a memcached bucket. memcached bucket types do not have any persistence. If you're using a Membase bucket and still not seeing the disk files grow, it's possible that you changed the on-disk path to data so you should look in there. You can see what the data path is by going to the Manage Servers screen and clicking on the server to expand the information
3 - Yes, a bucket's quota can be changed simply by editing it (either through the GUI or CLI) and the per-node quota can be changed by following these instructions: http://techzone.couchbase.com/wiki/display/membase/Membase+FAQ#MembaseFA...
Hope that helps, let me know if there's anything else we can do for you.
Perry
Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!