How to change "memcached item size" for memcached vbucket type?

Hi Johan, unfortunately "tunable" isn't actually as simple as it sounds. We'd like to avoid doing that, can you explain a bit more what you need?

For changing the data files, that is the correct link. Make sure if this is an already running system (with data in it) that you shut down the service and move the files in place after making the change.

Perry

Dear Perry,
actually I use memcached at the first place and caught up with the 1MB limitation for each item to stored in memcached. Then I use membase because item size is 20MB.
The nature of application that I'm doing right now:
1. user will use browser to access my apps
2. user 1 and user 2 and user 3 and so on will have their own unique set of displayed data
3. after 30s (for instance), each browser will request new data, and my apps will return partial data (changed data only, which have 3 condition: insert new data, delete data and update data)
In short I need:
1. big enough item size to stored in the membase bucket type (sufficient for now)
2. to save the "data copy" for each user (each data saved have their expiry time, about 3 minutes)
the problem goes to no 2, data growth of the data really fast draining the HDD (because of the nature of my apps)

The conclusion are these:
1. I choose membase bucket type because 20MB item size, but.... got problem with the persistence mechanism that draining my HDD
2. IF only I use memcached bucket type, 1MB item size is not a choice at all.

The questions are:
1. can I change the 1MB for memcached bucket type (via configuration file or by any mean)?
2. if using membase bucket type, is there any way to turn off the persistence feature?

The following problem because of HDD full, here are the condition happen:
1. the "Servers Down" counter become 1
2. the "server Nodes" becoming yellow (pending)
what should I do the make the server Up again?
btw... I use membase server version 1.7.0

please excuse me for such many questions...

Thanks in advance..
Johan

No worries Johan, happy to help.

In general, you don't want to run out of disk space at all (even traditional RDBMS don't behave well when running out of disk space).

At the moment, there is no way to run a Membase bucket without persistence. I will see if I can put together some instructions for changing the memcached object size limit, but due to the nature of memcached, I don't think this is a very good idea. It will lead to much more memory fragmentation.

Also, when using "large" objects (anything over 1mb really), you're losing a bit of the value that memcached/Membase provides. We can serve data very VERY quickly, but if it takes a long time to transfer over the network, you lose a lot of that value. You would be better served by breaking up those objects into multiple smaller ones. That will also give you the advantage of being able to upgrade individual ones without having to transfer the whole object over the network and back.

As for the disk full problem, you'll have to increase the disk space available. You can either delete the DB files entirely (losing the data) or move them to a larger partition and restart the server. I would also recommend upgrading to 1.7.1.

Hope that helps, take care.

Perry

Dear Perry,
I do agree regarding 1MB limit... smallest data will deliver fastest data.
But new question pop-up.... if membase also using memcached internally (with 1MB limitation per item) how can membase handle 20MB per item?
I'm guessing that if data beyond 1MB, the persistence come in the role... somehow the data will persisted to disk as soon as possible to save the above 1MB limit.. Please correct me if I'm wrong.
Just in case if 20MB limit also got break, are there any configuration to change the size?

btw.. I'll be doing refactoring for the apps that really fast draining my HDD space..

THanks in advance
Johan

The reason memcached has the 1mb object size limit is due to its internal memory allocation (it uses a slab allocator with 1mb slabs).

Membase has done away with this memory allocation and uses a more flexible, tcmalloc based architecture.

Perry