understanding cache evictions

Corey,
Your understanding of evictions is correct. When a new item is added and there is no space for it, an older item must be thrown away. One key thing to understand is that expiration and eviction have nothing to do with one another.

When an item expires, nothing actually happens. It is only upon the next access of that item that the server will notice it has expired and remove it from memory. There is also no guarantee that already expired items will be evicted first, though it practicality they are usually old enough that they tend to be among the first to get evicted (but nothing in the server controls this).

Now, onto the nuances of evictions. Within memcached there is a slab allocator which handles memory allocation for items. Basically, the whole memory space is broken up into 1mb pages which are then broken up into slabs of varying sizes. There are configuration options to control this, but that's the default mechanism. Each size is called a "slab class" and has its own eviction logic which means that if one slab class fills up, you can't reclaim memory from other slab classes. Depending on the variation in size of the objects that you are putting in, this could become more or less of a problem for you. Can you telnet to your servers and run 'stats' and 'stats slabs' and paste the output? That will let me understand exactly where memory is being taken up in your server.

Another key point to understand is that evictions are not always a bad thing. With changing workloads you can end up with lots of stale data still stored in cache that just needs to take some time to be pushed out. If everything seems to be running fine (with the exception of the evictions) then you probably don't really need to take any action. Are you seeing a large number of misses as well? High misses and high evictions can usually be correlated to mean that the application is expecting data to be there when it's not, and that would be a problem. If the application is receiving the data that it's asking for, your cache is working correctly.

Make sense?

Perry

makes sense.
thanks again Perry.

I will post my stats and slab stats here if I think there is a problem.

-Corey

Perry,
... I'm thinking about this some more.

I also just read another thread which discusses the same issue I am seeing:
[url]http://forums.northscale.com/showthread.php?114-Items-randomly-evicted[/url]

... it states: "The contract between a memcached client and the server is that the server may evict any item at any time for any reason."

My use case is storing web session data also. Is it possible to disable eviction for non-expired items? evicting active sessions would be problematic. (I have downtime every night and can flush the cache at that point).

-Corey

Corey,
Unfortunately this type of use case is not available in memcached (ours or the open source). Your best option would be to allow for enough memory to store all the sessions during the day (that you could then flush every night). Obviously not the most elegant option...

Also, if you are using one memcached pool for all of your application data, it might make sense to create a separate cluster for each dataset. That way your "expireable" data won't fight for room with your session data.

Let me also make a shameless plug for an upcoming NorthScale product release. Membase is going to be THE best KV-database available. It will be completely compatible with memcached which means that you already know how to use it! Take a look here for some very high-level information: [url]http://northscale.com/products/membase_server.html[/url]. We will be entering public beta in a few weeks.

Membase will definitely be the recommended technology for storing sessions since it will be a database and not a cache. That way you won't ever lose the sessions and it can be scaled up and down as need be.

Hope that helps Corey.

Perry

> Your best option would be to allow for enough memory
> to store all the sessions during the day (that you could
> then flush every night).

That is essentially what I am asking. From my reading, it sounds like that is not possible if "the server may evict any item at any time for any reason."

just to clarify...
non-expired items may get evicted (even though there is still available memory in the overall cache bucket).

is that a true statement?

also...
It seems that in the open source version, you can change this eviction behavior by setting the -M command-line option when starting memcached. This option forces an error to be returned when the memory has been exhausted, instead of automatically evicting older data.

Is that option available in the NorthScale version as well? I can't find any references to it.

-Corey

Corey,
The notion of "...may evict any item at any time for any reason" is a bit of an exaggeration. An eviction will only take place if a new item is being set and there is no space for that item. Now, "no space" can get slightly confusing as per my comments about slab classes above.

If you have enough memory to store the items, you will not experience evictions.

For the -M flag, it may solve your problems around items being evicted, but I also think it may introduce other problems for you. How will your app respond if it can't set a new session? What if you receive a flood of users and all of their sessions can't be cached? IMHO, I would rather the cache evict old sessions to make room for new ones, but that's a certainly a choice/design-decision for you to make.

While the NorthScale server will support the -M flag, there isn't currently a good mechanism for getting it in there. If that's something you'd still like to pursue (and honestly I wouldn't recommend it) I can put together some specific instructions.

-Perry