recent issues with couchbase; eviction of php sessions and expired keys

You definitely don't want to set min_data_age to that high of a value. When Couchbase grabs your items out of memory to persist them it compares how long they have been in memory (and not persisted) to the min_data_age. If they have been sitting in memory for a smaller time than min_data_age then it holds off persisting them and I don't think that that is what you want and it will likely make your problems much worst.

The issue you are likely having is that your items aren't expiring and this is likely an issue in your application code. Check that you are actually sending an expiration time with each request. Also, the expiry pager runs once per hour to clean up expired items. It can be configured to run at different intervals, but I sense that the problem is that the items in your database are set to never expire.

If you feel that your application code is correct then can you post a code snippet here?

i'm checking with our developers on how the application is caching it's information, but i do believe that there are expires times set there.

but, my main concern right now is the bucket where php is storing its sessions. from what i've read thus far, there's something with the php session handler that allows it to garbage collect from file based sessions, but doesn't handle cleanup well when using the memcached extension. we're using the PECL memcache extension. here's some relvant settings we're using in php though,

memcache:
memcache support => enabled
memcache.allow_failover => 1 => 1
memcache.chunk_size => 8192 => 8192
memcache.compress_threshold => 20000 => 20000
memcache.default_port => 11211 => 11211
memcache.hash_function => crc32 => crc32
memcache.hash_strategy => consistent => consistent
memcache.lock_timeout => 15 => 15
memcache.max_failover_attempts => 20 => 20
memcache.protocol => ascii => ascii
memcache.redundancy => 1 => 1
memcache.session_redundancy => 2 => 2
Registered save handlers => files user memcache

session:
session.auto_start => Off => Off
session.bug_compat_42 => Off => Off
session.bug_compat_warn => Off => Off
session.cache_expire => 180 => 180
session.cache_limiter => nocache => nocache
session.cookie_domain => example.com => example.com
session.cookie_httponly => Off => Off
session.cookie_lifetime => 0 => 0
session.cookie_path => / => /
session.cookie_secure => Off => Off
session.entropy_file => no value => no value
session.entropy_length => 0 => 0
session.gc_divisor => 1000 => 1000
session.gc_maxlifetime => 21600 => 21600
session.gc_probability => 1 => 1
session.hash_bits_per_character => 5 => 5
session.hash_function => 0 => 0
session.name => PHPSESSID => PHPSESSID
session.referer_check => no value => no value
session.save_handler => memcache => memcache
session.save_path => tcp://localhost:11211 => tcp://localhost:11211
session.serialize_handler => php => php
session.use_cookies => On => On
session.use_only_cookies => On => On
session.use_trans_sid => 0 => 0

where is the expiry pager configured? are there some ways to configure that to remove the old un-needed sessions? looks like there may be a way to use tap.py to script an explicit deletion (or update of expiry keys), but that really seems like overkill.

thanks,

-rtw

You can set the interval for the expiry pager to run by doing the following:

./cbflushctl localhost:11210 set exp_pager_stime 60

This would set the expiry pager to run every 60 seconds. If your items don't have an expiry time then changing the expiry pager run interval is not likely to do much. I suspect that the issue is either in the sdk or in your application. I will email our PHP developer and have him take a look at this.

Keep in mind that cbflushctl would need to be run once per node in the cluster, so running that on one system on "localhost" will just change the configuration for that one node. Tuning these variables would need to be redone after each restart as well-- we have an open enhancement request to have the cluster persist and reapply these as needed.

yea, i'm not sure changing the setting would have any effect because there aren't expire times on the sessions.

i've done a bit more digging and found that our application cache (in a separate bucket) sets an expire on the key itself, as well as an extra "expire" key (the key appended with _expire, done by cakephp).

i've also looked at a few sessions keys, and don't think i see any expiration set.

though, it does make some sense that php can't do garbage collection - because it can't get the expire time on the keys without getting them... but, if the expire time was set on them, then the expiry pager should pick them up, correct? any suggestions on setting expire times on items?

bump. anyone find anything?

What you set as the expiry time is totally dependent on the application. Users who use Couchbase as a session store for example typically use an expiration time of 20 minutes. I'm going to file an issue for you problem and have our PHP developer take a look at it to see if he can find anything wrong.

We verified that the Couchbase client is in fact setting expirations and working appropriately as part of http://www.couchbase.com/issues/browse/PCBC-81

The thing is, from this thread it seems to be pecl/memcached and there are some settings in the framework which we're not familiar with. Based on verification, it looks like the server is doing the right thing and I believe pecl/memcached probably is, so some digging in the framework will probably be needed.

Have you looked in there rtwheato?