Massive Disk Cache
Tue, 03/06/2012 - 14:25
I've been using membase cache for a while (over a year) and that's the bucket type I've gone with for all of my buckets. Today I noticed massive disk cache sizes and I'm wondering if there's any way to purge them since it can't possibly be right. I have 3 membase buckets, each with a max memory usage of 256MB. All of the buckets are using less than 100MB of memory, but it shows disk usage of: 9.32GB, 3.72GB, and 2.58GB for the 3 buckets. It must have old expired items in it. Is there a way to clean that up?
Here's my stats output if that helps:
STAT delete_misses 359 STAT ep_io_num_write 63575 STAT ep_store_max_concurrency 10 STAT rejected_conns 0 STAT connection_structures 2127 STAT ep_db_strategy multiDB STAT ep_num_eject_replicas 0 STAT limit_maxbytes 67108864 STAT decr_hits 0 STAT ep_pending_ops_max_duration 0 STAT ep_flush_duration_total 99 STAT ep_item_flush_expired 0 STAT ep_num_not_my_vbuckets 145271 STAT ep_too_young 0 STAT curr_connections 1681 STAT ep_io_write_bytes 69481055 STAT ep_total_cache_size 169772037 STAT ep_storage_age 0 STAT ep_flush_duration_highwat 5 STAT ep_flush_duration 0 STAT cas_misses 0 STAT ep_flusher_todo 0 STAT ep_pending_ops 0 STAT ep_db_cleaner_status complete STAT mem_used 83719949 STAT ep_dbshards 4 STAT ep_warmup_oom 0 STAT ep_vbucket_del 0 STAT get_misses 123229 STAT ep_num_value_ejects 0 STAT ep_queue_size 0 STAT ep_total_del_items 85 STAT bytes_read 517857311 STAT get_hits 440507 STAT decr_misses 0 STAT ep_commit_num 21337 STAT bucket_conns 263 STAT ep_num_non_resident 0 STAT ep_store_max_readwrite 1 STAT ep_tap_keepalive 0 STAT ep_oom_errors 0 STAT ep_too_old 0 STAT cmd_flush 0 STAT ep_max_txn_size 1000 STAT ep_version 1.6.2 STAT uptime 31152 STAT ep_data_age_highwat 9 STAT ep_queue_age_cap 900 STAT incr_hits 0 STAT time 1331074621 STAT ep_warmup_dups 0 STAT ep_total_persisted 63660 STAT daemon_connections 10 STAT ep_flusher_state running STAT pointer_size 64 STAT version 1.4.4_304_g7d5a132 STAT ep_max_data_size 268435456 STAT ep_tap_bg_fetched 0 STAT ep_tmp_oom_errors 0 STAT ep_commit_time_total 84 STAT ep_warmup_time 1283773055 STAT ep_item_commit_failed 0 STAT total_connections 3903864 STAT curr_items 147699 STAT ep_total_new_items 58976 STAT ep_data_age 3 STAT delete_hits 408 STAT ep_storage_type featured STAT curr_items_tot 147699 STAT ep_total_enqueued 77917 STAT ep_mem_low_wat 161061273 STAT ep_kv_size 58144181 STAT ep_vbucket_del_fail 0 STAT ep_min_data_age 0 STAT ep_io_num_read 89832 STAT ep_warmed_up 88808 STAT ep_item_flush_failed 0 STAT cas_hits 0 STAT ep_num_active_non_resident 0 STAT ep_warmup true STAT ep_dbname c:/Program Files/Membase/Server/data/ns_1/default STAT ep_num_expiry_pager_runs 8 STAT ep_commit_time 0 STAT auth_errors 0 STAT ep_store_max_readers 9 STAT ep_tap_bg_fetch_requeued 0 STAT ep_bg_fetched 0 STAT ep_storage_age_highwat 7 STAT threads 4 STAT pid 2228 STAT auth_cmds 1 STAT cas_badval 0 STAT cmd_set 181860 STAT ep_io_read_bytes 25625399 STAT cmd_get 563736 STAT ep_expired 251566 STAT conn_yields 0 STAT ep_warmup_thread complete STAT ep_flush_preempts 0 STAT ep_num_eject_failures 0 STAT bytes_written 2008200026 STAT libevent 2.0.7-rc STAT ep_num_pager_runs 0 STAT ep_mem_high_wat 201326592 STAT ep_dbinit 16 STAT incr_misses 0 STAT ep_pending_ops_total 0 STAT ep_item_begin_failed 0 STAT ep_pending_ops_max 0 STAT ep_overhead 25575768
I found that Couchbase cant get memory allocated for disk cache. So once a time you should clear cache with
echo 3 > /proc/sys/vm/drop_caches
In other way you will hit high watermark limit (if you have heavy load)