[MB-7053] Expired items are not excluded from production/dev views until expiry pager runs ( which deletes the items from database permanently )

Mon, 8 Apr 2013 06:52:57 -0500

Expired items are not excluded from production/dev views. This happens only after the item is "get".

Steps to reproduce:

1. Create default bucket and create 1 production view
curl -X PUT -H 'Content-Type: application/json' 'http://Administrator:asdasd@127.0.0.1:9500/default/_design/d1' -d '{"views":{"v1":{"map":"function(doc,meta){\nemit(meta.id,doc);\n}"}}}'
{"ok":true,"id":"_design/d1"}

2. Insert 2 items from memcached client with expiry set to 5 seconds

>>> import mc_bin_client
>>> mc = mc_bin_client.MemcachedClient(port=12001)
>>> mc.set("ab", 5, 0, "val")
(3306888435, 7973028514358, '')
>>> mc.set("ab1", 5, 0, "val")
(2896330942, 7997559610975, '')

3. Query the view with stale=false to build the index
curl -X GET 'http://127.0.0.1:9500/default/_design/d1/_view/v1?stale=false'
{"total_rows":2,"rows":[
{"id":"ab","key":"ab","value":"dmFs"},
{"id":"ab1","key":"ab1","value":"dmFs"}
]
}

4. Query the view after couple of minutes and expired items are still returned
curl -X GET 'http://127.0.0.1:9500/default/_design/d1/_view/v1?stale=false'
{"total_rows":2,"rows":[
{"id":"ab","key":"ab","value":"dmFs"},
{"id":"ab1","key":"ab1","value":"dmFs"}
]
}

If memcached get is used with these items, then these are excluded from the views. Otherwise these are always returned in view results.

Diagnostic is attached.

Checked with Mike on this and ep-engine seems to be doing things correctly:

"This does look like an issue, but not in the ep-engine side. Since ep-engine might take an hour to actually remove an expired item, it should be up to the view engine to filter out any expired items too. The reaon why doing a get will cause the item to disappear from the view results is that ep-engine will actually do the deletion."

Couchbase

[MB-7053] Expired items are not excluded from production/dev views until expiry pager runs ( which deletes the items from database permanently )