As you have correctly identified, due to its append-only nature Couchbase Server does not actually remove a document when it receives a delete operation. Instead it will create a new revision of the document which indicates that it has been ‘deleted’.
I am curious to know when the latest document (which is having only meta data) is going to delete from couchbase permanently.
As explained in our compaction documentation, this metadata is retained for three days by default after the document has been deleted, after which compaction will purge this metadata.
The reason that we retain this metadata for so long after the document has been deleted is to ensure data consistency between two clusters connected via XDCR.
Imagine a situation where you have two clusters, cluster A and cluster B, cluster A is linked bidirectionally to B via XDCR, where all documents are replicated from A->B and from B->A.
In this situation a document is deleted on cluster B but then immediately auto-compaction kicks in.
If Couchbase Server immediately removed metadata of deleted documents, then no matter what, cluster A would recreate this document on cluster B, based on its own data, this is unexpected behaviour and breaks the semantics of XDCR, creating an ‘inconsistency’.
However as we retain the metadata of deleted documents for a prolonged period time, the clusters can perform the conflict resolution based on revision number with the ‘deleted’ document on cluster B, deleting it also on cluster A.
This is an extreme example, however there are times where XDCR can take a very long time to replicate between two clusters, so it is important that we keep the metadata around long enough for this not to be a risk, even in very high latency environments.
Apologies for going into slightly more detail than you asked for, but I thought it was important to understand why Couchbase Server does not immediately remove the metadata of deleted documents, so that you can make an informed decision as to how to handle this issue.
If in your environment you are not using any XDCR and you wish to purge this metadata more often to conserve disk space, then I suggest altering the Metadata Purge Interval in the auto-compaction settings to a more suitable value for your use-case.