Inconsistent RAM usage and get temp OOM once a month

Couchbase Server Version: Community Edition 5.0.1 build 5003.
no. of nodes: 18
Memory Used: 2.28 TB
User Data In RAM (active & replica): 627 GB
Metadata in RAM (active & replica): 233 GB

I really wonder where the remaining 1 TB data are used for?

And we also frequently got temp OOM due to RAM used exceeds high water mark. Every time it happened, we will do rebalance for the cluster and we always see “Memory Used” drop down a lot while “User Data in RAM” drive up. So I really suspect there is memory leak in the Couchbase Server.

Has anyone see this issue before? Appreciate for any insights!