Very slow replication rate( less than 100 on average) with 2 unidirectional replications between 2 clusters.

- Setup two 3 node clusters.
- Load 3M items on bucket1, cluster1 and 3M on bucket2, cluster2 [No expires on any load]
Example
-nohup lib/perf_engines/mcsoda.py localhost:23201 vbuckets=1024 doc-gen=0 doc-cache=0 ratio-creates=1 ratio-sets=1 min-value-size=256 max-items=1000000 exit-after-creates=1 prefix=a_one
-nohup lib/perf_engines/mcsoda.py localhost:23202 vbuckets=1024 doc-gen=0 doc-cache=0 ratio-creates=1 ratio-sets=1 min-value-size=256 max-items=1000000 exit-after-creates=1 prefix=a_two
-nohup lib/perf_engines/mcsoda.py localhost:23203 vbuckets=1024 doc-gen=0 doc-cache=0 ratio-creates=1 ratio-sets=1 min-value-size=256 max-items=1000000 exit-after-creates=1 prefix=a_three

Start unidirectional from cluster1 bucket1 to cluster2 bucket1
Start unidirectional from cluster2 bucket2 to cluster1 bucket2

Keep load running on cluster1 -new load with deletes/expires
lib/perf_engines/mcsoda.py localhost:23203 vbuckets=1024 doc-gen=0 doc-cache=0 ratio-creates=1 ratio-sets=1 ratio-deletes=0.02 ratio-expirations=0.05 expirations=1200 min-value-size=256 max-items=1000000 exit-after-creates=1 prefix=a_four&


Observing very slow replication rate on cluster1 and close to 100 percent CPU used.

Replication rate on cluster 1 is between 10 - 115
Replication rate on cluster2 - is between 1-2k


Adding screenshots from cluster1.