Couchbase 3.0.2 XDCR not replication with some keys after upgrade

Dear Sir,

We set up 3 Couchbase servers with all bi-directional XDCR and upgraded them from CouchBase 2.0.1 CE to 3.0.2 EE directly by installing Windows amdx64 wizard in 10 days ago.

For now, there are still few documents are not sync to each other ( differences from 5 to 7 documents), the Items Count in their buckets are all different. Here is my case:

Items Count in SJ DC (SJDC) Server in MyBucket: 10007
Items Count in Asia DC (ADC) Server in MyBucket: 10000
Items Count in EMEA DC (EDC) Server in MyBucket: 10004

XDCR are set up as :

SJDC server <-------bi-directional XDCR ----> ADC
ADC server <-------bi-directional XDCR ----> EDC
EDC server <-------bi-directional XDCR ----> SJDC

Offline upgrade was used according to following steps:

  1. Stop EDC Couchbase services in Services (services.msc)
  2. Install couchbase-server-enterprise_3.0.2-windows_amd64.exe in EDC
  3. Stop ADC Couchbase services in Services (services.msc)
  4. Install couchbase-server-enterprise_3.0.2-windows_amd64.exe in ADC
  5. Stop SJDC Couchbase services in Services (services.msc)
  6. Install couchbase-server-enterprise_3.0.2-windows_amd64.exe in SJDC

Before the upgrade, there were some XDCR errors on ADC–>SJDC and EDC–>ADC, so next steps we did:

7 . Delete the XDCR ADC–> SJDC and recreate XDCR ADC–>SJDC
8. Delete XDCR ADC–>EDC and rereate XDCR ADC–>EDC
9. Delete XDCR EDC–>ADC and recreate XDCR EDC–>ADC
10. Delete XDCR EDC–>SJDC and recreate XDCR EDC–>SJDC

Outbound complete percentage are all 100 now and there are no errors on XDCR.

Please advise if there is any idea to find out the differences among the buckets and how to fix that without re-initializing the XDCR again.

Any idea is appreciated, thanks in advance.

Hi all,

You will be much appreciated if you have any idea regarding the differences checkup.

Thank you.

Best Regards,
David

Dear Sir,

I would like to comment the protocol used:

SJDC -> ADC uses version 1
SJDC --> EDC uses version 1
ADC --> SJDC uses version 2
ADC --> EDC uses version 1
EDC --> SJDC uses version 2
EDC --> ADC uses version 1

Is hybrid protocols possible to cause the differences?

Best Regards,
David