Configurando XDCR entre VPCs em Clusters do Amazon EKS

In modern distributed systems, the ability to replicate data between separate environments is crucial for ensuring high availability, disaster recovery, and performance optimization. Couchbase’s XDCR (Cross Data Center Replication) feature allows seamless replication of data between clusters, enabling robust data sharing across geographically or logically isolated environments.

This guide will walk you through setting up XDCR between two Couchbase clusters hosted in separate Amazon EKS (Elastic Kubernetes Service) clusters within different VPCs. We’ll dive into each step, from infrastructure setup to configuring DNS for cross-cluster communication and deploying Couchbase for real-time replication. By the end of this walkthrough, you’ll have a production-ready setup with the skills to replicate this in your environment.

Pré-requisitos

To follow this guide, ensure you have:

• AWS CLI installed and configured
• An AWS account with permissions for creating VPCs, EKS clusters, and security groups
• Familiarity with Kubernetes and tools like kubectl and Helm
• Helm installed to deploy Couchbase
• Basic knowledge of networking concepts, including CIDR blocks, routing tables, and DNS

---

Step 1: Deploy EKS clusters in separate VPCs

What are we doing?

We will create two Kubernetes clusters, Cluster1 and Cluster2, in separate VPCs using eksctl. Each cluster will operate independently and have its own CIDR block to avoid IP conflicts.

Why is this important?

This separation ensures:

1. Isolation for better security and management
2. Scalability and flexibility for handling workloads
3. Clear routing rules between clusters

Commands to create clusters

Deploy cluster1

Deploy cluster2

Expected outcome

• Cluster1 resides in VPC 10.0.0.0/16
• Cluster2 resides in VPC 10.1.0.0/16

---

Step 2: Peer the VPCs for Inter-Cluster Communication

What are we doing?

We are creating a VPC Peering connection between the two VPCs and configuring routing and security rules to enable inter-cluster communication.

Steps

2.1 Create a peering connection

• Go to the AWS Console > VPC > Peering Connections
• Click Create Peering Connection
• Select the Requester VPC (Cluster1 VPC) and Accepter VPC (Cluster2 VPC)
• Name the connection eks-peer
• Click Create Peering Connection

2.2 Accept the peering request

• Select the peering connection
• Click Actions > Accept Request

2.3 Update route tables

• For Cluster1 VPC, add a route for 10.1.0.0/16, targeting the peering connection
• For Cluster2 VPC, add a route for 10.0.0.0/16, targeting the peering connection

2.4 Modify security groups

Why is this necessary?

Security groups act as firewalls, and we must allow traffic between the clusters explicitly.

How to modify

1. Navegar para EC2 > Security Groups in the AWS Console
2. Identify the security groups associated with Cluster1 and Cluster2
3. For Cluster1’s security group:
   • Click Edit Inbound Rules
   • Add a rule:
     • Tipo: All traffic
     • Source: Security group ID of Cluster2
4. Repeat for Cluster2, allowing traffic from Cluster1’s security group

---

Step 3: Test Connectivity by Deploying NGINX in Cluster2

What are we doing?

We are deploying an NGINX pod in Cluster2 to verify that Cluster1 can communicate with it.

Why is this important?

This step ensures the networking between clusters is functional before deploying Couchbase.

Steps

3.1 Create a Namespace in Cluster1 and Cluster2

3.2 Deploy NGINX in Cluster1 and Cluster2

• Create nginx.yaml:
  
  YAML

  apiVersion: apps/v1 kind: Deployment metadata:   name: nginx spec:   replicas: 1   selector:     matchLabels:       app: nginx   template:     metadata:       labels:         app: nginx     spec:       containers:       – name: nginx         image: nginx         ports:         – containerPort: 80 — apiVersion: v1 kind: Service metadata:   name: nginx spec:   clusterIP: None   ports:   – port: 80     targetPort: 80   selector:     app: nginx

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

  apiVersion: apps/v1 gentil: Deployment meta-dados:   nome: nginx spec:   réplicas: 1   selector:     matchLabels:       aplicativo: nginx   template:     meta-dados:       labels:         aplicativo: nginx     spec:       contêineres:       – nome: nginx         image: nginx         ports:         – containerPort: 80 — apiVersion: v1 gentil: Service meta-dados:   nome: nginx spec:   clusterIP: None   ports:   – port: 80     targetPort: 80   selector:     aplicativo: nginx

3.3 Apply the YAML

3.4 Verify Connectivity from Cluster1

• Exec into the pod in Cluster1:
  
  Concha

  kubectl exec -it -n dev <pod-name> — /bin/bash

  1	kubectl exec –ele –n dev <pod–nome> — /bin/bash

3.5 Test connectivity to Cluster2

Expected outcome

A curl command will fail without DNS forwarding, highlighting the need for further DNS configuration.

---

Step 4: Configuring DNS forwarding

What are we doing?

We’ll configure DNS forwarding so that services in Cluster2 can be resolved by Cluster1. This is critical for allowing applications in Cluster1 to interact with services in Cluster2 using their DNS names.

Why is this important?

Kubernetes service discovery relies on DNS, and by default, DNS queries for one cluster’s services cannot resolve in another cluster. CoreDNS in Cluster1 must forward queries to Cluster2’s DNS resolver.

Steps

4.1 Retrieve Cluster2’s DNS Service Endpoint

• Run the following command in Cluster2 to get the DNS service endpoint:
  
  Concha

  kubectl get endpoints -n kube-system

  1	kubectl obter endpoints –n kube–system

• Look for the kube-dns ou coredns service and note its IP address. For example:
  
  —————————————- NAME       ENDPOINTS          AGE kube-dns   10.1.20.116:53     3h —————————————-

  1 2 3 4

  ———————————————————— NAME       ENDPOINTS          AGE kube–dns   10.1.20.116:53     3h ————————————————————

  

4.2 Edit the CoreDNS ConfigMap in Cluster1

• Open the CoreDNS ConfigMap for editing:
  
  Concha

  kubectl edit cm coredns -n kube-system

  1	kubectl edit cm coredns –n kube–system

   

4.3 Add the following block to the Corefile section

0

Replace 10.1.20.116 with the actual DNS endpoint IP from Cluster2.

Note: We only need to use one of the CoreDNS endpoints for this ConfigMap. The IP of CoreDNS pods rarely change but can if the Node goes down. The kube-dns ClusterIP service can be used but will require the IP and Port to be open on the EKS Nodes.

4.4 Restart CoreDNS in Cluster1

• Apply the changes by restarting CoreDNS:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  1

  

4.5 Verify DNS Forwarding

• Exec into any pod in Cluster1:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  2

• Test DNS resolution for an NGINX service in Cluster2:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  3

• You should see a response from the NGINX service
  

Expected outcome

DNS queries from Cluster1 to Cluster2 should resolve successfully.

---

Step 5: Deploying Couchbase

What are we doing?

We’ll deploy Couchbase clusters in both Kubernetes environments using Helm. Each cluster will manage its own data independently before being connected through XDCR.

Why is this important?

Couchbase clusters form the foundation of the XDCR setup, providing a robust and scalable NoSQL database platform.

Steps

5.1 Add the Couchbase Helm Chart Repository

4

5.2 Deploy Couchbase in Cluster1

• Switch to Cluster1:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  5

• Deploy Couchbase:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  6

5.3 Deploy Couchbase in Cluster2

• Switch to Cluster2:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  7

• Deploy Couchbase:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  8

5.4 Verify Deployment

• Check the Couchbase pods:
  
  Concha

  eksctl create cluster   –name cluster2   –region us-east-1   –zones us-east-1a,us-east-1b,us-east-1c   –node-type t2.medium   –nodes 2   –nodes-min 1   –nodes-max 3   –version 1.27   –vpc-cidr 10.1.0.0/16

  1 2 3 4 5 6 7 8 9 10

  eksctl criar aglomerado   —nome cluster2   —region us–east–1   —zones us–east–1a,us–east–1b,us–east–1c   —nó–tipo t2.medium   —nodes 2   —nodes–min 1   —nodes–max 3   —versão 1.27   —vpc–cidr 10.1.0.0/16

  9

  

• Ensure all pods are running.

Nota: If you encounter a deployment error, edit the CouchbaseCluster CRD to use a compatible image version:

0

Change:

1

To:

2

Expected outcome

Couchbase clusters should be running and accessible via their respective UIs.

---

Step 6: Setting Up XDCR

What are we doing?

We’ll configure XDCR to enable data replication between the two Couchbase clusters.

Why is this important?

XDCR ensures data consistency across clusters, supporting high availability and disaster recovery scenarios.

Steps

6.1 Get service name from Cluster2

• In cluster2 run the following command to retrieve the service name of one of the pods so we can port-forward to it.
  
  Concha

  kubectl create ns dev #in cluster1 kubectl create ns prod #in cluster2

  1 2	kubectl criar ns dev #in cluster1 kubectl criar ns prod #in cluster2

  3

6.2 Access the Couchbase UI for Cluster2

• Port-forward the Couchbase UI:
  
  Concha

  kubectl create ns dev #in cluster1 kubectl create ns prod #in cluster2

  1 2	kubectl criar ns dev #in cluster1 kubectl criar ns prod #in cluster2

  4

• Open a browser and navigate to:
  https://localhost:8091
  

• Log in using the credentials configured during the deployment.

6.3 View Documents in Cluster2

• In the Couchbase UI, go to Baldes
• Note that no Documents exist in the default balde

 

6.4 Access the Couchbase UI for Cluster1

• Port-forward the Couchbase UI:
  
  Concha

  kubectl create ns dev #in cluster1 kubectl create ns prod #in cluster2

  1 2	kubectl criar ns dev #in cluster1 kubectl criar ns prod #in cluster2

  5

  

• Open a browser and navigate to:
  https://localhost:8091
• Log in using the credentials configured during the deployment

6.5 Add a Remote Cluster

• In the Couchbase UI, go to XDCR > Add Remote Cluster
• Configure the remote cluster:
  • Cluster Name: Cluster2
  • IP/Hostname: <Cluster2 Service Name>.prod.svc.cluster.local
  • Username: Admin username for Cluster2
  • Password: Admin password for Cluster2
• Click Save
  

6.6 Set Up Replication

• In the Couchbase UI for Cluster1, go to XDCR > Add Replication
• Configure the replication:
  • Replicate From Bucket: Default bucket in Cluster1
  • Replicate To Bucket: Default bucket in Cluster2
  • Remote Cluster: Select Cluster2
• Click Save
  
  

6.7 Test Replication

• Add sample documents to the default bucket in Cluster1:
  • In the Couchbase UI, navigate to Buckets > Documents > Add Document
  • Give the Documento a unique ID and some data in JSON format
    
  • Verify that the documents appear in the default bucket of Cluster2:
    • Port-forward to Cluster2’s UI and log in:
      
      Concha

      kubectl create ns dev #in cluster1 kubectl create ns prod #in cluster2

      1 2	kubectl criar ns dev #in cluster1 kubectl criar ns prod #in cluster2

      6

  • Navigate to: https://localhost:8091

Expected outcome

Data added to Cluster1 should replicate to Cluster2 in real time.

---

Step 7: Cleanup

What are we doing?

We’ll cleanup our AWS environment and delete all the resources we deployed.

Why is this important?

This will prevent you from incurring unnecessary charges.

Steps

7.1 Access the AWS Console

• Go to the AWS Console > VPC > Peering Connections
• Select and delete the peering connection
• Go to the AWS Console > CloudFormation > Stacks
• Select and delete the two nodegroup stacks
• Once the two nodegroup stacks have finished deleting select and delete the cluster stacks

Expected outcome

All resources created for this tutorial are deleted from account.

---

Conclusão

Through this guide, we’ve successfully established XDCR between Couchbase clusters running in separate EKS clusters across AWS VPCs. This setup highlights the power of combining AWS networking with Kubernetes for robust, scalable solutions. With cross-cluster replication in place, your applications gain enhanced resilience, reduced latency for distributed users, and a solid disaster recovery mechanism.

By understanding and implementing the steps outlined here, you’re equipped to tackle real-world challenges involving multi-cluster setups, expanding your expertise in both cloud networking and distributed database management.

• Saiba mais sobre Couchbase Cross Data Center Replication (XDCR)
• Read the XDCR documentation
• Read how XDCR is essential for globally distributed data, disaster recovery and high availability