Configuração do XDCR Cross-VPC entre clusters do Amazon EKS

In modern distributed systems, the ability to replicate data between separate environments is crucial for ensuring high availability, disaster recovery, and performance optimization. Couchbase’s XDCR (Cross Data Center Replication) feature allows seamless replication of data between clusters, enabling robust data sharing across geographically or logically isolated environments.

This guide will walk you through setting up XDCR between two Couchbase clusters hosted in separate Amazon EKS (Elastic Kubernetes Service) clusters within different VPCs. We’ll dive into each step, from infrastructure setup to configuring DNS for cross-cluster communication and deploying Couchbase for real-time replication. By the end of this walkthrough, you’ll have a production-ready setup with the skills to replicate this in your environment.

Pré-requisitos

To follow this guide, ensure you have:

• • CLI DO AWS installed and configured
  • An AWS account with permissions for creating VPCs, EKS clusters, and security groups
  • Familiarity with Kubernetes and tools like kubectl and Helm
  • Helm installed to deploy Couchbase
  • Basic knowledge of networking concepts, including CIDR blocks, routing tables, and DNS

Step 1: Deploy EKS clusters in separate VPCs

What are we doing?

We will create two Kubernetes clusters, Cluster1 and Cluster2, in separate VPCs using eksctl. Each cluster will operate independently and have its own CIDR block to avoid IP conflicts.

Why is this important?

This separation ensures:

1. 1. Isolation for better security and management
   2. Scalability and flexibility for handling workloads
   3. Clear routing rules between clusters

Commands to create clusters

Deploy cluster1

Deploy cluster2

Expected outcome

• • Cluster1 resides in VPC 10.0.0.0/16
  • Cluster2 resides in VPC 10.1.0.0/16

Step 2: Peer the VPCs for Inter-Cluster Communication

What are we doing?

We are creating a VPC Peering connection between the two VPCs and configuring routing and security rules to enable inter-cluster communication.

Etapas

2.1 Create a peering connection

• • Go to the AWS Console > VPC > Peering Connections
  • Clique em Criar conexão de peering
  • Selecione o Requester VPC (Cluster1 VPC) and Accepter VPC (Cluster2 VPC)
  • Name the connection eks-peer
  • Clique em Criar conexão de peering

2.2 Accept the peering request

• • Select the peering connection
  • Clique em Actions > Accept Request

2.3 Update route tables

• • For Cluster1 VPC, add a route for 10.1.0.0/16, targeting the peering connection
  • For Cluster2 VPC, add a route for 10.0.0.0/16, targeting the peering connection

2.4 Modify security groups

Why is this necessary?

Security groups act as firewalls, and we must allow traffic between the clusters explicitly.

How to modify

1. 1. Navegue até EC2 > Security Groups in the AWS Console
   2. Identify the security groups associated with Cluster1 and Cluster2
   3. For Cluster1’s security group:
      • Clique em Edit Inbound Rules
      • Add a rule:
        • Tipo: All traffic
        • Fonte: Security group ID of Cluster2
   4. Repeat for Cluster2, allowing traffic from Cluster1’s security group

Step 3: Test Connectivity by Deploying NGINX in Cluster2

What are we doing?

We are deploying an NGINX pod in Cluster2 to verify that Cluster1 can communicate with it.

Why is this important?

This step ensures the networking between clusters is functional before deploying Couchbase.

Etapas

3.1 Create a Namespace in Cluster1 and Cluster2

3.2 Deploy NGINX in Cluster1 and Cluster2

• • Create nginx.yaml:
    
    
    YAML

    apiVersion: apps/v1 kind: Deployment metadata:   name: nginx spec:   replicas: 1   selector:     matchLabels:       app: nginx   template:     metadata:       labels:         app: nginx     spec:       containers:       - name: nginx         image: nginx         ports:         - containerPort: 80 --- apiVersion: v1 kind: Service metadata:   name: nginx spec:   clusterIP: None   ports:   - port: 80     targetPort: 80   selector:     app: nginx

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

    Versão da API: apps/v1 gentil: Implantação metadados:   nome: nginx especificação:   réplicas: 1   seletor:     matchLabels:       aplicativo: nginx   modelo:     metadados:       rótulos:         aplicativo: nginx     especificação:       contêineres:       - nome: nginx         imagem: nginx         portos:         - containerPort: 80 --- Versão da API: v1 gentil: Serviço metadados:   nome: nginx especificação:   clusterIP: None   portos:   - porto: 80     porta de destino: 80   seletor:     aplicativo: nginx

3.3 Apply the YAML

3.4 Verify Connectivity from Cluster1

• • Exec into the pod in Cluster1:
    
    
    Concha

    kubectl exec -it -n dev <pod-name> -- /bin/bash

    1	kubectl executar -ele -n dev <cápsula-nome> -- /caixa/bash

3.5 Test connectivity to Cluster2

Expected outcome

O enrolar command will fail without DNS forwarding, highlighting the need for further DNS configuration.

Step 4: Configuring DNS forwarding

What are we doing?

We’ll configure DNS forwarding so that services in Cluster2 can be resolved by Cluster1. This is critical for allowing applications in Cluster1 to interact with services in Cluster2 using their DNS names.

Why is this important?

Kubernetes service discovery relies on DNS, and by default, DNS queries for one cluster’s services cannot resolve in another cluster. CoreDNS in Cluster1 must forward queries to Cluster2’s DNS resolver.

Etapas

4.1 Retrieve Cluster2’s DNS Service Endpoint

• • Run the following command in Cluster2 to get the DNS service endpoint:
    
    
    Concha

    kubectl get endpoints -n kube-system

    1	kubectl obter pontos finais -n cubo-sistema

  • Look for the kube-dns ou coredns service and note its IP address. For example:
    
    
    ---------------------------------------- NAME       ENDPOINTS          AGE kube-dns   10.1.20.116:53     3h ----------------------------------------

    1 2 3 4

    ---------------------------------------- NOME       ENDPOINTS          IDADE cubo-dns   10.1.20.116:53     3h ----------------------------------------

    
    

4.2 Edit the CoreDNS ConfigMap in Cluster1

• • Open the CoreDNS ConfigMap for editing:
    
    
    Concha

    kubectl edit cm coredns -n kube-system

    1	kubectl editar cm coredns -n cubo-sistema

    
     

4.3 Add the following block to the Corefile section

Replace 10.1.20.116 with the actual DNS endpoint IP from Cluster2.

Note: We only need to use one of the CoreDNS endpoints for this ConfigMap. The IP of CoreDNS pods rarely change but can if the Node goes down. The kube-dns ClusterIP service can be used but will require the IP and Port to be open on the EKS Nodes.

4.4 Restart CoreDNS in Cluster1

• • Apply the changes by restarting CoreDNS:
    
    
    Concha

    kubectl rollout restart deployment coredns -n kube-system

    1	kubectl lançamento reiniciar implantação coredns -n cubo-sistema

    
    

4.5 Verify DNS Forwarding

• • Exec into any pod in Cluster1:
    
    
    Concha

    kubectl exec -it -n default <pod-name> -- /bin/bash

    1	kubectl executar -ele -n padrão <cápsula-nome> -- /caixa/bash

  • Test DNS resolution for an NGINX service in Cluster2:
    
    
    Concha

    curl nginx.prod.svc.cluster.local

    1	enrolar nginx.prod.svc.cluster.local

  • You should see a response from the NGINX service
    

Expected outcome

DNS queries from Cluster1 to Cluster2 should resolve successfully.

Step 5: Deploying Couchbase

What are we doing?

We’ll deploy Couchbase clusters in both Kubernetes environments using Helm. Each cluster will manage its own data independently before being connected through XDCR.

Why is this important?

Couchbase clusters form the foundation of the XDCR setup, providing a robust and scalable NoSQL database platform.

Etapas

5.1 Add the Couchbase Helm Chart Repository

5.2 Deploy Couchbase in Cluster1

• • Switch to Cluster1:
    
    
    Concha

    kubectl config use-context <cluster1-context>

    1	kubectl configuração uso-contexto <cluster1-contexto>

  • Deploy Couchbase:
    
    
    Concha

    helm install couchbase couchbase/couchbase-operator --namespace dev

    1	leme instalar couchbase couchbase/couchbase-operador --espaço de nome dev

5.3 Deploy Couchbase in Cluster2

• • Switch to Cluster2:
    
    
    Concha

    kubectl config use-context <cluster2-context>

    1	kubectl configuração uso-contexto <cluster2-contexto>

  • Deploy Couchbase:
    
    
    Concha

    helm install couchbase couchbase-operator --namespace prod

    1	leme instalar couchbase couchbase-operador --espaço de nome prod

5.4 Verify Deployment

• • Check the Couchbase pods:
    
    
    Concha

    kubectl get pods -n dev  # For Cluster1 kubectl get pods -n prod # For Cluster2

    1 2	kubectl obter cápsulas -n dev  # For Cluster1 kubectl obter cápsulas -n prod # For Cluster2

    
    
  • Ensure all pods are running.

Observação: If you encounter a deployment error, edit the CouchbaseCluster CRD to use a compatible image version:

Change:

To:

Expected outcome

Couchbase clusters should be running and accessible via their respective UIs.

Step 6: Setting Up XDCR

What are we doing?

We’ll configure XDCR to enable data replication between the two Couchbase clusters.

Why is this important?

XDCR ensures data consistency across clusters, supporting high availability and disaster recovery scenarios.

Etapas

6.1 Get service name from Cluster2

• • In cluster2 run the following command to retrieve the service name of one of the pods so we can port-forward to it.
    
    
    Concha

    kubectl get services -n prod

    1	kubectl obter serviços -n prod

6.2 Access the Couchbase UI for Cluster2

• • • Port-forward the Couchbase UI:
      
      
      YAML

      kubectl port-forward -n prod cluster2-0000 8091:8091

      1	kubectl port-forward -n prod cluster2-0000 8091:8091

• • Open a browser and navigate to:
    http://localhost:8091
    

• • Log in using the credentials configured during the deployment.

6.3 View Documents in Cluster2

• • In the Couchbase UI, go to Baldes
  • Note that no Documents exist in the padrão balde

6.4 Access the Couchbase UI for Cluster1

• • Port-forward the Couchbase UI:
    
    
    Concha

    kubectl port-forward -n dev cluster1-0000 8091:8091

    1	kubectl porto-avançar -n dev cluster1-0000 8091:8091

    
    
  • Open a browser and navigate to:
    http://localhost:8091
  • Log in using the credentials configured during the deployment

6.5 Add a Remote Cluster

• • In the Couchbase UI, go to XDCR > Add Remote Cluster
  • Configure the remote cluster:
    • Cluster Name: Cluster2
    • IP/Hostname: <Cluster2 Service Name>.prod.svc.cluster.local
    • Nome de usuário: Admin username for Cluster2
    • Senha: Admin password for Cluster2
  • Clique em Salvar
    

6.6 Set Up Replication

• • In the Couchbase UI for Cluster1, go to XDCR > Add Replication
  • Configure the replication:
    • Replicate From Bucket: Default bucket in Cluster1
    • Replicate To Bucket: Default bucket in Cluster2
    • Remote Cluster: Select Cluster2
  • Clique em Salvar
    
    

6.7 Test Replication

• • Add sample documents to the default bucket in Cluster1:
    • In the Couchbase UI, navigate to Buckets > Documents > Add Document
    • Give the Documento a unique ID and some data in JSON formato
      
    • Verify that the documents appear in the default bucket of Cluster2:
      • Port-forward to Cluster2’s UI and log in:
        
        
        Concha

        kubectl port-forward -n prod cluster2-0000 8091:8091

        1	kubectl porto-avançar -n prod cluster2-0000 8091:8091

    • Navegue até: http://localhost:8091

Expected outcome

Data added to Cluster1 should replicate to Cluster2 in real time.

Step 7: Cleanup

What are we doing?

We’ll cleanup our AWS environment and delete all the resources we deployed.

Why is this important?

This will prevent you from incurring unnecessary charges.

Etapas

7.1 Access the AWS Console

• • Go to the AWS Console > VPC > Peering Connections
  • Select and delete the peering connection
  • Go to the AWS Console > CloudFormation > Stacks
  • Select and delete the two nodegroup stacks
  • Once the two nodegroup stacks have finished deleting select and delete the cluster stacks

Expected outcome

All resources created for this tutorial are deleted from account.

Conclusão

Through this guide, we’ve successfully established XDCR between Couchbase clusters running in separate EKS clusters across AWS VPCs. This setup highlights the power of combining AWS networking with Kubernetes for robust, scalable solutions. With cross-cluster replication in place, your applications gain enhanced resilience, reduced latency for distributed users, and a solid disaster recovery mechanism.

By understanding and implementing the steps outlined here, you’re equipped to tackle real-world challenges involving multi-cluster setups, expanding your expertise in both cloud networking and distributed database management.

• • Saiba mais sobre Couchbase Cross Data Center Replication (XDCR)
  • Read the Documentação do XDCR
  • Read how XDCR is essential for globally distributed data, disaster recovery and high availability