Couchbase lite synching with couchbase server using sync_gateway

Hello,

We are trying to create a simple console application in .Net(Windows machine) which connects couchbase lite → sync-gateway → couchbase server (hosted in a remote machine).

We followed the steps mentioned in the support doc(https://docs.couchbase.com/couchbase-lite/2.7/csharp.html), but the data is not pulled from the server to the lite.

here is the configuration used for sync_gateway -

{
“log”: [““],
“databases”: {
“testdb”: {
“server”: “http://167.87.210.35:8091”,
“bucket”: “Test”,
“username”: “System”,
“password”: “welcome”,
“enable_shared_bucket_access”: true,
“import_docs”: true,
“num_index_replicas”: 0,
“users”: {
“GUEST”: { “disabled”: false, “admin_channels”: [”
”] }
},
“sync”: function (doc, oldDoc) { if (doc.sdk) { channel(doc.sdk); } }
}
}
}

and here is the sample console application we have created -

using Couchbase.Lite;
using Couchbase.Lite.Query;
using Couchbase.Lite.Sync;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace CouchBase_sample
{
class Program
{
static void Main(string argss)

    {
        // Get the database (and create it if it doesn't exist)
        var database = new Database("testdb");         

        // Create replicator to push and pull changes to and from the cloud
        var targetEndpoint = new URLEndpoint(new Uri("ws://localhost:4984/testdb"));
        var replConfig = new ReplicatorConfiguration(database, targetEndpoint);

        // Add authentication
        replConfig.Authenticator = new BasicAuthenticator("System", "welcome");


        // Create replicator (make sure to add an instance or static variable
        // named _Replicator)
        Replicator _Replicator = new Replicator(replConfig);
        _Replicator.AddChangeListener((sender, args) =>
        {
            if (args.Status.Error != null)
            {
                Console.WriteLine($"Error :: {args.Status.Error}");
            }
        });

        _Replicator.Start();


        // Later, stop and dispose the replicator *before* closing/disposing the database


        using (var query = QueryBuilder.Select(SelectResult.All())
            .From(DataSource.Database(database)))
        {
            // Run the query
            Console.WriteLine($"Query :: {query.ToString()}");
            var result = query.Execute();
            Console.WriteLine($"Result ::  {result.Count()}");
        }
    }

}

}

Is there any other doc/video for the same and is there any existing sample code/application.

Please find the screenshot which shows the error we get.

NOTE: sorry for previous screenshot. It was wrong one. I have provided correct screenshot for the error we are getting when we run the sample console application

That error indicates there is either already a Sync Gateway instance running using port 4984 and 4985, or some other service is running using those ports.

Ben’s response should fix your issue. But FFR, specifically on your question , you can refer to tutorials page. Also, here is a video walkthrough of this very simple Xamarin tutorial

Thanks for your reply. I really appreciate it. I have updated my question. Please respond to it if possible

Thanks for your reply! I have updated my post. Please go through it and respond if possible.

Share the logs from sync gateway (assuming it is running)
What does output of http://localhost:4984 say?

Hi

as requested by you. please find below -

Syncgateway log -

Microsoft Windows [Version 10.0.17134.1365]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\Program Files\Couchbase\Sync Gateway>sync_gateway.exe C:\Users\z003wsjz\Desktop\sync-gateway-config.json
2020-04-02T20:32:04.043+05:30 ==== Couchbase Sync Gateway/2.7.2(2;583d2dc) CE ====
2020-04-02T20:32:04.044+05:30 [WRN] Using deprecated config option: “log”. Use “logging.console.log_keys” instead. – rest.(*ServerConfig).deprecatedConfigLoggingFallback.func5() at config.go:763
2020-04-02T20:32:04.045+05:30 [INF] Logging: Console to stderr
2020-04-02T20:32:04.045+05:30 [INF] Logging: Files disabled
2020-04-02T20:32:04.045+05:30 [ERR] No log_file_path property specified in config, and --defaultLogFilePath command line flag was not set. Log files required for product support are not being generated. – base.(LoggingConfig).Init.func1() at logging_config.go:62
2020-04-02T20:32:04.045+05:30 [INF] Logging: Console level: info
2020-04-02T20:32:04.045+05:30 [INF] Logging: Console keys: [
HTTP]
2020-04-02T20:32:04.045+05:30 [INF] Logging: Redaction level: none
2020-04-02T20:32:04.045+05:30 [INF] Logging stats with frequency: 1m0s
2020-04-02T20:32:04.045+05:30 [INF] Opening db /testdb as bucket “Test”, pool “default”, server http://167.87.195.91:8091/
2020-04-02T20:32:04.045+05:30 [INF] GoCBCustomSGTranscoder Opening Couchbase database Test on http://167.87.195.91:8091/ as user “System”
2020-04-02T20:32:04.045+05:30 [INF] Auth: Attempting credential authentication http://167.87.195.91:8091?http_idle_conn_timeout=90000&http_max_idle_conns=64000&http_max_idle_conns_per_host=256&kv_pool_size=2&n1ql_timeout=75000&operation_tracing=false
2020-04-02T20:32:05.134+05:30 [INF] Successfully opened bucket Test
2020-04-02T20:32:05.515+05:30 [INF] Set query timeouts for bucket Test to cluster:1m15s, bucket:1m15s
2020-04-02T20:32:05.515+05:30 [INF] Initializing indexes with numReplicas: 0…
2020-04-02T20:32:07.560+05:30 [INF] Verifying index availability for bucket Test…
2020-04-02T20:32:07.900+05:30 [INF] Indexes ready for bucket Test.
2020-04-02T20:32:07.900+05:30 [INF] delta_sync enabled=false with rev_max_age_seconds=86400 for database testdb
2020-04-02T20:32:08.067+05:30 [INF] Created background task: “CleanAgedItems” with interval 1m0s
2020-04-02T20:32:08.067+05:30 [INF] Cache: Initializing changes cache for database testdb with options {ChannelCacheOptions:{ChannelCacheMinLength:50 ChannelCacheMaxLength:500 ChannelCacheAge:1m0s MaxNumChannels:50000 CompactHighWatermarkPercent:80 CompactLowWatermarkPercent:60} CachePendingSeqMaxWait:5s CachePendingSeqMaxNum:10000 CacheSkippedSeqMaxWait:1h0m0s}
2020-04-02T20:32:08.067+05:30 [INF] Created background task: “InsertPendingEntries” with interval 2.5s
2020-04-02T20:32:08.067+05:30 [INF] Created background task: “CleanSkippedSequenceQueue” with interval 30m0s
2020-04-02T20:32:08.067+05:30 [INF] DCP: Attempting to start import DCP feed…
2020-04-02T20:32:08.067+05:30 [INF] DCP: Registering PindexImplType for syncGateway-import-testdb
2020-04-02T20:32:08.067+05:30 [INF] DCP: Starting DCP import feed for bucket: “Test”
2020-04-02T20:32:09.160+05:30 [INF] DCP: Previously persisted backfill sequences found - will resume
2020-04-02T20:32:09.308+05:30 [INF] DCP: c:Test-SGI Restarting vb 0 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:09.462+05:30 [INF] DCP: c:Test-SGI Restarting vb 1 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:09.630+05:30 [INF] DCP: c:Test-SGI Restarting vb 2 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:09.786+05:30 [INF] DCP: c:Test-SGI Restarting vb 3 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:09.945+05:30 [INF] DCP: c:Test-SGI Restarting vb 4 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:10.115+05:30 [INF] DCP: c:Test-SGI Restarting vb 5 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:10.271+05:30 [INF] DCP: c:Test-SGI Restarting vb 6 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:10.566+05:30 [INF] DCP: c:Test-SGI Restarting vb 7 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:10.717+05:30 [INF] DCP: c:Test-SGI Restarting vb 8 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:10.883+05:30 [INF] DCP: c:Test-SGI Restarting vb 9 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:11.040+05:30 [INF] DCP: c:Test-SGI Restarting vb 10 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:11.199+05:30 [INF] DCP: c:Test-SGI Restarting vb 11 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:11.418+05:30 [INF] DCP: c:Test-SGI Restarting vb 12 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:11.579+05:30 [INF] DCP: c:Test-SGI Restarting vb 13 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:11.745+05:30 [INF] DCP: c:Test-SGI Restarting vb 14 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:11.897+05:30 [INF] DCP: c:Test-SGI Restarting vb 15 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.046+05:30 [INF] DCP: c:Test-SGI Restarting vb 16 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.198+05:30 [INF] DCP: c:Test-SGI Restarting vb 17 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.356+05:30 [INF] DCP: c:Test-SGI Restarting vb 18 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.513+05:30 [INF] DCP: c:Test-SGI Restarting vb 19 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.683+05:30 [INF] DCP: c:Test-SGI Restarting vb 20 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.848+05:30 [INF] DCP: c:Test-SGI Restarting vb 21 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:12.998+05:30 [INF] DCP: c:Test-SGI Restarting vb 22 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:13.147+05:30 [INF] DCP: c:Test-SGI Restarting vb 23 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:13.307+05:30 [INF] DCP: c:Test-SGI Restarting vb 24 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:13.608+05:30 [INF] DCP: c:Test-SGI Restarting vb 25 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:13.766+05:30 [INF] DCP: c:Test-SGI Restarting vb 26 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:13.938+05:30 [INF] DCP: c:Test-SGI Restarting vb 27 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:14.102+05:30 [INF] DCP: c:Test-SGI Restarting vb 28 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:14.257+05:30 [INF] DCP: c:Test-SGI Restarting vb 29 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:14.408+05:30 [INF] DCP: c:Test-SGI Restarting vb 30 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:14.572+05:30 [INF] DCP: c:Test-SGI Restarting vb 31 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:14.726+05:30 [INF] DCP: c:Test-SGI Restarting vb 32 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:14.886+05:30 [INF] DCP: c:Test-SGI Restarting vb 33 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:15.041+05:30 [INF] DCP: c:Test-SGI Restarting vb 34 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:15.351+05:30 [INF] DCP: c:Test-SGI Restarting vb 35 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:15.506+05:30 [INF] DCP: c:Test-SGI Restarting vb 36 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:15.667+05:30 [INF] DCP: c:Test-SGI Restarting vb 37 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:15.830+05:30 [INF] DCP: c:Test-SGI Restarting vb 38 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:15.996+05:30 [INF] DCP: c:Test-SGI Restarting vb 39 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:16.145+05:30 [INF] DCP: c:Test-SGI Restarting vb 40 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:16.296+05:30 [INF] DCP: c:Test-SGI Restarting vb 41 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:16.499+05:30 [INF] DCP: c:Test-SGI Restarting vb 42 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:16.689+05:30 [INF] DCP: c:Test-SGI Restarting vb 43 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:16.889+05:30 [INF] DCP: c:Test-SGI Restarting vb 44 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:17.088+05:30 [INF] DCP: c:Test-SGI Restarting vb 45 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:17.245+05:30 [INF] DCP: c:Test-SGI Restarting vb 46 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:17.398+05:30 [INF] DCP: c:Test-SGI Restarting vb 47 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:17.558+05:30 [INF] DCP: c:Test-SGI Restarting vb 48 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:17.717+05:30 [INF] DCP: c:Test-SGI Restarting vb 49 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:17.894+05:30 [INF] DCP: c:Test-SGI Restarting vb 50 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:18.046+05:30 [INF] DCP: c:Test-SGI Restarting vb 51 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:18.194+05:30 [INF] DCP: c:Test-SGI Restarting vb 52 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:18.357+05:30 [INF] DCP: c:Test-SGI Restarting vb 53 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:18.520+05:30 [INF] DCP: c:Test-SGI Restarting vb 54 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:18.698+05:30 [INF] DCP: c:Test-SGI Restarting vb 55 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:18.846+05:30 [INF] DCP: c:Test-SGI Restarting vb 56 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:19.059+05:30 [INF] DCP: c:Test-SGI Restarting vb 57 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:19.226+05:30 [INF] DCP: c:Test-SGI Restarting vb 58 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:19.401+05:30 [INF] DCP: c:Test-SGI Restarting vb 59 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:19.547+05:30 [INF] DCP: c:Test-SGI Restarting vb 60 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:19.696+05:30 [INF] DCP: c:Test-SGI Restarting vb 61 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:19.847+05:30 [INF] DCP: c:Test-SGI Restarting vb 62 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:20.008+05:30 [INF] DCP: c:Test-SGI Restarting vb 63 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:20.157+05:30 [INF] DCP: c:Test-SGI Restarting vb 64 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:20.336+05:30 [INF] DCP: c:Test-SGI Restarting vb 65 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:20.487+05:30 [INF] DCP: c:Test-SGI Restarting vb 66 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:20.652+05:30 [INF] DCP: c:Test-SGI Restarting vb 67 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:20.810+05:30 [INF] DCP: c:Test-SGI Restarting vb 68 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:21.110+05:30 [INF] DCP: c:Test-SGI Restarting vb 69 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:21.268+05:30 [INF] DCP: c:Test-SGI Restarting vb 70 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:21.431+05:30 [INF] DCP: c:Test-SGI Restarting vb 71 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:21.648+05:30 [INF] DCP: c:Test-SGI Restarting vb 72 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:21.818+05:30 [INF] DCP: c:Test-SGI Restarting vb 73 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:22.011+05:30 [INF] DCP: c:Test-SGI Restarting vb 74 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:22.218+05:30 [INF] DCP: c:Test-SGI Restarting vb 75 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:22.389+05:30 [INF] DCP: c:Test-SGI Restarting vb 76 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:22.548+05:30 [INF] DCP: c:Test-SGI Restarting vb 77 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:22.702+05:30 [INF] DCP: c:Test-SGI Restarting vb 78 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:22.905+05:30 [INF] DCP: c:Test-SGI Restarting vb 79 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:23.059+05:30 [INF] DCP: c:Test-SGI Restarting vb 80 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:23.220+05:30 [INF] DCP: c:Test-SGI Restarting vb 81 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:23.369+05:30 [INF] DCP: c:Test-SGI Restarting vb 82 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:23.536+05:30 [INF] DCP: c:Test-SGI Restarting vb 83 using metadata sequence 0 (backfill 0 not in [0-0])
2020-04-02T20:32:23.696+05:30 [INF] DCP: c:Test-SGI Restarting vb 84 using metadata sequence 0 (backfill 0 not in [0-0])

response from

http://localhost:4984

{“couchdb”:“Welcome”,“vendor”:{“name”:“Couchbase Sync Gateway”,“version”:“2.7”},“version”:“Couchbase Sync Gateway/2.7.2(2;583d2dc) CE”}

@priya.rajagopal please have a look of the logs captured.

Thanks in advance!

@bbrks @priya.rajagopal I am very new to couchbase and syncgateway. Could you please explain the “sync” property in .json config file . How can we create it. I am doubting that property is not correctly created for us -

“sync”: function (doc, oldDoc) { if (doc.sdk) { channel(doc.sdk); } }

The sync function has nothing to do with the error you are seeing. It looks like connection is failing from the client due to the call to WinHttpGetProxyForUrl failing for some reason . What kind of Windows machine is this - OS version? Check the proxy configuration …do you have client proxies configured on that machine. Disable and try again.

On your question on sync function :

The sync function is a Javascript function that you would define to specify the access control and data routing policies.

Did you follow the tutorials that I shared with you earlier ? That explains this in the context of the example app. I would encourage you to walk through that as a solid understanding of those fundamentals would be important as you start building out your app.

In the example sync function - If you have a document with a sdk property , then that would be routed to a channel with name corresponding to that property. So if you don’t have any such matching documents, then it is just passed through by sync gateway . If you want to , you can also delete this property from your JSON config and use default sync function.

@priya.rajagopal

I am using Win 10. I have disabled the proxy configuration as well. Couchbase server is installed through docker as mentioned in the tutorial - Do a Quick Install | Couchbase Docs

I installed the couchbase server in my local machine only, still I can see that syncgatway is not able to reach server . Below is the error I am getting continuosly in log -

2020-04-05T11:48:55.181+05:30 [WRN] c:sync_new-SG Error processing DCP stream - will attempt to restart/reconnect if appropriate: worker connect, server: 172.17.0.2:11210, err: dial tcp 172.17.0.2:11210: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond… – base.(*DCPReceiver).OnError() at dcp_receiver.go:61

I am not sure what server ip signifies here

server: 172.17.0.2:11210

shouldn’t it connect to

http://localhost:8091

??
I am hoping if I can resolve above mentioned error then I will be able to acheive the synching between couchbase lite and couch base server.

I would really aprreciate if you can provide any leads.

The error you post keeps changing every time you respond with no insight into what exactly changed on your end and why since the last response. Pls. keep your system in a consistent state as you incorporate what is suggested in the responses- we cannot debug a moving target.

Now, to your latest question

If you have set up your server deployed using docker, then your sync gateway also needs to be deployed on the same docker network. Refer to this blog on how to do that.
Alternatively, just install the server binary directly and connect to it from Sync Gateway via localhost.

@priya.rajagopal I am very sorry to confuse you. Initially we tried with remote machine but now I tried in my local machine itslef…Thanks a lot for your suport! now my couchbase lite is syncing with couchbase server. I have installed the couchbase binary directly(not deployed in docker) and now syncgateway is able to reach server. though am getting below error sometimes -

2020-04-06T12:33:55.018+05:30 [ERR] Error opening database test2: 502 Unable to connect to Couchbase Server (connection refused). Please ensure it is running and reachable at the configured host and port. Detailed error: failed to connect to any of the specified hosts – rest.RunServer() at config.go:1028

but still I am able to push and pull data in server. I dont see any issue in databse syncing.