Couchbase lite replication error - Method not found: StateConfiguration

Please see log below.

w3wp.exe Error: 0 : Task 31 faulted System.AggregateException: One or
more errors occurred. —> Sharpen.RuntimeException: Runtime
Exception —> System.IO.FileNotFoundException: Could not load file
or assembly ‘Mono.Security, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=0738eb9f132ed756’ or one of its dependencies. The system
cannot find the file specified.
at Couchbase.Lite.Security.SSLGenerator.GetOrCreateClientCert()
at Couchbase.Lite.Support.CouchbaseLiteHttpClientFactory.BuildHandlerPipeline(Boolean chunkedMode)
at Couchbase.Lite.Support.CouchbaseLiteHttpClientFactory.GetHttpClient(Boolean chunkedMode)

at Couchbase.Lite.Replication.SendAsyncRequest(HttpMethod method, Uri
url, Object body, RemoteRequestCompletionBlock completionHandler,
CancellationTokenSource requestTokenSource)
at
Couchbase.Lite.Replication.SendAsyncRequest(HttpMethod method, String
relativePath, Object body, RemoteRequestCompletionBlock
completionHandler, CancellationTokenSource requestTokenSource)
at Couchbase.Lite.Replicator.Pusher.ProcessInbox(RevisionList inbox)
at Couchbase.Lite.Replication.<.ctor>b__129_0(IList1 inbox)
— End of inner exception stack trace —
at Couchbase.Lite.Replication.<.ctor>b__129_0(IList1 inbox)
at Couchbase.Lite.Support.Batcher1.ProcessNow()
at Couchbase.Lite.Support.Batcher1.b__21_0(Task t)
at System.Threading.Tasks.ContinuationResultTaskFromTask1.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
— End of inner exception stack trace —
—>
(Inner Exception #0) Sharpen.RuntimeException: Runtime Exception
—> System.IO.FileNotFoundException: Could not load file or assembly
’Mono.Security, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=0738eb9f132ed756’ or one of its dependencies. The system
cannot find the file specified.
at Couchbase.Lite.Security.SSLGenerator.GetOrCreateClientCert()
at Couchbase.Lite.Support.CouchbaseLiteHttpClientFactory.BuildHandlerPipeline(Boolean chunkedMode)
at Couchbase.Lite.Support.CouchbaseLiteHttpClientFactory.GetHttpClient(Boolean chunkedMode)

at Couchbase.Lite.Replication.SendAsyncRequest(HttpMethod method, Uri
url, Object body, RemoteRequestCompletionBlock completionHandler,
CancellationTokenSource requestTokenSource)
at
Couchbase.Lite.Replication.SendAsyncRequest(HttpMethod method, String
relativePath, Object body, RemoteRequestCompletionBlock
completionHandler, CancellationTokenSource requestTokenSource)
at Couchbase.Lite.Replicator.Pusher.ProcessInbox(RevisionList inbox)
at Couchbase.Lite.Replication.<.ctor>b__129_0(IList1 inbox)
— End of inner exception stack trace —
at Couchbase.Lite.Replication.<.ctor>b__129_0(IList1 inbox)
at Couchbase.Lite.Support.Batcher1.ProcessNow()
at Couchbase.Lite.Support.Batcher1.b__21_0(Task t)
at System.Threading.Tasks.ContinuationResultTaskFromTask1.InnerInvoke()
at System.Threading.Tasks.Task.Execute()<—

If I upgrade to the latest master branch, will my current way of "
install a certificate and bind it to the appropriate port " still work
or I need to do it a different way? Sorry, I need to get it working
pretty quickly so I won’t have time to go through code to figure out how
to configure it for the master branch.

Ah yes I forgot about that path from then. This path, ironically, is only needed for Mono as well (its implementation of HTTPS on HttpListener forces the requirement of a client certificate). This is not needed for Windows, and if you need to stay on the commit you are on you can safely comment out this line when building. Just be aware that the longer you stay here the longer you remain on unsupported code. I think with this one line gone it will work alright for your testing and give you some time to start migrating.

The switch to master will not be as simple as you indicate, but that’s not to say that it will be very difficult. Probably the weirdest part will be converting your cer and pvk file into one PFX file. There is a stack overflow answer on the subject. PFX and P12 are the same so if you have a p12 file you can use that (I think most major SSL certificate providers have an option to get the certificate / key in this format). In the master branch there is an additional argument on the CouchbaseLiteTcpListener constructor that accepts an object of type X509Certificate2. Instead of installing it, you will instantiate an X509Certificate2 object using your certificate file and pass it in (there is an X509Certificate2 constructor that takes a file path as an argument). You can see an example in my unit tests. If you already have a certificate ready to use, you don’t need to worry about the X509Manager call. You can see in the implementation that it will prefer an already existing file, and you can just use that one line.

Thank you @borrrden. I commented the line out and now it is working.

This morning when the tester was testing the application, we got below error in the log… The app was doing delete, not insert. I am guessing it was the line “doc.Delete()” caused the error. Replication was working.

The code that threw this error are:

var doc = database.GetExistingDocument(id);
if (doc == null) return null;
var result = ConvertToMessage(doc);
doc.Delete();
database.Compact();
return result;

Could you please shed some light on this?

“exception”: “19: \r\nSQLitePCL.Ugly.ugly+sqlite3_exception: Exception of type ‘SQLitePCL.Ugly.ugly+sqlite3_exception’ was thrown.\r\n at SQLitePCL.Ugly.ugly.step(sqlite3_stmt stmt)\r\n at Couchbase.Lite.SqlitePCLRawStorageEngine.<>c__DisplayClass41_0.b__0()” }

I am guessing… the app was trying to delete the same document twice and somehow tried to insert a revision with the same data as an existing one?

I haven’t seen an error like that in quite a while. But without the surrounding logs or a stack trace there is not much more I can say about it. If you can reproduce it reliably then please file an issue on the github repo. Trying to insert an identical document is not necessarily a fatal error.

I captured the error. This is a bit urgent so if you could get back to me asap, that would be great… Thank you.

w3wp.exe Information: 0 : Begin transaction (level 1) w3wp.exe Information: 0 : RawQuery sql: SELECT lastsequence FROM views WHERE name=? (messages) w3wp.exe Information: 0 : RawQuery sql: SELECT seq FROM sqlite_sequence WHERE name='revs' () w3wp.exe Information: 0 : Checking indexes of (messages) for messages w3wp.exe Information: 0 : Begin transaction (level 2) w3wp.exe Information: 0 : RawQuery sql: SELECT seq FROM sqlite_sequence WHERE name='revs' () w3wp.exe Information: 0 : RawQuery sql: SELECT lastsequence FROM views WHERE name=? (messages) w3wp.exe Information: 0 : Committing transaction (level 2) w3wp.exe Information: 0 : RawQuery sql: SELECT lastsequence FROM views WHERE name=? (messages) w3wp.exe Information: 0 : RawQuery sql: SELECT lastsequence FROM views WHERE name=? (messages) w3wp.exe Information: 0 : Query messages: SELECT key, value, docid, revs.sequence FROM 'maps_1', revs, docs WHERE 1 AND key IN (?) AND revs.sequence = 'maps_1'.sequence AND docs.doc_id = revs.doc_id ORDER BY key, docid LIMIT ? OFFSET ? Arguments: ["\"dev\"",-1,0] w3wp.exe Information: 0 : RawQuery sql: SELECT key, value, docid, revs.sequence FROM 'maps_1', revs, docs WHERE 1 AND key IN (?) AND revs.sequence = 'maps_1'.sequence AND docs.doc_id = revs.doc_id ORDER BY key, docid LIMIT ? OFFSET ? ("dev", -1, 0) w3wp.exe Information: 0 : Query messages: Found row with key=System.Byte[], value=System.Byte[], id=-2gtiRxWxI0Op5dfHeLyDkg w3wp.exe Information: 0 : Query messages: Found row with key=System.Byte[], value=System.Byte[], id=-I0w8DjohvUmUbi8GdxaFmA w3wp.exe Information: 0 : Query messages: Found row with key=System.Byte[], value=System.Byte[], id=-VpidQ4STsEKRu2rapl57Ug w3wp.exe Information: 0 : Query messages: Found row with key=System.Byte[], value=System.Byte[], id=-gwFgXJ1zM0GinInfmOyBCw w3wp.exe Information: 0 : Query messages: Returning iterator w3wp.exe Information: 0 : Committing transaction (level 1) w3wp.exe Information: 0 : PUT _id=-I0w8DjohvUmUbi8GdxaFmA, _rev=2-7399c7a18a56ae2fcc781f8de9bad673, _deleted=True, allowConflict=False w3wp.exe Information: 0 : Begin transaction (level 1) w3wp.exe Information: 0 : RawQuery sql: SELECT revid, deleted FROM revs WHERE doc_id=? and current=1 ORDER BY deleted asc, revid desc LIMIT ? (555, 2) w3wp.exe Information: 0 : RawQuery sql: SELECT sequence FROM revs WHERE doc_id=? AND revid=? AND current=1 LIMIT 1 (555, 2-7399c7a18a56ae2fcc781f8de9bad673) w3wp.exe Information: 0 : Preparing statement: 'INSERT INTO revs (doc_id,revid,parent,current,deleted,no_attachments,json) VALUES (?,?,?,?,?,?,?)' with values: 555, 3-7d4e51e911acec4b17070d5ce1c90f60, 1703, True, True, True, System.Byte[] w3wp.exe Warning: 0 : Failed to run transaction w3wp.exe Information: 0 : CANCEL transaction (level 1)

I didn’t click on the reply button so not sure if you getting notification… Please see above.

I don’t see where the actual exception is in your log, though I imagine it occurs between the 3rd to last and 2nd to last line. There’s nothing that looks suspicious in the code you posted either except I don’t know what “ConvertToMessage” is doing. The error is that a duplicate revision is attempting to be inserted (deletions are also revisions). That kind of error is set up to be ignored, because if the revision is already there then nothing needs to be done. It’s odd that it is showing up again after so many months, so I’m interested in how to reproduce this.

That is logging set to verbose… I don’t see the exception either. I’d say it was the insert into revs part. The exception stack points to doc.Delete()… CovertToMessage is a method that maps document properties to a DTO object.

    private Message ConvertToMessage(Document document)
    {
        return new Message
        {
            Id = document.Id,
            Body = document.GetProperty<string>("Body"),
            CreatedAt = document.GetProperty<DateTime>("CreatedAt"),
            Sequence = document.Properties.ContainsKey("Sequence") ? document.GetProperty<long>("Sequence") : 0
        };
    }

We have a recurring job (runs every minute) that accepts a list of document ids. The job does a foreach ids -> call Db.Delete(id); The content of Db.Delete(id) is listed above (The one that has ConvertToMessage(doc))… We have replication turned on. There is nothing else I could think of… Was the issue fixed in a later version or you couldn’t replicate it? We had it happening to us 2 times in a day.

I have your source code, so if you could point me to the right direction, I can fix it in our custom build. In SaveRevision.cs Line 255, the allow conflict flag is set to false… Would it be the problem?

    public SavedRevision CreateRevision(IDictionary<String, Object> properties) {
        return Document.PutProperties(properties, RevisionInternal.GetRevId(), false);           
    }

false is correct in that method. I don’t remember doing anything significant to this logic since the commit that you have. I’ve tried a number of different scenarios of deleting the same document twice but I haven’t successfully been able to get the exception you mentioned yet (on the latest source anyway, if I get a chance I’ll put the same tests into the old commit).

If you didn’t find the exception in the logs, then where did you find it?

Our application cough the exception. The recursive job kept processing the same record because it was not deleted. The job ended up throwing the error every time it get executed. It doesn’t happen all the time. Happened 8:00 this morning then 16:50 this afternoon… We cannot go-live with this error… Please let me know if you need more info.

“exception”: “19: \r\nSQLitePCL.Ugly.ugly+sqlite3_exception: Exception of type ‘SQLitePCL.Ugly.ugly+sqlite3_exception’ was thrown.\r\n at SQLitePCL.Ugly.ugly.step(sqlite3_stmt stmt)\r\n at Couchbase.Lite.SqlitePCLRawStorageEngine.<>c__DisplayClass41_0.b__0()” }

Would it be possible that it was caused by compacting the database immediately after each delete?

Are you sure that exception is unhandled and not just output? Sometimes Windows will write a log for every exception that occurs (even if it is later handled). It’s especially suspicious because the exception has no stack trace on it. An error code 19 can only come from an INSERT and in the code path for inserting revisions all exceptions are wrapped in a CouchbaseLiteException before being thrown.

However, I think I spotted a difference between your version and the current version. In the version you have there is an unguarded call to InsertRevision. Notice that in the current version it has a try / catch on it. This still won’t cause the exception you mentioned to be thrown, but it will cause an exception to be thrown in some cases on a related code path so it would be worth getting this change.

I will never say 100% sure but I’d say 99.9% sure it was an exception. :grin: The document never get deleted… If the document was removed, I’d say it was just an output message.
In regarding to the changes, I think we might be using
ForestDBCouchStore… because on VS project Couchbase.Lite.Net45, there
is a Conditional Compilation Symbol FORESTDB in the Build tab. I
assumed that was what we should be using and didn’t muck around with the
settings. Am I misunderstanding the way it works? Should I change the
Conditional Compilation Symbol and then patch up SqliteCouchStore.cs?

You can ignore that symbol. It is just used for compiling in support for ForestDB or not (it doesn’t exist anymore, since I’ve refactored storage engine support into plugins). You are definitely not using it since you have so many SQLite statement logs and you would know if you were using it because it requires the extra step of compiling or downloading the native ForestDB libraries ;).

When you say “the document never gets deleted” do you mean that GetExistingDocument() always returns a non-null value?

Let me clarify it more. In our recursive job, we run a query on a view to get a list of messages from CB Lite. Then the job process them and then delete them one by one.
What’s happening now is, the job kept getting the same document again and again while querying the view because the document never get deleted…
Here is the code that query the database.

messageView came from calling CreateMessagesByEnvironmentView() from constructor so we are using the same view instance to run querys… Is that a problem?

    public List<Environment> GetAll(string environment = "")
    {
        var result = new List<Message>();
        var orderedQuery = messagesView.CreateQuery();
        if (!string.IsNullOrEmpty(environment))
        {
            var keys = new List<string> { environment.ToLower() };
            orderedQuery.Keys = keys;
        }
        try
        {
            var results = orderedQuery.Run();
            var list = results.ToList();
            list.ForEach(r =>
            {
                var doc = r.Document;
                result.Add(ConvertToMessage(doc));
                log.Debug("Found document with id: {0}, CreatedAt = {1}, Message = {2}", r.DocumentId, doc.GetProperty<DateTime>("CreatedAt"), doc.GetProperty<string>("Message"));
            });
        }
        catch (CouchbaseLiteException e)
        {
            log.Error("Error querying db view: {0}", e.Message);
            throw new ApplicationException(string.Format("Error querying db view: {0}", e.Message), e);
        }

        return result;
    }

private View GetView()
{
View cbView;
try
{
cbView = database.GetView(“messages”);
}
catch (CouchbaseLiteException e)
{
const string message = “Cannot get view from Couchbase Lite.”;
log.Error(e, message);
throw new ApplicationException(message, e);
}
return cbView;
}

private View CreateMessagesByEnvironmentView()
{
var messagesByEnvView = GetView();
if (messagesByEnvView.Map == null)
{
messagesByEnvView.SetMap((props, emit) =>
{
if (props.ContainsKey(“Environment”))
emit(props[“Environment”], null);
},
“2”);
}
return messagesByEnvView;
}

Another question, I put the try catch block around InsertRevision. Is Revision important for replication? If there was an exception and CB Lite didn’t insert a revision, will it cause the replication not happen for the delete operation?