Issues pulling data from sync gateway, "received invalid doc ID from _changes"

Mobile
1

I have an android app that’s running using Couchbase Server version 4.1.0 and Sync Gateway/CBLite both version 1.2.0

After I updated the app to these more recent versions of CB and SG, I had it working briefly. Then I tried to add some authentication by disabling the GUEST user and using the Admin REST API to create a new user and then authenticate by using BasicAuthenticator in the app. This didn’t work the way I expected so I tried to regroup by putting it back the way it was- removing the new user and re-enabling the GUEST user. However, now even with these settings back the way they were when it was working before, the app will not work, it is not recieving any information from Couchbase anymore.

I have tried uninstalling and reinstalling to the state it should have been at right after I updated and it did not work. I am explaining this process mainly for context as it seems like too coincidental for the fact that its not working now to be wholly unrelated to the work I was doing with users and authentication?

This is the error I see in the log when I try to receive data from Couchbase server:

com.couchbase.lite.replicator.PullerInternal@3f7dcd29: Received invalid doc ID from _changes: {seq=1, id=_user/GUEST, changes=}

This is what I get when I query the public REST API for information about _changes:

{“results”:[
{“seq”:1,“id”:“_user/GUEST”,“changes”:}
],
“last_seq”:“1”}

Shouldn’t the “id” here be the id of a couchbase document? Why is it picking up “_user/GUEST” as an id and is there a way to make it stop? To me, that seems like a meta object that shouldn’t be in the changefeed, but I’m not sure. Is there something off in my configuration?

Also, here is the contents of my sync_gateway.json configuation file:

{
“interface”:“:4984”,
“adminInterface”:“:4985”,
“log”:[“HTTP+”],
“databases”:{
“phr_production”:{
“server”:“http://localhost:8091”,
“bucket”:“phr_production”,
“sync”:‘function(doc) {channel(doc.channels);}’,
“users”: { “GUEST”: {“disabled”: false, “admin_channels”: [“*”] } }
}
}
}

Additionally, in the sync gateway logs, I get a 404 error when trying to retrieve _local, this may or may not be related, I’m not sure.

2016-04-11T20:05:38.973Z HTTP: #002: GET /phr_production/_local/fbe09fefe5adf317a077e0607c9e493be8c9402e
2016-04-11T20:05:38.973Z HTTP: #002: → 404 missing (0.5 ms)
2016-04-11T20:05:38.979Z HTTP: #003: GET /phr_production/_local/64a07c9c0aa76109302ad23037a3787f01d4f934
2016-04-11T20:05:38.979Z HTTP: #003: → 404 missing (0.3 ms)

I’ve looked for similar posts on this topic but they’ve all either ended up being slightly unrelated or didn’t have enough information, so help would be greatly appreciated. Thanks!

2

The _user entries on the _changes feed are expected, and should be getting ignored by Couchbase Lite.

It’s possible that Couchbase Lite is logging an error, but that isn’t actually breaking replication or causing the problems you’re seeing.

The behaviour you’re seeing looks a lot like what you’d see for a user that doesn’t have security access to any documents. However, it does look like you’re granting the GUEST user access to the * channel (based on the config) - how did you re-enable the GUEST user? Re-adding to the config and restarting Sync Gateway, or through the REST API?

3

Through the config- I disabled the GUEST just by changing “disabled” to “true” in the config, I reenabled by changing it back to “false” and then restarting everything

4

You could try issuing a _changes request against the admin API, and compare that to what you’re seeing when calling the public API as GUEST.

You can also call :4985/[dbname]/_user/GUEST to validate the channel set for the GUEST user - the all_channels collection should show the full set of channels GUEST has access to.

5

Another question - what Couchbase Lite platform are you using when you see the “Received invalid doc ID” message?

6

I believe it’s Java, based on the class names in the log.

I just verified that the Obj-C (iOS/Mac) implementation simply ignores any docIDs in the changes feed that start with an underscore:

- (void) changeTrackerReceivedSequence: (id)remoteSequenceID
                                 docID: (NSString*)docID
                                revIDs: (NSArray*)revIDs
                               deleted: (BOOL)deleted
{
    // Process each change from the feed:
    if (![CBLDatabase isValidDocumentID: docID])
        return;
    ...

Sounds like the Java codebase should do the same.

7

CBL Android/Java printed a warning message for invalid document ID. Removed unnecessary log message that make user confused.

8

But if that’s not the issue, why am I unable to pull any data from couchbase and see it in my app?

I have another version of the app that’s rigged up to a staging server - same versions of everything, same config file - and it successfully pulls data that I can see in my app. I can’t figure out what’s different, the reason I latched onto this “received invalid doc ID” error was because I was seeing it when I unsuccessfully tried to launch from my production server, but not when I successfully launched from my staging server.

The only other thing I can see different is the HTTP log I see when I start Sync Gateway. My staging server shows a handful of POST requests to /phr_staging/_changes that all resolve quickly and then it stops. But my production server will hang for a while on a POST request to /phr_production/_changes, eventually complete it (after maybe 15-20 minutes) and then requery the same POST request without end, with each one taking about that much time to resolve.

Any thoughts on what could be causing that?

9

That’s normal. It’s called a “long poll” or “hanging GET” — a type of HTTP push notification. The client issues a request to _changes, but the server waits until there’s new data (i.e. a change has occurred) before sending a response. A long delay is expected if the client replicator has already ‘caught up’ (downloaded all current docs) and there are no new docs for it on the server.

So the real issue here is why you’re not able to pull data on the production server. Is the data on the two identical? If not, are you sure the user accounts and channels are set up such that the client has data accessible to it?

10

Ah, I see, but the problem remains that it doesn’t download any docs - database.getDocumentCount() always returns 0.

The data is not identical, although I did at one point try restoring the production server with the backups from staging and it still didn’t pull any documents. How do I determine for sure that the client has data accessible? I don’t think I really have my head around the channels, but I thought I was covering my basis by enabling the GUEST user with admin_channels set to ["*"]

11

Send a GET request to Sync Gateway with the path /dbname/_changes, authenticated as that user. Something like:

curl --user jens http://sghost:4984/dbname/_changes

This will list all docs accessible to that user.

12

How does this change when I’m only using the GUEST user? I ran that curl command without the user authentication, just the GET request to /dbname/_changes and on my working staging server, I got a whole long list of changes but on my non-working production server, all I see is this:

{“results”:[
{“seq”:1,“id”:“_user/GUEST”,“changes”:}
],
“last_seq”:“1”}

Why would that be?

13

@jhampton The latter would be the expected response if the GUEST user hasn’t been granted access to any documents.

To get to the bottom of why they don’t have access, you could try the approach mentioned I mentioned about - check the user’s channel information by calling:
:4985/[dbname]/_user/GUEST

14

Here’s what I see:

{"name":"GUEST","admin_channels":["*"],"all_channels":["!","*"]}

15

And you’ve definitely got documents in your production server? If you run :4985/dbname/_changes, you see the full set of docs?

16

No, I don’t see them when I run that. The only reason I know they exist is because I see them when I log on to the web console at :8091 and they’re there in the bucket.

17

Then your Sync Gateway must not be configured to use that bucket…

18

Then how do I make sure it IS configured to use that bucket? I just double checked that the name of my bucket in my sync_gateway.json file (“phr_production” as you can see in my original post where I pasted sync_gateway.json) matches the name of my bucket when I log into the Couchbase web console.

19

As far as I know Couchbase doesn’t currently support doing a backup and restore if you are using sync gateway. There is always the chance that the last_seq doc is changed while the backup is run and isn’t the right value on the restored db. When this happens to us our whole DB is a little screwed up and not suitable for use. We tried a few times to create a prod copy server for performance testing using backup and restore but it never worked properly due to this issue.

Also, did you backup the sync_gateway bucket and restore it? I know there are only 2 docs in there but we usually back that one up and restore it too. Not sure if it’s required though.

20

Hi all ,
I am also facing same issue (regarding pull replication). read out entire conversation but not really getting an idea , each discussion is going lil’ bit out of context. please help and suggest the way.I am facing same issue mentioned in this base topic.