Problem with "Grant for channel is after the current sequence - skipped for this iteration."

Hi all,

I’m hoping I can get some help here with something strange I’m seeing since moving to sync gateway 1.4. Everything works fine in our client when using SG 1.3. When we move to SG 1.4 I see the following 2 lines in my logs that I believe is causing an issue for me:

Auth: Saved _sync:user:firstfirstfirst.7SgDfPa1SFyWPpDu-xe0DTD_: {"name":"firstfirstfirst.7SgDfPa1SFyWPpDu-xe0DTD_","admin_channels":{"rwvKaVUn34aPjFy614es4Uva37":10941},"all_channels":null,"sequence":10941,"passwordhash_bcrypt":"JDJhJDEwJG5CZmRwZHp4RWVmQk5yNmJJMUdDNGVIdWtGcWtXeXgzQVhsRnlIR2Q1cWZ0UDljd2tBMU9X","rolesSince":{}}
Grant for channel [rwvKaVUn34aPjFy614es4Uva37] is after the current sequence - skipped for this iteration.  Grant:[10941] Current:[10940]   (to firstfirstfirst.7SgDfPa1SFyWPpDu-xe0DTD_)

I initiate a one-shot pull replicator from our client. I basically wait for that replicator to stop (at which point I assume that I have all the data). Now when I see the above logs the replicator stops without any errors, but it also has not pulled any data back so when I try to proceed the client encounters an error. I believe this is because the “Auth” operation above has incremented the sequence for this channel and the SG wants the cache to be caught up before it proceeds with sending the data. However as soon as the cache is caught up, I don’t see any data being sent by the SG.

I realize that my logic above is not the best way of doing things. The supported method according to the documentation is to determine the last sequence and not proceed until the revision noted by the last sequence is accounted for, but when the replicator stops due to the above problem the last sequence I receive is null as indicated by this Couchbase warning:

W/Sync: Error converting lastSequence: null to long. Using 0

I’m sure from the logs above that the SG realizes the last sequence is “10941”, so why do I get a null value back?

My question is basically how can I successfully perform a one-shot pull of my data when the SG is in a state where it is waiting for the cache to be updated?

The only thing I can think of is to keep sending one-shot requests until I see a non-null last sequence value and then use that to obtain all the data. I wouldn’t like to proceed this way since it will cause a lot of overhead in my application to have to keep retrying and I have no guarantee that I will ever see a non-null last sequence depending on the situation .

Thanks for taking the time to review my question!

The logging message mentioned - “Grant for channel … is after the current sequence” means that the document doing the channel grant hasn’t yet been buffered and added to the channel cache. The channel backfill associated with that new channel grant won’t be processed until that happens.

This change was made 1.4 to ensure that any documents earlier than the granting sequence are sent to the client before the backfill kicks off.

A one-shot replication always needs to be limited to what’s been cached, to ensure there aren’t any missed sequences.

It sounds like your one-shot replication is being kicked off immediately after a change to the user document to do a channel grant, and Sync Gateway hasn’t yet received notification of the channel grant update over the server mutation feed.

However, I don’t think there should be a case where the last sequence is returned as null. If you can share more details on the request/response where you’re seeing the last_sequence is null, that would help.