Is stale=false supported?

I'm having the same problem. Did you end up finding a solution?
Thanks.

I did not. I tried the Java API but no luck. My only solution was to incorporate the fields into the key of a new bucket and rely on that.

stale=false is supported. There is a setStale() function in the Query class in the java client. Here is the problem it sounds like you are having. When you set a value and you get a response back from the server that the set was a success it only means that the value is in memory in Couchbase. The problem is that the value must hit disk for you to be able to do a query with stale=false and see it in the view results.

Right now in the developer previews there is no good way to do this and the work around is to have sleep statements in your client code. This is obviously bad application design to use sleep statements so we have come up with a new command called observe with will allow your client to know when certain keys (the ones you want to observe) have different actions happen on them, one of which is hit disk.

The observe command is being implemented in the server and client sdks right now and will definitely be ready for the Couchbase 2.0 release. There will be more documentation about this command and the specific issues you guys are facing as soon. I will post back here when we get that documentation published.

Cool. Thanks for the prompt reply.

Mike, we're having the same problem and just wanted to clarify: what is stale=false is suppose to do? Doesn't 'stale' mean that there is data in memory that will not come back in a view because it hasn't been written to disk yet? If that's the case it would seem stale = false means exactly "wait until the data was written to disk."

So I understand there may be this other "observe" command, but I'm left not understanding what 'stale' and stale=false mean.

TIA!

Stale in this case means do you want stale data? If you set stale=false this means that you want the most up to date data so the view will make sure all of the most recent data has been indexed before sending you the result. Other options for stale are "ok" and "update_after". The "ok" option means that you don't mind if the result doesn't reflect the most recent data and "update_after" means that you are ok getting old results, but you want the view to get up to date immediately after Couchbase sends your request so that next time you do a query the results will be up to date.

The observe command is used to guarantee higher levels of durability. If you do a set right now the set will return as soon as the data is written into memory. In the background Couchbase will take care of replicating the data to other nodes and persisting the data to disk. Let's say for example your data is really important and you really want to make sure it is written to disk. You can use the observe command and it will only return to you once the item has hit disk.

So how does this apply to views?

Views are indexed off of disk, but a normal set returns when the data is written to memory. This means if you set some data that should appear in a view response and then you query that view you will not be guaranteed to see the data you just set in the view response with stale=false as a parameter. This is another reason for using observe. So when you do a set and need that data to be in a view response you should observe that the data has hit disk before sending your query.

Let me know if you have any other questions.

So let me restate that and tell me if this is correct. I think what you are saying is, for a newly set document there are three related states is can be in. 1) In memory, 2) on disk but NOT yet indexed as part of a given view's indexing, 3) on disk and indexed. "stale" has nothing to do with in memory verses on disk. 'stale' data are essentially in state #2. Non-stale data is in state 3.

When you set "stale=false" you are forcing an indexing on the view (and as I understand it, all views in the same design), against disk data only. So you are forcing all data in state #2, to state #3, before the view data is assembled.

So then, the observer functionality will allow us, on a given view, to know if there is data that has shown up in memory since the last indexing on that view, but not been persisted to disk? So I'm assuming some kind of code like this (setting aside performance for now) would guarantee up to the second view data; is this what you are coding in observer (java):
//***********
Query query = new Query();
query.setKey(key);
query.setReduce(true);
query.setIncludeDocs(true);
query.setStale(Stale.FALSE);
query.setObserver( Observer.onDisk ); //Or something like this? //******the new call?

View view = client.getView( designIn, viewIn );
result = client.query( view, query ).getMap();
//**********

So the point is...I don't care about syntax or even the precise mechanism, but as of release 2.0 we'll be given some way, before a specific view call, to know all in memory data at that point in time has been written to disk (or even more fancy, all data received up to a specified period of time before the view call; that way we could be sure we were getting data within an acceptable timeframe, but be more likely that the data is indeed already available on disk)?

There is one mistake in your response. The observe command works at the key level. A sample of how doing an observe might look like this:

int persisted = 1;
int replicated = 0;
client.set("key1", "value1");
client.set("key2", "value2");
client.set("key3", "value13");
client.observe("key2", persisted, replicated);
 
Query query = new Query();
query.setStale(Stale.FALSE);
 
View view = client.getView( designIn, viewIn );
result = client.query( view, query ).getMap();

Then this would only make sure that key2 was in the query. Also, the stale=false argument will only update the index on the view that you are doing the query on.

Other than that it sounds like you have the right understanding of how things work.

Let me know if you have any other questions.

Really appreciated! Actually yes, one more: is there any guarantee that all docs have been persisted chronologically prior to the key we observe? Or would we in theory have to test every key? I guess if no guarantee...I'd be kind of lost on how it would help. Not trying to be obtuse but to understand how it would be used.

Thanks again!

You would have to test every key (updates to documents are not guaranteed to be persisted in order), but there are thoughts on an efficient way to do so in batch.

Could you look over what we have on the wiki and perhaps post some comments there?
http://www.couchbase.com/wiki/display/couchbase/Observe