Query matching

Hi,

no to complicate... write a map funciton like this

function (doc) {
 
  if (doc._id != 'banana')  
  emit(doc._id, doc);
}

Hi,

This is not a solution because the exclusion token can be anything, can be apples, oranges, or other fruit

My question is, is no way to make filtering in a view by a set of parameter?

in SQL I can do a where clouse like this: where fruilt not in ('banana') and color = 'red' but I see that cocuhbase view can't perform this feature

Hi,

i see. You looking for temporary views.. look here:

http://www.couchbase.com/docs/couchbase-api/couchbase-api-db_db-temp-vie...

Its up to you, sending your pre-parameterized request to the cluster. In C# the Hammock API for CouchDB had the capacity to write a linq-like query and executes that related map function on the server(s).

Ok, this resolves the problem creating a view on the fly, but in my case it's impossible to use, I've got 300 million documents in mi cluster and this operation is very expensive in CPU and disk access.

Couchbase has a fantastic concept to make the replication process, but talking about querying, mongodb has a powerful tool, allows to search by any field of document, but has a painfull replica process and global blocking operation

Querying from mongodb + replication from couchbase = The definitive store engine for noSQL

Yes, the query capacities of MongoDB are very well.

I agree in MongoDB you only query your example like this:

..
operators.put(“$all”, [“apple”,”orange”,..]);
BasicDBObject myquery = new BasicDBObject(“_id”, operators);
..

That’s nice and very simple.

Couchbase pronounced that they will develop something similar called unQL (look here http://www.couchbase.com/press-releases/unql-query-language)

Within our project we have similar needs to query dynamic data. In our case we know about the data structure to query only at runtime. I shortly share our thoughts at current stage of discussion relating querying..
First, i guess you wont your user to display 300 millions data in a grid when this guy is querying your data. So the user might be interested in the actually count of resulting data and maybe scrolling a bit through the results.

So scrolling through the resulting data isn’t a big challenge. In this case your webserver creates a view on the fly and limit its results for example for the first page to display (I suppose you ll use paging mechanics). Well, that data comes within a second and cause the map/reduce is parallelized on the database cluster you can push the continuing results to the client as soon as they ll be available. In practice this means for example, the user push the button to display the next page. Meanwhile there is enough data available to display to the user by Couchbase. I guess a user couldn’t be as fast clicking the “next page” button as the database generates the results ;)

Okay that’s a one solution for scrolling through resulting data. What’s about counting the query results? This gets a bit tricky, because we cant hide the loading process from the user.. Well but we can look into the logic an sql server is using in this case. For example MS SQL is storing the count information for each table separately. A request like “select count(fruitsname) from fruits” is a cheap operation for the sql server just looking up its pre-calculated value.. well that’s fine, but whats about giving him a more complex query (with a lot of where and join conditions).. Exactly that’s why sql can be so slow at the time, because the server first has to fetch all data (PLAN-phase) and then is calculating the results. But what can we do within an in-memory key/value storage instead? In assumption you know about all values your user can query.. you can update a counter for each entry at each CRUD operation. For example writing a “banana” document, results in incrementing the “banana” counter, deleting a banana-doc results in decrementing the banana-counter.. as you already assumed now we can calculate the resulting count very, very fast by transforming the query (your example) in counterApple + .. + counterOranage - counterBanana

For sure in your example “oranage” can have more attributes to query (f.e. the origin country).. so counterOrange has to be calculated separately. I guess you might say.. “oh gosh, that’s sounds like a lot of work”.. its not too hard, but you ll harvest a fast couting or query mechanism and your users will be astound. You can do the same by not just index count information.. you can index the documents ids the same way to get a query result.

There are many many more possibilities to index data, depending on your needs (look at starkey and endkey with wildcards is a powerful feature). But I agree its different to sql, because you have to care yourself for fast indexing.

I personally love the composition of an document store and key/value store, because I have the flexibility to index my data as I like (and don’t have to trust the index capacity of a database). A few years ago me and my team have been working for a company in the financial industry and puhhh, we had to handle a lot of data and.. it was mean.. additionally new data came every second. We have been part of a greater project and the solution we wrote was “just” calculating a lot of statistics about that incoming and existing data (the whole project was about what they calling an automated trading system where latency really matters). I wished we had something like Couchbase at that time. I experienced at that time that pre-indexing and pre-calculating things by the special needs of an application allow you to do extreme fast database queries. So i think that this is a big advantage over an sql database, that indexing is up to you.

So just a few thoughts to give you some ideas..

Happy coding!
Tom

Hi,

"There are many many more possibilities to index data, depending on your needs (look at starkey and endkey with wildcards is a powerful feature). But I agree its different to sql, because you have to care yourself for fast indexing."

Can you make an example with range keys and wildcards? and where I can read documentations about this feature?

Thanks

Yago