synchronizing views with a bucket
hey there.
is it possible to synchronize a subset of documents (eg. the documents within a view) with an other bucket?
so that the other bucket documents are always a direct subset of the "master" bucket?
if so, isn't that to much expensive in terms of perfomance? or does couchbase have any functionality to only create deeplinks to the documents instead of copying it?
thank you in advance!
____
i created a stackoverflow for this: http://t.co/Y8L69Vqq
sure :)
delsol: Yes, i also think that some well designed design documents should make any subsets unnecessary. But in my actual usecase i have to filter documents for many attributes like attr1 > 10, attr2 <= 20, attr3 = 30, attr4 between 1 and 2 order by attr5, attr4 etc.. so i thought it should be helpfull building a bucket subset with predefined filters to minimize the amount of existing documents, build some commonly used/filtered key/value pairs and filter the rest at application side.
scalabl3: I think your logic is sound. I think your concept of bucket subset, is, well what a view is. You mentioned the deeplinks in your comment with @JChris. You can do referential links, but they are not followed by Couchbase itself, but you can follow them client side. I am not sure bucket copy is needed for what you're trying to accomplish. Am I understanding correctly: You want to be able to constantly have a subset of documents that match certain characteristics based on the json k/v's reduce the dataset on the bucket based on these for additonal m/r?
Yes, I think that's exactly what I actually need. Let's say I want to have two sets (buckets) of documents S1 and S2. S2 is a subset of S1. Each set contains the same views V1, V2 and V3 since I want to be able to query any of them with the same logic/interface. In my case set S2 is build per user/company/store/whatever, in production there should be like 1000ish subsets S2 - to stay abstract let's call them S2a S2b and S2c.
The selection of documents which to be contained in any subset is done by a filtering instance (for example a view). Let's call these filtering instances F1 for filtering S1 to S2 hence F1a, F1b and F1c.
So with my actual knowledge of couchbase this results in the following design/view architecture: I've got the three "base" views to display V1,V2 and V3, and to realize S2a, S2b and S2c I must create the design views S2aV1, S2aV2, S2aV3, S2bV1, S2bV2, etc. (9 Views).
One could say "Well choose your keys wisely and you can avoid the sub views" but in my opinion this isn't that easy because of the following circumstances: In worst case the filter parameters change every minute and contain many WHERE IN constraints which could (at my actual point of view) not be handled efficiently querying k/v lists.
This leads to the following thoughts and the question I initially asked. If I use the same views in any subset (defined by a filter) shouldn't it be possible to build up an entity which helps me handling complex filtering? For example a function which is called during runtime while generating the view output? This could look like /design/view?filter=F1 or something like that.
Or do you have any other ideas to solve this problem? Or should I use SQL since it's more capable of handling frequently changing filters?
Hey, sorry I've been sick and didn't get a chance to reply back. I am researching solution possibilities for this, it's a very interesting paradigm of m/r/query on a mapped subset. I don't have an obvious solution at the moment, so have to do some thinking. :)
So i hope you are better now! :)
What i thought about the last days is the following solution: What about spending each document another internally handled attribute called "tag" (array). it's is filled by the previously mentioned function which has the possibility to tag("xyz"); instead of emit(id, val); a document. Each document is processed on creation and on update.
While filtering the view documents the additional GET parameter "?tag=xyz" forces the system to skip all documents which aren't tagged like the value in the "tag" parameter.
IMHO this approach offers couchbase developers a huge benefit and flexibility without changing the coucbase architecture that much? I'm not familiar with the couchbase core code and internal architecture, could anyone tell me how much effort this means? Or is this totally rubbish?
Within a view you can do a matching based on json param like:
user::scalabl3
{
"type": "user".
"username": "scalabl3"
}
View Javascript:
function (doc) {
if (doc.type == "user") {
emit(doc._id, null); //replace with actual code you want
}
}
But there is no way to tag/group certain documents based on keywords or even partial keys. Although I think this is an interesting concept that I will share with the team!
Yes I saw it on Stack Overflow, would you rather we continue the conversation here? :)
@scalabl3
Technical Evangelist
Couchbase Inc.