automatic full indexing is being triggered for dev_ views

Filipe Manana added a comment - 04/Jul/12 5:41 AM I don't think I have ever heard a decision/requirement from PM regarding this. I'm pretty sure, if prompted for this, they'll ask to make it configurable (and maybe a ton of more parameters).

Aleksey Kondratenko added a comment - 04/Jul/12 9:53 AM I think if there would be option to disable automatic updating of some particular set index, ns_server could use it to prevent dev_ views being sole reason of automatically building full indexes

Filipe Manana added a comment - 04/Jul/12 10:16 AM Do you want to take over all of the automatic index updating (moved to ns_server)?

Aleksey Kondratenko added a comment - 04/Jul/12 10:18 AM No. But that's option as well.

Aleksey Kondratenko added a comment - 04/Jul/12 10:19 AM I mean I'd fully support it if you think it makes sense from the point of overall architecture and we can agree on APIs used.

Filipe Manana added a comment - 04/Jul/12 10:26 AM I'm happy to pass it to you. I just add you an API entry point in couch_set_view, as: -spec update(SetName::binary(), DDocId::binary(), MinNumberOfChangesToTriggerUpdate::non_neg_integer()) -> ok. Should be enough for now. Like it is now, it should be configurable to be completely disable as well (useful for my own testing in the whole stack). Let me know, and we officially do the switch.

Dipti Borkar added a comment - 05/Jul/12 1:42 AM what is the impact of not automatically building dev indexes besides them being behind on updates? does this have to be on the full data set?

Filipe Manana added a comment - 05/Jul/12 6:00 AM The impact might be unexpected/inconsistent behaviour for some users. I don't think there's a need to have this feature in the view-engine, neither do I have special technical interest for maintaining it there. Alk, do you still want to own this?

Aleksey Kondratenko added a comment - 05/Jul/12 12:53 PM It's not about want IMHO. Yes we'll own it. But a this time we don't have time to actually take ownership. Perhaps interim 'fix' can be just disabling of this autoupdating of active/replica indexes.

Volker Mische added a comment - 06/Aug/12 11:23 AM I don't like the idea of dev views behaving much different than production views. Having them more and more different is dangerous as people first use dev views and they learn to use them with some assumptions that are different from production views. And it is also overhead for the docs team, as all differences must be documented. And not to forget about potential bugs. It's good to have the same code paths/same calls for both, else, you might end up with "my dev view is working, but my production view not". It's always a good idea to keep complexity low and I think this is a case where the advantages are lower than the risks we take.

Perry Krug added a comment - 06/Aug/12 11:33 AM That makes good sense Volker, perhaps we can come to some compromise to prevent dev views from automatically kicking of the indexing while a user is trying to work with and create the view?

Volker Mische added a comment - 06/Aug/12 11:35 AM I forgot to quote Peter about this topic [1]: "We don't need a distinction between dev and prod views, this is only to change all indexes on a cluster level." [1] http://www.couchbase.com/issues/browse/CBD-423?focusedCommentId=33573&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-33573

Farshid Ghods added a comment - 06/Aug/12 11:36 AM I agree with Volker on this. having two different behaviors for production and dev views increase the complexity and requires additional testing when view mode is updated from dev->production or vice versa so keeping these two in sync is preferred.

Perry Krug added a comment - 06/Aug/12 11:42 AM Just to be clear, I don't disagree with those sentiments. I'm simply raising the fact that many users will NOT want the indexes to update automatically...especially in dev mode. So hopefully we'll have a way of tuning the threshold or turning it off altogether for all view processing.

Volker Mische added a comment - 06/Aug/12 11:45 AM Perry, thanks for chiming in. This is a typical issue that you can really comment on from a end users perspective. I think your proposal makes it even more difficult. Things need to be simple to understand and not depend on certain other things. This is called "state" in programming and if you want to know more about why it adds so much complexity/problems/bugs please read [1]. It's bad to have things like "it does this, but only in case you are currently doing y while setting z is a". Hence my proposal is, having even the dev views updated automatically, because they behave like the production views. The solution to not taking to much resources is simple, just don't have many of those. I don't think on a production system, you would have your developers creating zillions of dev views. I would expect to have the development of views taking place in some dev environment, then they get migrated to the production system as a dev view, and shortly (basically immediately) afterwards, you'll deploy them as production views. And during development you will be happy that they behave like a production view. You just need to read the documentation once and there it will state a warning that you shouldn't have too many dev views in your production system. [1] http://shaffner.us/cs/papers/tarpit.pdf

Aleksey Kondratenko added a comment - 06/Aug/12 11:47 AM Hm. Folks. What's the point of dev views? Ask this question to yourself. Or imagine Frank (year ago, I think that was his decision) and ask him. _Not_ automatically indexing dev views is _exactly_ the point of having them. Peter's comment was about scope of ticket.

Aleksey Kondratenko added a comment - 06/Aug/12 11:47 AM And btw this is automagically addressed by Aliaksey's work on index auto-update API

Perry Krug added a comment - 06/Aug/12 11:54 AM I like automagic...is there a design spec I can read up on for that? Volker, totally agree with your comments. It would be nice to have a per-view auto-update that can be turned on and off, applying to both dev and prod so they all behave the same way.

Volker Mische added a comment - 06/Aug/12 12:50 PM Alk, thanks for the info. I must admit that I don't really know the point of dev views. When I had to deal with them the last time, the difference was that they only work on a subset. So yes, I think I reached one goal already, which is: think twice. If no auto-updates is the point of dev views it should of course be coded like that (and of course there shouldn't be a way to enable it, else you end up again in feature creep :) Perry, the configuration (which also means disabling the auto update) is ticket CBD-423. Though again, think twice if a per design document setting is a must have (or a nice to have), or if the current global one is just enough. The ticket only says "it would be good to have it per design document". But what we really need here is the reason *why* it is a good idea. Have customers asked for it? What is their use case? This already states the solution and not the problem it tries to solve. Developers are there for solving problems, not coding solutions.

Filipe Manana added a comment - 15/Aug/12 12:24 PM This is now done and controlled by ns_server.