Thanks for the reply, Todd. So you’re saying instead of inserting one document representing one download to the customer, containing an array of all 10 or 25 or 300 keys that the customer did in that batch, to insert 10 or 25 or 300 documents, each with the same timestamp and actor and action, but with just one key? That seems plausible — it would increase our storage requirements, but perhaps not badly — but then how does that help the desire to pull back a list of all distinct keys retrieved by that customer over a time period? Doesn’t it still require that MapReduce return the large result set that it’s seeming to have a problem returning?
I was thinking of a way that the returning of the big result set could be avoided… in the code for our check for whether the customer is allowed to have the next little result set they’re asking for, we don’t actually need to know the full set of keys; we really only need to know if it is going to be more than the daily limit. So it’s possible we could only return the count of distinct keys over the time period, and the Couchbase-calling client would just compare that single integer against the configured daily limit, but I don’t think this would work if the Reduce had to be done in stages, i.e., “rereduce”. If it were guaranteed that all of the input documents would be sent into just one call of the Reduce function, then that one call could insert them all into the associative array, then at the end it could just return the length of that array, and it would be done. But this doesn’t work if the Reduce needs to be done in stages, with Reduce output being passed as input to another Reduce stage… each time that happened, you would lose the identity of the individual keys, and therefore you have no way to continue the DISTINCTivness all the way down to the final result. Is there some flag we can set (either at the general bucket level or in a specific client call) to force a one-step Reduce, i.e., force absolutely no Re-Reduce steps?