I have installed couchbase server 3.0.3 on ubuntu 12.04.
I have 1 bucket and it has 1 document of size ~570 KB.
i have created view to retrieve certain information (few fields of json doc) from the document.
when i click on show result or query through API (java SDK), result is empty.
when i reduce document size to say 15 KB or reduce the number of fields in view o/p( thats what i tried), same veiw returns result.
Is there any limit on size of document that view can process ? As far as i know couch base supports document size upto 20 MB.
Is there any configuration parameter that we can set to process big document in view ?
myex1, main group
_design/dev_Test, error mapping document
b26cd7ba-7e5a-43c9-9f59-ebbe2563ec2d:xyz:1 for view
test: too much data emitted: 1048640 bytes
I tried setting few parameters in /opt/couchbase/etc/couchdb/default.ini, like
indexer_max_doc_size = 0 ; set to unlimited
max_kv_size_per_doc = 10485760000 ; some large value. tried by setting it to 0
but no luck.
Note: I am emitting 10 K records for a single doc. Can this be a problem (i think) ? any work-around?
I’m sorry that this isn’t yet in our documentation.
The limits are set out in this bug report:
However, the default max doc limit is 1MB, I believe, Usually we’d suggest splitting documents if you can as Couchbase works more efficiently that way.
@vmx are you able to share any insight here?
Thanks for the reply.
We need to have few billions(~10 billions) of items stored in DB as key value.
What is better :
- store them directly in couchbase bucket as key-value, Or
- store them in view index ? ( i tried to emit key-value through map , but facing above mentioned issue).
What is difference between storage structure of view index Vs bucket ?
I would like to know performance implications of each of them.
Could you please comment on my reply?
Looking for your suggestion .
I’m not sure I understand the question. Presumably you need to store them as key-value documents, whether you use a view index or not. So, perhaps I’ve misunderstood, in which case please correct me.
With that many documents, my suggestion is that you attempt to find a way to do as much pure key-value access as you can. That’s not to say you shouldn’t use views but I wouldn’t recommend them as your primary index.
If you describe your use case a little more, I’d be happy to talk in a bit more depth.
Storing a standard key-value pair in Couchbase is the least resource heavy method and gives you:
- Strong consistency: read your own writes.
- You make full use of the cache (view indexes are stored on disk).
So, using key-value gives you sub-millisecond response times for cached items.
Views run at a slight delay, depending on how long it takes to run the view that builds the index. So, in effect they’re eventually consistent. There’s also a CPU and RAM impact from running the view query.
I hope that helps.
Hi @kbaswaraj did you have any luck updating those settings? It looks like we are having the same problem as described in this topic. Thanks.