Couchbase queue Disks and Index Services question

Hello,
I 've two conceptual questions about CB architecture.
1.When you does a SET operation, the ack to the client is sending immediatly, after CB write the doc in memory.
3.CB server replicate the data out to memory to other nodes. (If you have more nodes!)
2.At the same time it is put the data into a write queue to be persisted to disk.

If only have a node, and the disk queue with new DOcs is lost due a system failure.
I lose the new document?

Other question is about the distribution of Index services in isolated nodes from Data Services.
Which operations are located on this nodes? Index acces, Index storage, Index compactation ?
Are the data indexes located at the same nodes that data indexes?

Many Thanks
Arturo

Hi Arturo

If only have a node, and the disk queue with new DOcs is lost due a system failure.
I lose the new document?

Yes you’ll lose those documents

Which operations are located on this nodes? Index acces, Index storage, Index compactation ?

Here you’ll see what gets handled by the index service.

For a better overview of the architecture I recommend to read this:

Are the data indexes located at the same nodes that data indexes?

I’m not sure about this because I’m not officially from couchbase. But I think yes

Hello Mathias,
Thanks for the reply.

It’s a shame that nobody from Couchbase, can clarify this issue a little more.
Because customers want to know, and you lost data can be given in various failure scenarios that may occur.
It would also be important to know the behavior of Disks Queues, if configurable interval synchronization with the disk, or thresholds documents that generate I / O to disk.

Any way, Thanks!
Arturo

That’s not quite fair. It hasn’t even been 24 hours yet!

@mathias: thanks for pitching in to help!

Regarding your first question on node failure, note that as described that’s the default behavior: you would lose data. However, as a developer you have the option to specify durability requirements for the operation. Those allow your application to block until the requested durability is achieved, or you can decide what to do if it may not have been achieved. This can be done at both the replication and persistence level as covered in the doc.

Regarding indexes, there are a couple of different forms of indexes and they each have tradeoffs, as discussed in the documentation.

A quick summary…
Views: indexes are local to the data being indexed and can store reductions on interior nodes of the index, making them great for aggregations.
USING VIEWS indexes created through N1QL: are local to the data being indexed. Since you have the aggregate compute power of the whole cluster for index recalculations and you don’t need to move much data, this is great when you have fast changing data, but pay a penalty when range querying it because you’ll need to merge sort across all nodes.
USING GSI indexes created through N1QL: are wherever you declare the index to be or on a cluster assigned index node if you don’t specify one.

Each has automatic compaction.

As you can see, that’s quite flexible. I believe this is all well explained in the docs but if you find something missing, please hit the “Feedback On This Page” in the lower right and let us know.

I have to agree with @ingenthr. They have a lot to do and this is a forum like stackoverflow and not the live chat with the developers.

The documentations are very well written and I recommend you strongly to first read the complete docs and follow the guides and references

Hello Ingenthr,
Thanks for you note and references about durability options.
Excuse my haste to get an answer.
I just like to understand how things work!

Best Regards!
Arturo