Details
-
Type:
Improvement
-
Status:
Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 2.0
-
Fix Version/s: None
-
Component/s: documentation
-
Security Level: Public
-
Labels:None
Description
Sorry if this is already started or in existence...
Need to be able to help users understand the differences in performance/scale/behavior between using views or managing indexes within k/v.
Something i just sent to a customer:
It's really a matter of understanding two things: the consistency your application requires between writes and those writes being seen in a view update, and the latency/throughput requirements you have for the queries of the views themselves.
Consistency: Because views are processed after data has been written to disk, there will be a delay between when you make a write and when it is available in the view. You can mitigate this with our "observe" function, but that will introduce a performance penalty on the write itself since you will be waiting for it to be written to disk...not something we'd recommend for every write. The amount of delay is really dependent on the resources available to the system and the amount of load you're putting on it. The nice thing is that this scales out very linearly by adding more nodes since each node only has to process the data that it is responsible for and more nodes means each node does less work.
Query latency/throughput: This is where the scale and performance questions really come in. Because a query will always have to gather data from all nodes of the cluster, it's not something that necessarily scales out with more nodes. Additionally, the query results are coming from disk. The nice thing here is that not all requests actually have to get served from disk, the OS will do a very good job of caching the disk IO and actually sending from RAM. However, it will be a bit variable and unpredictable.
Also:
-need specific use case examples
-need instructions on how to accomplish within k/v
Need to be able to help users understand the differences in performance/scale/behavior between using views or managing indexes within k/v.
Something i just sent to a customer:
It's really a matter of understanding two things: the consistency your application requires between writes and those writes being seen in a view update, and the latency/throughput requirements you have for the queries of the views themselves.
Consistency: Because views are processed after data has been written to disk, there will be a delay between when you make a write and when it is available in the view. You can mitigate this with our "observe" function, but that will introduce a performance penalty on the write itself since you will be waiting for it to be written to disk...not something we'd recommend for every write. The amount of delay is really dependent on the resources available to the system and the amount of load you're putting on it. The nice thing is that this scales out very linearly by adding more nodes since each node only has to process the data that it is responsible for and more nodes means each node does less work.
Query latency/throughput: This is where the scale and performance questions really come in. Because a query will always have to gather data from all nodes of the cluster, it's not something that necessarily scales out with more nodes. Additionally, the query results are coming from disk. The nice thing here is that not all requests actually have to get served from disk, the OS will do a very good job of caching the disk IO and actually sending from RAM. However, it will be a bit variable and unpredictable.
Also:
-need specific use case examples
-need instructions on how to accomplish within k/v