Is Couchbase good for high write and high read with need of sorting and filtering TBs of data daily

Couchbase Server
1

Hi,

Is Couchbase good for high write and high read with need of sorting and filtering TBs of data daily?.

High write 200mb/minute. and 1000 user / second filtering and sorting data from 100TBs of data.

Is couchbase suitable for such type of use case?

2

Hi,

This is a difficult question to answer :slight_smile: But I will give my perspective.

I think we need to look at your system’s performance requirements before we can answer the question.
A few questions to start:

  1. What’s your read/write latency requirement?
  2. What’s your filtering/sorting requirement for 100TB of data? Do you need replicas?
  3. How many RAM/Core/Disk you would like to use for such system?

Couchbase is a high performance system but I think to fully evaluate your use cases, more details would be helpful.

Thanks,
Qi

3

I am just evaluating for learning purpose. Suppose if i had daily 1 TB data generated and If i want to do search sort filter from 1000s of TB, is couchbase suitable for such application.
Assume that i can afford expensive servers. Replica is needed. Assume I am filtering by 10 diff keys and sorting by one of them key. Assume that I need 100 filtered records from 1000TBs of data in less than 1 second.

Thanks

4

Assume we can have enough RAM/CPU/network bandwidth, my answer would be yes.

On a separate, on Couchbase 4.0, we introduced global secondary index and N1QL.
Based on your use case, it seems to be a good fit to take advantage of these two features.
Doc here:
http://docs.couchbase.com/4.0/indexes/query-intro.html
http://docs.couchbase.com/4.0/n1ql/index.html

Thanks,
Qi

1 Like