{"id":5957,"date":"2018-11-14T22:29:16","date_gmt":"2018-11-15T06:29:16","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=5957"},"modified":"2023-06-12T23:53:27","modified_gmt":"2023-06-13T06:53:27","slug":"its-the-workload-stupid","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/its-the-workload-stupid\/","title":{"rendered":"It’s the Workload, Stupid!"},"content":{"rendered":"

As an application architect, eventually, you\u2019d have to choose the database or database as a service (DBaaS<\/a>) to power your newest application or a micro-service. Selecting one of the databases among relational databases was easier. \u00a0The use cases were roughly divided into OLTP and OLAP (decision support). The workload differences between OLTP and OLAP were well known. OLTP workloads consist of short transactions<\/a> on few random rows, expecting millisecond responses on pre-compiled queries; OLAP workloads consist of data loads, long-running queries scanning millions of rows of a fact table of a star\/snowflake schema. Each had the performance benchmark and TCO well defined, measured and audited via<\/span> TPC benchmarks<\/span><\/a>. \u00a0You can make use of these numbers, approximate your workload, understand the needs and capabilities match on other fronts like administration. \u00a0<\/span><\/p>\n

Then, there are No-SQL databases. NoSQL databases were invented to handle the web scale performance of operational applications. \u00a0It had to be elastic to handle the scale and tolerate nodes going down (aka partition tolerance). That sparked the innovation to create databases on a variety of data models and use cases. There are databases for JSON, graphs, time-series and more. From Azure databases to ZODB, from Couchbase to Cassandra. \u00a0MongoDB to TiDB, spatial to JSON databases — so many different kinds of databases. In fact,<\/span> NoSQL-databases.org<\/span><\/a> lists 225 databases and DBaaS<\/a> as of November, 2018.<\/span><\/p>\n

E-Commerce applications need to generate the sales report, shopping cart applications need to report outstanding shopping carts, etc. Every application makes progress on the workflow on behalf of its user or customer. These operational queries can be simple key-value operations, short-range queries or complex search queries in NoSQL. This workload is the bread and butter of most businesses.\u00a0\u00a0<\/span><\/p>\n

The solutions for the high-volume transaction and high-volume analysis is seemingly contradictory.\u00a0 The lookup operations or queries (e.g. searching for flights) requires very efficient lookup, quick data flow back to the application. \u00a0Milliseconds matter.\u00a0<\/span><\/p>\n

\u00a0\u201cBy the way, most workloads are mixed workloads\u201d —\u00a0<\/span>Larry Ellison<\/span><\/a>, Founder & CTO of Oracle.<\/span><\/p><\/blockquote>\n

 <\/p>\n

\"\"<\/p>\n

Source: BI Research<\/span><\/p>\n

Recognizing the need to address mixed workloads, OLTP databases have added features for complex queries (e.g. hash joins, window functions), \u00a0NoSQL databases have added SQL. SQL databases have added JSON. We may be reaching the state of the<\/span> paradox of choice<\/span><\/a>!<\/span><\/p>\n

Until we have a universal solution, we\u2019d need to evaluate and stitch different products and services to provide the solution to support the business workload and business outcome. \u00a0E.g. You could run SQL Server for the OLTP and Teradata for OLAP. Use Couchbase for an e-commerce application and Hadoop for machine learning. First, you\u2019ll have to drill down the workload to understand the operations within a workload.<\/span><\/p>\n

So, what\u2019s a workload? \u00a0<\/span><\/p>\n

The article \u00a0<\/span>What\u2019s your definition of the workload?<\/span><\/a> gives examples database workload from various angles. Application designers may define by the application SLAs at a given concurrency. \u00a0DBAs may define the workload by the CPU, memory usage, I\/O throughput, etc. The advice is to understand the resource usage and tie them to higher level entities like the queries, users, and applications.<\/span><\/p>\n

TPC-C<\/span><\/a> describes the benchmark workload and measurement in the following way:\u00a0<\/span>The most frequent transaction consists of entering a new order which, on average, is comprised of ten different items. Each warehouse tries to maintain stock for the 100,000 items in the Company’s catalog and fill orders from that stock. However, in reality, one warehouse will probably not have all the parts required to fill every order. Therefore, TPC-C requires that close to ten percent of all orders must be supplied by another warehouse of the Company. Another frequent transaction consists in recording a payment received from a customer. Less frequently, operators will request the status of a previously placed order, process a batch of ten orders for delivery, or query the system for potential supply shortages by examining the level of stock at the local warehouse. A total of five types of transactions, then, are used to model this business activity. The performance metric reported by TPC-C measures the number of orders that can be fully processed per minute and is expressed in tpm-C.<\/span><\/i><\/p>\n

For modern applications, as an architect, you\u2019ll have to understand all of the application operations, patterns, and their SLAs. Application performance isn\u2019t measured by a simple benchmark, but its ability to perform at the business scale over a long period of time<\/span><\/em>. Adding the period of time<\/span><\/em> factor means, you\u2019ll have to consider the scale, SLA and cost of the infrastructure on a slow summer day and the day 100 million customers on your site trying to add their favorite toy to the shopping cart and buy it.<\/span><\/p>\n