{"id":9061,"date":"2020-08-12T08:42:37","date_gmt":"2020-08-12T15:42:37","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=9061"},"modified":"2025-06-13T20:19:28","modified_gmt":"2025-06-14T03:19:28","slug":"external-datasets-extend-your-reach-with-couchbase-analytics","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/external-datasets-extend-your-reach-with-couchbase-analytics\/","title":{"rendered":"External Datasets: Accessing AWS S3 in Couchbase Analytics"},"content":{"rendered":"

Introduction to external datasets<\/h3>\r\n

Couchbase is very excited to announce its new \u201cExternal Datasets\u201d Analytics Services<\/a> feature in the latest Couchbase Server 6.6 release<\/a>. External datasets empower customers to access externally stored data in real-time from Amazon Web Services (AWS) Simple Storage Service (S3) and to combine S3-resident data with existing Couchbase data for analysis.<\/p>\r\n

Customer use case<\/h3>\r\n

Some customers use AWS S3 to reduce storage costs and store data (e.g., multiple years of historical data, offline business data for machine learning, product reviews, etc.). They have expressed a desire to combine, query, and utilize S3 data in real-time to make this data available to business users for analytics. You can read more about other Analytics use cases <\/span>here<\/a>.<\/span><\/p>\r\n

How do external datasets work?<\/h3>\r\n

External datasets provide the ability to dynamically query and analyze data residing in AWS S3, allowing users to easily combine data in real-time from both inside and outside their Couchbase analytics nodes. This is achieved in three simple steps:<\/p>\r\n

    \r\n
  1. Set up an S3 link by using a REST API call<\/a> or the command-line interface (CLI)<\/a><\/li>\r\n
  2. Create an external dataset on the S3 link<\/li>\r\n
  3. Query the dataset using SQL++<\/a> (or your favorite BI tool)<\/li>\r\n<\/ol>\r\n

    Let\u2019s walk through a simple example. iMaz, an e-commerce company, sells consumer products online. Their order, product, and user data are stored on a Couchbase cluster with both data and analytics services (on separate sets of nodes in the cluster). They use the Analytics Service to run ad hoc and complex queries to analyze their business. iMaz also stores their product reviews on AWS S3, and they would like to combine and analyze the top 3 most highly rated products using the Couchbase Analytics Service.<\/p>\r\n

    Sample product data:<\/p>\r\n

    [\r\n{\r\n\"id\": \"Product_1\",\r\n\"docType\": \"Product\",\r\n\"productId\": 1,\r\n\"price\": 811.76,\r\n\"salePrice\": 70.14,\r\n\"productName\": \"Ergonomic Cotton Ball\",\r\n\"desc\": \"Plastic fused metallic Ergonomic Cotton Ball\",\r\n}\r\n]<\/pre>\r\n
    Sample review data:<\/p>\r\n
    {\r\n\"id\": \"Review_0001764a17a844279a2227e137cc4e36\",\r\n\"docType\": \"Review\",\r\n\"reviewId\": \"0001764a17a844279a2227e137cc4e36\",\r\n\"productId\": 1,\r\n\"userId\": 5862,\r\n\"reviewerName\": \"M. Schaefer\",\r\n\"reviewerEmail\": \"...@mmail.com\",\r\n\"rating\": 5,\r\n\"title\": \"Works well and meets expectations.\",\r\n\"review\": \"Product works great and will buy one more for my extended family.\",\r\n\"reviewDate\": 1597273484\r\n}<\/pre>\r\n
    Let\u2019s follow the three steps from above with sample setup code along with a SQL++ query.<\/p>\r\n
    Step 1: Set up S3 link<\/h4>\r\n
    We\u2019ll create an S3 link using a REST API call<\/a>. (Alternatively, you can use the CLI to create S3 links<\/a>.). We\u2019ll need to provide:<\/p>\r\n