{"id":1675,"date":"2012-11-13T09:00:00","date_gmt":"2012-11-13T08:00:00","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=1675"},"modified":"2021-09-08T03:21:27","modified_gmt":"2021-09-08T10:21:27","slug":"loading-json-data-couchbase","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/","title":{"rendered":"Loading JSON data into Couchbase"},"content":{"rendered":"

If you’re writing a web application, you’re probably already familiar with JSON documents. Couchbase supports JSON documents and sooner or later you will need to import some JSON documents into Couchbase Server<\/span><\/strong><\/p>\n

\n

But just because you inserted data into Couchbase doesn\u2019t mean that it goes directly to disk. Your data will first be inserted into the in-memory object managed cache and later in the background written to the disk asynchronously – decoupled completely from your action. <\/span><\/b><\/p>\n

\n

But what tools does a developer have to get a bunch of JSON data into Couchbase? This blog describes the <\/span>cbdocloader<\/span> tool in more detail. It saved me a ton of time by allowing me to import an entire Vancouver tree dataset that I was playing with.<\/span><\/b><\/p>\n

<\/div>\n
\n

Using cbdocloader <\/span><\/b><\/h2>\n

Following are the different command line parameters for the <\/span>cbdocloader<\/span> tool : <\/span>
\n\/opt\/couchbase\/bin\/tools\/cbdocloader -u Administrator -p password -n 10.3.2.54:8091 -b bucket_zip -s 10 output<\/span><\/p>\n

where \u00a0<\/span>
\n-s denotes the RAM quota in MB. This is an optional parameter (100 MB by default) <\/span>
\n-n is the node ip address<\/span>
\n-b the bucket name (If the bucket does not exist, an error will be thrown)<\/span>
\n-u username <\/span>
\n-p password<\/span><\/b><\/p>\n

The Vancouver Tree Dataset<\/span><\/b><\/p>\n<\/div>\n

The <\/span> City of Vancouver <\/a> added a new dataset of <\/span> street trees <\/a> to the city\u2019s open data catalog. This dataset includes a full address listing of all boulevard trees on the streets of Vancouver, along with the tree type and other characteristics.<\/span><\/b><\/p>\n

 <\/p>\n

Each JSON file in the dataset contains information for all the trees in a particular area. Using a <\/span> simple python script <\/a>, we split each JSON into multiple files to produce one JSON file per tree. We then loaded the data into Couchbase using the cbdocloader tool.<\/span><\/b><\/p>\n

Loading the individual JSON files into Couchbase<\/span><\/b><\/p>\n

The source documents fed into cbdocloader can be in a particular directory or in .zip format.<\/span><\/b><\/p>\n

cbdocloader to load JSON documents in a folder: <\/span><\/b>\/opt\/couchbase\/bin\/tools\/cbdocloader -u Administrator -p password -n 10.3.2.54:8091 -b bucket -s 1000 output<\/span><\/b><\/p>\n

cbdocloader to load a zipped folder (that contains json documents)<\/span>:<\/span>\u00a0<\/span><\/span>\/opt\/couchbase\/bin\/tools\/cbdocloader -u Administrator -p password -n 10.3.2.54:8091 -b bucket_zip -s 1000 output.zip<\/span><\/b><\/p>\n

Interesting Data Facts<\/span><\/b><\/p>\n

So can you guess how many trees are in the Vancouver Tree dataset?<\/span> <\/b>
\nClick <\/span> here <\/a> for the answer. <\/span>Clue:<\/span> It is the item count in the bucket shown.<\/span><\/b><\/p>\n

Do you know which Vancouver neighborhood has the tallest tree in the city?<\/span><\/b><\/p>\n

Now that you have loaded the data into Couchbase, try to write a simple view to figure out the answer. We will revisit this question in our view blog series so stay tuned folks!<\/span><\/b><\/p>\n

\n

—<\/span><\/b>
\nThanks to Abhinav for putting the screenshots together.<\/span><\/b><\/p>\n","protected":false},"excerpt":{"rendered":"

If you’re writing a web application, you’re probably already familiar with JSON documents. Couchbase supports JSON documents and sooner or later you will need to import some JSON documents into Couchbase Server But just because you inserted data into Couchbase […]<\/p>\n","protected":false},"author":2,"featured_media":13873,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"ppma_author":[8968],"class_list":["post-1675","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"acf":[],"yoast_head":"\nLoading JSON data into Couchbase - The Couchbase Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Loading JSON data into Couchbase\" \/>\n<meta property=\"og:description\" content=\"If you’re writing a web application, you’re probably already familiar with JSON documents. Couchbase supports JSON documents and sooner or later you will need to import some JSON documents into Couchbase Server But just because you inserted data into Couchbase […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\" \/>\n<meta property=\"og:site_name\" content=\"The Couchbase Blog\" \/>\n<meta property=\"article:published_time\" content=\"2012-11-13T08:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-09-08T10:21:27+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2022\/11\/couchbase-nosql-dbaas.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1800\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"The Couchbase Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"The Couchbase Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\"},\"author\":{\"name\":\"The Couchbase Team\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/764f4a6771ee19bc7af70b70a326fb93\"},\"headline\":\"Loading JSON data into Couchbase\",\"datePublished\":\"2012-11-13T08:00:00+00:00\",\"dateModified\":\"2021-09-08T10:21:27+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\"},\"wordCount\":451,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\",\"name\":\"Loading JSON data into Couchbase - The Couchbase Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"datePublished\":\"2012-11-13T08:00:00+00:00\",\"dateModified\":\"2021-09-08T10:21:27+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png\",\"width\":1800,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.couchbase.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Loading JSON data into Couchbase\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"name\":\"The Couchbase Blog\",\"description\":\"Couchbase, the NoSQL Database\",\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\",\"name\":\"The Couchbase Blog\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"width\":218,\"height\":34,\"caption\":\"The Couchbase Blog\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/764f4a6771ee19bc7af70b70a326fb93\",\"name\":\"The Couchbase Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/7befc37d02226b59499817eafdec60c3\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b4c18c758421903398e84d6c9560f319f39c665798d7d23e6a6f9dff8a8f984e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b4c18c758421903398e84d6c9560f319f39c665798d7d23e6a6f9dff8a8f984e?s=96&d=mm&r=g\",\"caption\":\"The Couchbase Team\"},\"description\":\"Jennifer Garcia is a Senior Web Manager at Couchbase Inc. As the website manager, Jennifer has overall responsibility for the website properties including design, implementation, content, and performance.\",\"sameAs\":[\"https:\/\/www.couchbase.com\"],\"url\":\"https:\/\/www.couchbase.com\/blog\/author\/jennifer-garcia\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Loading JSON data into Couchbase - The Couchbase Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/","og_locale":"en_US","og_type":"article","og_title":"Loading JSON data into Couchbase","og_description":"If you’re writing a web application, you’re probably already familiar with JSON documents. Couchbase supports JSON documents and sooner or later you will need to import some JSON documents into Couchbase Server But just because you inserted data into Couchbase […]","og_url":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/","og_site_name":"The Couchbase Blog","article_published_time":"2012-11-13T08:00:00+00:00","article_modified_time":"2021-09-08T10:21:27+00:00","og_image":[{"width":1800,"height":630,"url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2022\/11\/couchbase-nosql-dbaas.png","type":"image\/png"}],"author":"The Couchbase Team","twitter_card":"summary_large_image","twitter_misc":{"Written by":"The Couchbase Team","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#article","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/"},"author":{"name":"The Couchbase Team","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/764f4a6771ee19bc7af70b70a326fb93"},"headline":"Loading JSON data into Couchbase","datePublished":"2012-11-13T08:00:00+00:00","dateModified":"2021-09-08T10:21:27+00:00","mainEntityOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/"},"wordCount":451,"commentCount":1,"publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/","url":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/","name":"Loading JSON data into Couchbase - The Couchbase Blog","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","datePublished":"2012-11-13T08:00:00+00:00","dateModified":"2021-09-08T10:21:27+00:00","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#primaryimage","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2022\/11\/couchbase-nosql-dbaas.png","width":1800,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/blog\/loading-json-data-couchbase\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Loading JSON data into Couchbase"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/blog\/#website","url":"https:\/\/www.couchbase.com\/blog\/","name":"The Couchbase Blog","description":"Couchbase, the NoSQL Database","publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/blog\/#organization","name":"The Couchbase Blog","url":"https:\/\/www.couchbase.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","width":218,"height":34,"caption":"The Couchbase Blog"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/764f4a6771ee19bc7af70b70a326fb93","name":"The Couchbase Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/7befc37d02226b59499817eafdec60c3","url":"https:\/\/secure.gravatar.com\/avatar\/b4c18c758421903398e84d6c9560f319f39c665798d7d23e6a6f9dff8a8f984e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b4c18c758421903398e84d6c9560f319f39c665798d7d23e6a6f9dff8a8f984e?s=96&d=mm&r=g","caption":"The Couchbase Team"},"description":"Jennifer Garcia is a Senior Web Manager at Couchbase Inc. As the website manager, Jennifer has overall responsibility for the website properties including design, implementation, content, and performance.","sameAs":["https:\/\/www.couchbase.com"],"url":"https:\/\/www.couchbase.com\/blog\/author\/jennifer-garcia\/"}]}},"authors":[{"term_id":8968,"user_id":2,"is_guest":0,"slug":"jennifer-garcia","display_name":"The Couchbase Team","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/b4c18c758421903398e84d6c9560f319f39c665798d7d23e6a6f9dff8a8f984e?s=96&d=mm&r=g","author_category":"","last_name":"Garcia","first_name":"Jennifer","job_title":"","user_url":"https:\/\/www.couchbase.com","description":"Jennifer Garcia is a Senior Web Manager at Couchbase Inc. As the website manager, Jennifer has overall responsibility for the website properties including design, implementation, content, and performance."}],"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/1675","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/comments?post=1675"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/1675\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media\/13873"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media?parent=1675"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/categories?post=1675"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/tags?post=1675"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1675"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}