{"id":7155,"date":"2019-07-09T05:08:36","date_gmt":"2019-07-09T12:08:36","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=7155"},"modified":"2025-06-13T23:42:49","modified_gmt":"2025-06-14T06:42:49","slug":"how-to-use-n1ql-to-migrate-oracle-data-into-couchbase","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/","title":{"rendered":"How to use N1QL to migrate Oracle data into Couchbase"},"content":{"rendered":"<p>Getting the Couchbase server to run on your PC or Mac involves a few simple steps to download the software and spin up a cluster with all the Couchbase services that you need (<a href=\"https:\/\/docs.couchbase.com\/server\/6.0\/getting-started\/start-here.html\">https:\/\/docs.couchbase.com\/server\/6.0\/getting-started\/start-here.html<\/a>). Sample buckets are available with the software for you to begin using the product in a matter of minutes.<\/p>\n<p>If you need to migrate your relational database to Couchbase, there are connectors available (<a href=\"https:\/\/docs.couchbase.com\/server\/6.0\/connectors\/intro.html\">https:\/\/docs.couchbase.com\/server\/6.0\/connectors\/intro.html<\/a>) that would enable you to achieve the goal. However if you are familiar with both RDBMS and Couchbase database tools, you could leverage your database data exporting tool, then use Couchbase <strong>cbimport<\/strong> to load the data into a Couchbase bucket.<\/p>\n<p>With either approach, you will still need to make a few decisions, because of the differences between the databases:<\/p>\n<ol>\n<li>RDBMS tables vs. Couchbase buckets.<\/li>\n<li>Database primary keys and bucket document keys.<\/li>\n<li>Last but not least, how to transform your relational database schema to a Couchbase JSON document database.<\/li>\n<\/ol>\n<p>In this blog, I will discuss these differences, and outline the different strategies that you could consider to transform your relational database schema to the Couchbase NoSQL database. While the migration technique can be generalized for many RDBMS, the actual process of querying the source database leverages its specific REST APIs. For this reason, I will use Oracle and its sample HR schema as the source data, and leverage its REST Data Services for the data extraction.<\/p>\n<h4>What tools do you need<\/h4>\n<p>The migration techniques in this blog leverage the basic services of the Oracle database and the Couchbase N1QL Query language. You do not need anything else.<\/p>\n<h4>How to use these migration techniques<\/h4>\n<p>The provided N1QL scripts address the first two issues of document type and document key. For the transformation of the relational schema to Couchbase JSON database, the scripts cover these three scenarios:<\/p>\n<ol>\n<li>Direct mapping from table to document type. No transformation is involved.<\/li>\n<li>Denormalization of parent into child object.<\/li>\n<li>Denormalization of child objects into the parent as an array field.<\/li>\n<\/ol>\n<p>The above scenarios, with the document type and document key solutions, should cover the majority of the use case to transform a relational schema to a JSON document database.<\/p>\n<h4>The prerequisites<\/h4>\n<ol>\n<li>Access to an Oracle Database Server with the HR sample schema.<\/li>\n<li>Access to a Couchbase Database, where you need to create a bucket <strong>cbhr<\/strong> as the target for the Oracle HR schema migration.<\/li>\n<\/ol>\n<h4>The steps<\/h4>\n<ol>\n<li>Enable your Oracle schema for REST Data Services access.<\/li>\n<li>Set up a Couchbase server and configure the bucket to receive the Oracle HR data.<\/li>\n<li>Decide on the data model transformation techniques for your migration requirement.<\/li>\n<li>Edit and execute the N1QL script to migrate the data.<\/li>\n<\/ol>\n<h3>Oracle Database with the HR schema<\/h3>\n<p>The Oracle HR schema is available in your Oracle installation. Please follow the Oracle documentation to deploy the schema. <a href=\"https:\/\/docs.oracle.com\/cd\/E11882_01\/server.112\/e10831\/installation.htm#COMSC001\">https:\/\/docs.oracle.com\/cd\/E11882_01\/server.112\/e10831\/installation.htm#COMSC001<img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-7161\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM.png\" alt=\"\" width=\"971\" height=\"572\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM.png 1880w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM-300x177.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM-1024x604.png 1024w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM-768x453.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM-1536x905.png 1536w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM-20x12.png 20w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.26.40-PM-1320x778.png 1320w\" sizes=\"auto, (max-width: 971px) 100vw, 971px\" \/><\/a><\/p>\n<h3>Enable Oracle schema for REST services<\/h3>\n<p>By default, the Oracle REST data service is not enabled on schema. You will need to enable this for the HR schema.<\/p>\n<p>Login as the user who will issue the REST calls. For this example, login as <strong>hr<\/strong>, and run the following script.<\/p>\n<pre class=\"lang:plsql decode:true \">BEGIN\r\n   ORDS.ENABLE_SCHEMA( p_enabled =&gt; TRUE\r\n                     , p_schema =&gt; 'HR'\r\n                     , p_url_mapping_type =&gt; 'BASE_PATH'\r\n                     , p_url_mapping_pattern =&gt; 'hr'\r\n                     , p_auto_rest_auth =&gt; FALSE);\r\n   commit;\r\nEND;<\/pre>\n<p>Reference: <a href=\"https:\/\/blogs.oracle.com\/oraclemagazine\/get-your-rest-post-your-sql\">https:\/\/blogs.oracle.com\/oraclemagazine\/get-your-rest-post-your-sql<\/a><\/p>\n<p>Verify that you can query your Oracle with a REST call.\u00a0 https:\/\/&lt;your-oracle-server&gt;:8080\/ords\/hrrest\/employees\/<\/p>\n<p>Note: By default, Oracle REST Enabled SQL service is turned off. To configure REST Enabled SQL service settings, see <a href=\"https:\/\/docs.oracle.com\/database\/ords-17.3\/AELIG\/configuring-REST-data-services.htm#AELIG7195\">Configuring REST Enabled SQL Service Settings<\/a>.<\/p>\n<h3>Prepare your Couchbase server<\/h3>\n<p>There are two steps that you need to complete in the setup of the Couchbase server.<\/p>\n<ol>\n<li>Create a bucket with the name <code>cbhr<\/code>. The size of the bucket will depend on the volume of the data that you plan to migrate.<\/li>\n<li>Ensure that you enable the CURL() Function Access in the Couchbase Server setting.<\/li>\n<\/ol>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-7185\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-09-at-7.32.05-AM.png\" alt=\"\" width=\"516\" height=\"157\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-09-at-7.32.05-AM.png 516w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-09-at-7.32.05-AM-300x91.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-09-at-7.32.05-AM-20x6.png 20w\" sizes=\"auto, (max-width: 516px) 100vw, 516px\" \/><\/p>\n<p>3. You will also need to create a primary index on the <code>cbhr<\/code> bucket to allow the N1QL to query the bucket as part of the migration.<\/p>\n<div>\n<pre class=\"lang:default decode:true \">CREATE PRIMARY INDEX `#primary` ON `cbhr`<\/pre>\n<p>&nbsp;<\/p>\n<\/div>\n<h3>Data Model transformation<\/h3>\n<p>Migration from Relational to NoSQL is a big step, and this blog does not cover the pros and cons of each of these databases. However, because of the differences on how the data is stored, you will need to make a decision on how you want to manage these changes. The N1QL scripts in this blogs will transform the Oracle HR schema into the the following Couchbase JSON data model, using the following strategies:<\/p>\n<ol>\n<li>Regions, Countries, Locations are direct and indirect parent entities of Departments. So we could denormalize the parent\/grandparent\/great grand parent into the Department entity. This will reduce the need for JOINs when these information are needed in the query. Furthermore, other associated entities such as Employees and Job_History only have a reference the department_id.\u00a0 For this reason, it would make sense to denormalize the information about the region,\u00a0 country, and location of a department into then &#8216;department&#8217; object in the Couchbase JSON model.<\/li>\n<li>The entity Jobs includes the information about pay scale. This could be sensitive data. For this reason, we will use a direct migration of this object without any transformation.<\/li>\n<li>The Employees entity includes other associated information, such as job_id and department_id, which are important attributes for an employee. It would make sense to denormalize the Job title and Department name into &#8217;employee&#8217; object.<\/li>\n<li>The Job_History entity is really a child object of the Employees entity. Therefore it would make sense to include the job history of the employee into the &#8217;employee&#8217; object.<\/li>\n<\/ol>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-7164 size-full\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM.png\" alt=\"\" width=\"1412\" height=\"620\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM.png 1412w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM-300x132.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM-1024x450.png 1024w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM-768x337.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM-20x9.png 20w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.44.35-PM-1320x580.png 1320w\" sizes=\"auto, (max-width: 1412px) 100vw, 1412px\" \/><\/p>\n<h4>Table vs Document Type<\/h4>\n<p>In Couchbase the concept of table does not apply. All documents can be stored in a single bucket with the use of a <strong>type<\/strong> field to differentiate document type. This is possible because there is no schema restriction between different documents in Couchbase NoSQL database.<\/p>\n<p>The included script takes care of this by including a <strong>doc_type<\/strong> field with the value of the source table name.<\/p>\n<pre class=\"lang:default decode:true \">SELECT 'employee' doc_type ...\u00a0 FROM hr.employees...<\/pre>\n<h4>Primary key mapping<\/h4>\n<p>An Oracle table would normally have a primary key, and you can see this in the HR schema. Couchbase documents also need a primary key. However, because all the Oracle table data will reside in a single Couchbase bucket, we need a way to differentiate between the key values of these document types in Couchbase.<\/p>\n<p>The included script takes care of this by constructing the <strong>doc_key<\/strong> using the Oracle HR table name and its primary key value.<\/p>\n<pre class=\"lang:default decode:true \">SELECT 'employee:' ||e.employee_id doc_key FROM hr.employees ...<\/pre>\n<h3>Relational to Couchbase JSON document<\/h3>\n<p>There is no hard and fast rule as to how you should transform your relational schema to NoSQL. You could migrate all the tables into a Couchbase bucket, each with its own <strong>doc_type<\/strong> field.<\/p>\n<h4>Direct table to document<\/h4>\n<p>This is the simplest case, where only the <strong>doc_type<\/strong> and <strong>doc_key<\/strong> are added the Couchbase document. The relational object does not require any transformation during the migration process.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-7159\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-06-at-12.21.02-PM.png\" alt=\"\" width=\"710\" height=\"200\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.21.02-PM.png 1022w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.21.02-PM-300x85.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.21.02-PM-768x216.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.21.02-PM-20x6.png 20w\" sizes=\"auto, (max-width: 710px) 100vw, 710px\" \/><\/p>\n<pre class=\"lang:default decode:true\">UPSERT INTO cbhr (key ndoc.doc_key,value ndoc) \r\n   SELECT ndoc.doc_key, ndoc\r\n   FROM CURL(\"https:\/\/&lt;your_server_ip&gt;:8080\/ords\/hr\/_\/sql\", \r\n              { \"request\":\"POST\",\"header\":\"Content-Type: application\/sql\"\r\n              , \"data\":\"SELECT 'job' doc_type, 'job:'||j.job_id doc_key, j.* \r\n                      FROM jobs j\"\r\n              , \"user\":'HR:oracle'}\r\n             ) r\r\n     UNNEST r.items[0].resultSet.items as ndoc<\/pre>\n<h5>Notes:<\/h5>\n<ol>\n<li>UPSERT is used so that the query can be re-run without affecting the result in the Couchbase bucket.<\/li>\n<li>The N1QL CURL command calls a REST endpoint in the Oracle Rest Data Services, with a query to be processed by the Oracle server.<\/li>\n<li>The N1QL CURL returns a JSON document with the query result set in the array field <em>r.items[0].resultSet.items.<\/em><\/li>\n<li>The N1QL query uses the UNNEST command to flatten the <em>r.items[0].resultSet.items<\/em> array, returning each Oracle record as a separate JSON document.<\/li>\n<li>N1QL UPSERT inserts each document into the Couchbase <strong>cbhr<\/strong> bucket.<\/li>\n<\/ol>\n<h4>Denormalization<\/h4>\n<p>This transformation combines several Oracle tables into a single object. In this example, the tables Regions, Countries, and Locations have a direct parent-child relationship, which allows for the parent fields to be added to the child object. The final result is a single Department object that includes its location, country, and region. This transformation is a single step process.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-7163 size-full\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png\" alt=\"\" width=\"1020\" height=\"398\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png 1020w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM-300x117.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM-768x300.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM-20x8.png 20w\" sizes=\"auto, (max-width: 1020px) 100vw, 1020px\" \/><\/p>\n<pre class=\"lang:default decode:true \">UPSERT INTO cbhr (KEY doc_key, VALUE ndoc)\r\n  SELECT ndoc.doc_key , ndoc\r\n  FROM CURL( \"&lt;your_database_server_ip&gt;:8080\/ords\/hr\/_\/sql\"\r\n           , { \"request\":\"POST\",\"header\":\"Content-Type: application\/sql\"\r\n             , \"data\": \"SELECT 'department' doc_type, 'department:' ||d.department_id doc_key, \r\n                            d.department_id, d.department_name, d.manager_id , l.* , c.country_name, r.* \r\n                        FROM departments d\r\n                             INNER JOIN locations l ON d.location_id=l.location_id\r\n                             INNER JOIN countries c ON l.country_id = c.country_id\r\n                             INNER JOIN REGIONS r ON c.region_id = r.region_id\"\r\n             ,\"user\":'HR:oracle'}\r\n           ) res\r\n       UNNEST res.items[0].resultSet.items as ndoc<\/pre>\n<h5>Notes:<\/h5>\n<ol>\n<li>UPSERT is used so that the query can be re-run without affecting the result in the Couchbase bucket.<\/li>\n<li>The N1QL CURL command calls a REST endpoint in the Oracle Rest Data Services, with a query to be processed by the Oracle server.<\/li>\n<li>The N1QL CURL returns a JSON document with the query result set in the array field <em>r.items[0].resultSet.items.<\/em><\/li>\n<li>The N1QL query uses the UNNEST command to flatten the <em>r.items[0].resultSet.items<\/em> array, returning each Oracle record as a separate JSON document.<\/li>\n<li>N1QL UPSERT inserts each document into the Couchbase <strong>cbhr<\/strong> bucket.<\/li>\n<\/ol>\n<h4>Denormalization &#8211; Adding child records as an array field to the parent object<\/h4>\n<p>One of the key features of a NoSQL database is the use of arrays. The Couchbase NoSQL database stores documents in JSON format, in which a field can be an array. For this exercise, we will add the JOB_HISTORY table into the parent EMPLOYEES table. This effectively adds a new <strong>job_history<\/strong> array field to the EMPLOYEE document.<\/p>\n<p>This transformation is a <strong><em>two step process<\/em><\/strong>. The first step is to migrate the employee data. The second step merges the <strong>employee<\/strong> data, which is already in the Couchbase <strong>cbhr<\/strong> bucket, with the Oracle query of <strong>job_history<\/strong>.<\/p>\n<h5><strong>The parent document<\/strong><\/h5>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-7157\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-06-at-12.13.30-PM.png\" alt=\"\" width=\"857\" height=\"362\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.13.30-PM.png 1056w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.13.30-PM-300x127.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.13.30-PM-1024x432.png 1024w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.13.30-PM-768x324.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.13.30-PM-20x8.png 20w\" sizes=\"auto, (max-width: 857px) 100vw, 857px\" \/><\/p>\n<pre class=\"lang:default decode:true\">UPSERT INTO cbhr (key ndoc.doc_key,value ndoc) \r\n  SELECT ndoc.doc_key, ndoc\r\n  FROM CURL(\"https:\/\/192.168.1.117:8080\/ords\/hr\/_\/sql\", \r\n             { \"request\":\"POST\",\"header\":\"Content-Type: application\/sql\"\r\n             , \"data\":\"SELECT 'employee' doc_type, 'employee:' ||e.employee_id doc_key, e.* , j.job_title, d.department_name\r\n                       FROM employees e\r\n                         INNER JOIN jobs j ON e.job_id = j.job_id\r\n                         INNER JOIN departments d ON e.department_id = d.department_id\"\r\n             , \"user\":'HR:oracle'\r\n             }) r\r\n      UNNEST r.items[0].resultSet.items as ndoc<\/pre>\n<h5><strong>The child records as an array field in the parent document<\/strong><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-7165 size-full\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2019\/07\/Screen-Shot-2019-07-06-at-1.08.03-PM.png\" alt=\"\" width=\"998\" height=\"430\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-1.08.03-PM.png 998w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-1.08.03-PM-300x129.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-1.08.03-PM-768x331.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-1.08.03-PM-20x9.png 20w\" sizes=\"auto, (max-width: 998px) 100vw, 998px\" \/><\/h5>\n<pre class=\"lang:default decode:true\">MERGE INTO cbhr e \r\n  USING ( SELECT ndoc.employee_id, ARRAY_AGG(ndoc) all_jobs\r\n          FROM CURL( \"https:\/\/&lt;your_database_server&gt;:8080\/ords\/hr\/_\/sql\"\r\n                   , {\"request\":\"POST\",\"header\":\"Content-Type: application\/sql\"\r\n                     , \"data\": \"SELECT h.employee_id, h.start_date,h.end_date, h.job_id, j.job_title, h.department_id\r\n                                FROM job_history h\r\n                                  INNER JOIN jobs j ON h.job_id=j.job_id\"\r\n                     , \"user\":'HR:oracle'\r\n                     }\r\n                   ) res\r\n              UNNEST res.items[0].resultSet.items as ndoc\r\n           GROUP BY ndoc.employee_id ) source ON KEY 'employee:'||to_string(source.employee_id)\r\nWHEN MATCHED \r\n   THEN UPDATE SET e.job_history = source.all_jobs<\/pre>\n<h5>Notes:<\/h5>\n<ol>\n<li>The N1QL query uses the MERGE command to combine documents from the <strong>cbhr<\/strong> bucket whose <strong>doc_type<\/strong> is <strong>employee<\/strong> with the result of the SELECT from the CURL REST read of the Oracle <strong>job_history<\/strong>.<\/li>\n<li>The query uses the N1QL ARRAY_AGG to group all the jobs by <strong>employee_id<\/strong>.<\/li>\n<li>The MERGE uses <em>&#8217;employee:&#8217;||to_string(source.employee_id)<\/em> as the key to match the two result sets.<\/li>\n<\/ol>\n<h3>Limitations<\/h3>\n<p>Note that there is a limitation of how much data N1QL CURL() can retrieve. Currently the maximum size is set at 64MB, which cannot be modified. This is not a lot if you plan to migrate large Oracle tables. That said, Oracle does have support for OFFSET and FETCH NEXT, which would allow you to break down the migration process into smaller chunks.<\/p>\n<p>Furthermore the main reasons of this blog is to highlight what you need to consider when migrating relational schema into a Couchbase JSON document database, and how N1QL can help to transform your relational schema directly in the migration process.<\/p>\n<p>Please drop me a comment below if you have questions or feedback.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Getting the Couchbase server to run on your PC or Mac involves a few simple steps to download the software and spin up a cluster with all the Couchbase services that you need (https:\/\/docs.couchbase.com\/server\/6.0\/getting-started\/start-here.html). Sample buckets are available with the [&hellip;]<\/p>\n","protected":false},"author":26326,"featured_media":7163,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1814,1815,1816,1819,1812],"tags":[],"ppma_author":[8919],"class_list":["post-7155","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-application-design","category-best-practices-and-tutorials","category-couchbase-server","category-data-modeling","category-n1ql-query"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.1 (Yoast SEO v26.1.1) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How to use N1QL to migrate Oracle data into Couchbase - The Couchbase Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to use N1QL to migrate Oracle data into Couchbase\" \/>\n<meta property=\"og:description\" content=\"Getting the Couchbase server to run on your PC or Mac involves a few simple steps to download the software and spin up a cluster with all the Couchbase services that you need (https:\/\/docs.couchbase.com\/server\/6.0\/getting-started\/start-here.html). Sample buckets are available with the [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\" \/>\n<meta property=\"og:site_name\" content=\"The Couchbase Blog\" \/>\n<meta property=\"article:published_time\" content=\"2019-07-09T12:08:36+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-14T06:42:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1020\" \/>\n\t<meta property=\"og:image:height\" content=\"398\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Binh Le\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Binh Le\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\"},\"author\":{\"name\":\"Binh Le\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/f89064928e262c71eb43bee996c48c63\"},\"headline\":\"How to use N1QL to migrate Oracle data into Couchbase\",\"datePublished\":\"2019-07-09T12:08:36+00:00\",\"dateModified\":\"2025-06-14T06:42:49+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\"},\"wordCount\":1747,\"commentCount\":2,\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png\",\"articleSection\":[\"Application Design\",\"Best Practices and Tutorials\",\"Couchbase Server\",\"Data Modeling\",\"SQL++ \/ N1QL Query\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\",\"name\":\"How to use N1QL to migrate Oracle data into Couchbase - The Couchbase Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png\",\"datePublished\":\"2019-07-09T12:08:36+00:00\",\"dateModified\":\"2025-06-14T06:42:49+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png\",\"width\":1020,\"height\":398},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.couchbase.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to use N1QL to migrate Oracle data into Couchbase\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"name\":\"The Couchbase Blog\",\"description\":\"Couchbase, the NoSQL Database\",\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\",\"name\":\"The Couchbase Blog\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"width\":218,\"height\":34,\"caption\":\"The Couchbase Blog\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/f89064928e262c71eb43bee996c48c63\",\"name\":\"Binh Le\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/5b68c37e30928a9d7b2c8470b1a303b7\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a939f48df6447844a8780bec264bb3be21d589336f3915fabc557075a68fa374?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a939f48df6447844a8780bec264bb3be21d589336f3915fabc557075a68fa374?s=96&d=mm&r=g\",\"caption\":\"Binh Le\"},\"description\":\"Binh Le is a Principal Product Manager for Couchbase Query service. Prior to Couchbase, he worked at Oracle and led the product management team for Sales Cloud Analytics and CRM OnDemand. Binh holds a Bachelor's Degree in Computer Science from the University of Brighton, UK.\",\"url\":\"https:\/\/www.couchbase.com\/blog\/author\/binh-le-2\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to use N1QL to migrate Oracle data into Couchbase - The Couchbase Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/","og_locale":"en_US","og_type":"article","og_title":"How to use N1QL to migrate Oracle data into Couchbase","og_description":"Getting the Couchbase server to run on your PC or Mac involves a few simple steps to download the software and spin up a cluster with all the Couchbase services that you need (https:\/\/docs.couchbase.com\/server\/6.0\/getting-started\/start-here.html). Sample buckets are available with the [&hellip;]","og_url":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/","og_site_name":"The Couchbase Blog","article_published_time":"2019-07-09T12:08:36+00:00","article_modified_time":"2025-06-14T06:42:49+00:00","og_image":[{"width":1020,"height":398,"url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png","type":"image\/png"}],"author":"Binh Le","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Binh Le","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#article","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/"},"author":{"name":"Binh Le","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/f89064928e262c71eb43bee996c48c63"},"headline":"How to use N1QL to migrate Oracle data into Couchbase","datePublished":"2019-07-09T12:08:36+00:00","dateModified":"2025-06-14T06:42:49+00:00","mainEntityOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/"},"wordCount":1747,"commentCount":2,"publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png","articleSection":["Application Design","Best Practices and Tutorials","Couchbase Server","Data Modeling","SQL++ \/ N1QL Query"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/","url":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/","name":"How to use N1QL to migrate Oracle data into Couchbase - The Couchbase Blog","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png","datePublished":"2019-07-09T12:08:36+00:00","dateModified":"2025-06-14T06:42:49+00:00","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#primaryimage","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2019\/07\/Screen-Shot-2019-07-06-at-12.42.23-PM.png","width":1020,"height":398},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/blog\/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How to use N1QL to migrate Oracle data into Couchbase"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/blog\/#website","url":"https:\/\/www.couchbase.com\/blog\/","name":"The Couchbase Blog","description":"Couchbase, the NoSQL Database","publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/blog\/#organization","name":"The Couchbase Blog","url":"https:\/\/www.couchbase.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","width":218,"height":34,"caption":"The Couchbase Blog"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/f89064928e262c71eb43bee996c48c63","name":"Binh Le","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/5b68c37e30928a9d7b2c8470b1a303b7","url":"https:\/\/secure.gravatar.com\/avatar\/a939f48df6447844a8780bec264bb3be21d589336f3915fabc557075a68fa374?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a939f48df6447844a8780bec264bb3be21d589336f3915fabc557075a68fa374?s=96&d=mm&r=g","caption":"Binh Le"},"description":"Binh Le is a Principal Product Manager for Couchbase Query service. Prior to Couchbase, he worked at Oracle and led the product management team for Sales Cloud Analytics and CRM OnDemand. Binh holds a Bachelor's Degree in Computer Science from the University of Brighton, UK.","url":"https:\/\/www.couchbase.com\/blog\/author\/binh-le-2\/"}]}},"authors":[{"term_id":8919,"user_id":26326,"is_guest":0,"slug":"binh-le-2","display_name":"Binh Le","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/a939f48df6447844a8780bec264bb3be21d589336f3915fabc557075a68fa374?s=96&d=mm&r=g","author_category":"","last_name":"Le","first_name":"Binh","job_title":"","user_url":"","description":"Binh Le is a Principal Product Manager for Couchbase Query service. Prior to Couchbase, he worked at Oracle and led the product management team for Sales Clould Analytics and CRM OnDemand. Binh holds a Bachelor's Degree in Computer Science from the University of Brighton, UK."}],"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/7155","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/users\/26326"}],"replies":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/comments?post=7155"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/7155\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media\/7163"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media?parent=7155"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/categories?post=7155"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/tags?post=7155"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=7155"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}