Migrating Oracle DB to Couchbase

Couchbase Server
1

Hi, I’m working on project which Migrating data from oracle Db to Couchbase ,
The volume is quite big and the BI logic can be very complex.
First ,we extract the data into json and from there we loading it using couchbase loader.
Im not coniviced this would be the best approach from perfroamnce aspects , what do u think?

Second Q - for the data validation part ,we need to compare the source and target
(after load) , is there any tool we can use? is any scripting language can do that good and fast?
Perl?Phyton?
any other tips are welcome!

2

Hi @BgOKcomp,

That sounds like a decent approach to me. But if it’s not performing well enough, you might want to check out the Golden Gate connector for Couchbase: https://github.com/mahurtado/CouchbaseGoldenGateAdapter

As for comparing the data after the load, Couchbase offers a number of SDKs: Python, but also Java, C, Go, .NET, .NET Core, PHP, Node, and several others that are community supported. There is probably a Perl SDK, but not an officially supported one.

3

@BgOKcomp - Did you find the way to move the huge volume of data in efficient way ?
@matthew.groves - this talks about data Sync with change data capture but no migration .
It is sad that moving to couchbase there is no great guidelines a publication how client can handle huge volume of data migration (relational to couchbase)
thanks

4

@eldorado,

There are many approaches and tools to migrating data from relational to Couchbase, and recommendations vary greatly depending on your requirements and use case. In addition to the GoldenGate connector, here are some resources to start with:

1 Like
5

Thanks for providing the links to read in . The only link which I can resembles to migration is below https://blog.couchbase.com/how-to-use-n1ql-to-migrate-oracle-data-into-couchbase/
However CURL lacks with limitations …Anyway do you know with same approach in CURL with N1QL we can have parameters to access on huge volume of Oracle data pragmatically and keep the connection live for billions of records ?
Talend is extra tool to buy in just for migration is not sounds reasonable for us … sorry

6

@eldorado,

I’d recommend starting a new thread in the N1QL forum: https://www.couchbase.com/forums/c/n1ql

I’m tagging @binh.le to see if he has any additional insight for you.

7

thanks Matt .I am trying to get @binh.le attention on other threads already . Just waiting for his feedback on 3 weeks old post

8

hi @eldorado and @BgOKcomp, i found your topic just today and i would like to ask you if you’ve already found a workaround

Otherwise i would suggest you or any further reader about the availability of a connector, called GlueSync that officially supports Oracle and Ms SQL Server RDBMS within Couchbase, establishing a 2 way real-time connection.

Here is a link about the presentation on this year’s couchbase connect event - https://connectonline.couchbase.com/forum/t/empower-your-legacy-business-apps-keep-your-rdbms-and-couchbase-in-sync-with-gluesync/214

or if you would like to know more here is the website: https://mobilebridge.it/gluesync/

Hope this will help!

1 Like
9

thanks @Daniele_Angeli but I see the support for Oracle is yet to come . Any ETA ?

10

Hi @eldorado, i just saw your message on our website, Oracle support has been certified last week and it is currently GA (generally available), we’re updating our website

Feel free to contact me at daniele.angeli@molo17.com

thanks

1 Like
11

BgOKcomp another approach for verification of your data conversion might be to use the Evenitng service. Once the data has been loaded into Couchbase you can “flatten” the data again put in another bucket (or buckets) and export the flattend “table” representations and then use sort/diff to compare.

Refer to https://docs.couchbase.com/server/current/eventing/eventing-handler-simpleFlatten.html for an example of the “flatten” step.

12

Here it is -> https://molo17.com/gluesync-connector-couchbase/

1 Like
13

Thanks for the update. We will review and get-back . Do you know if GS can support sync from Oracle On prem to Cloud SQL DB database ? and how Molo17 compared to Striim ? what is the competitive differences.

14

Sure thing @eldorado, pleased to answer

GS is meant to work both on on-prem and on-cloud scenarios but we have customers working on mixed environments having the RDBMS on-prem and Couchbase in the cloud. Also we’re compatible with CBaaS as soon as Eventing will be supported in the next release (@jon.strabala)

regarding GS vs Striim comparison:

  • we have a strong and long running partnership with Couchbase, with proven specific skill-set (https://www.linkedin.com/posts/matthewmcain_cbconnectonline-activity-6734465141150220288-Omci)
  • GS is tailored around Couchbase future-proof latest technologies, in that way our workloads are isolated (eventing service)
  • works also with the SyncGateway APIs if you have a Couchbase lite / mobile deployment in place
  • we support SQL views (useful for data modeling)
  • GS has been selected as an official Couchbase connector tool
  • Doesn’t generate conflicts or loops when you have in the same bucket also the SyncGateway

Happy to jump in a call with you to discuss your use case :slightly_smiling_face:

15

Thank you but on my first question I mean to say do you guys provide solution to any other target database except couchbase to sync from on-prem to cloud . I guess Striim has varity of source and targets work vice versa for sync .
for e.g. Oracle on-Prem to postgressql cloud or Oracle On Prem to Microsoft SQL Server cloud

16

Hi @eldorado, we’re specialized around the Couchbase world so the GS connector has currently a specific support for that target only

Striim and others competitors have a wider variety of targets, some of them support also Couchbase others not

So basically it depends on your specific use case and what you’re looking for: a specific tool for Couchbase or a general purpose solution that can let you plug different targets

Best regards