Bulk insert speed gap when multiple machines are used

Go SDK
1

Hi,

A go utility reads data from Flat files and bulk loads it into couchbase. The application is able to insert data at a speed of up to 21K writes per second when 2 writer threads are being executed on a single machine (Destination is a remote couchbase cluster with one server within the network).

But When 2 writer threads are executed from 2 different machines (1 thread each), the insertion Speed is reduced to half (10K writes/sec).
Since both the machines are using their own RAM and CPU for insertion, plus the utility has shown the speed of upto 25K writes per second, the network doesn’t seem to be the issue ( I checked the network utilization as well and it is below 50 percent when multiple machines are used.)

Note: All machines that are used have i7 3.40GHz quad-core processor and 8GB RAM. The total amount of data being inserted is up to 500MB.
Bucket Configuration: 5.00GB RAM, Bucket disk I/O priority : High

I need to know what’s causing this speed gap. Please help…

Regards,
Krishan

1 Like
2

Hey @krishan.jangid,

Have you recently made any changes to your code or development environment?

Cheers, Brett

3

Hi @brett19, apart from installing two Reporting tools, there’s no change in the environment as far as I remember.