Rapid multiple inserts in Couchbase

Couchbase newbie question (clubbing 5 related questions together):

I am trying to insert 1 million records in couchbase in quick succession, but I see that nearly about 0.5 million records get inserted (Admin console shows 517365 as Item Count). Also, from the admin GUI, I can only see 1000 records (10 pages of 100 records each)

Am wondering where rest of the records are vanishing !

I suspect when Couchbases's internal buffer is full, further requests are dropped. Can somebody please help me with the following ?

1) How can I configure the couchbase internal buffer size or atleast check its default size?

2) I used client.set(n + "", 0, n + "").get() to make sure that the data is persisted. But still 0.7 million records were set. Also, I don't like it as I am doing a unnecessary get operation here.

3) When I close the couchbase client, are the set operations queued in couchbase buffer aborted ?

4) Which log file I should be looking at to find insertion failure errors ? (There are tad too many files in Couchbase\Server\var\lib\couchbase\logs folder)

5) Does Couchbase GUI not paginate all the records ?

public class Test {
    public static void main(String[] args) {
        ArrayList<URI> nodes = new ArrayList<URI>();
        String cbUrl = "http://127.0.0.1:8091/pools";
        String dbName = "default";
        CouchbaseClient client = null;
        try {
            nodes.add(URI.create(cbUrl));
 
            client = new CouchbaseClient(nodes, dbName, "");
 
            insertRecords(client);
 
            System.out.println("Test Over");
 
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            // client.shutdown();
        }
    }
 
    public static void insertRecords(CouchbaseClient client) throws Exception {
        int num = 1000000;
 
        for (int n = 1; n <= num; n++) {
            System.out.println("Adding: " + n);
            client.set(n + "", 0, n + "");
        }
    }
}

0 Answers

No answers yet