Issues Creating Partitioned Indexes on 5.5 Beta

N1QL
1

@deepkaran.salooja

In my experiments with the recent upgrade to 5.5 Beta, I’m seeing some odd behaviors related to creating partitioned indexes with node assignments.

My testing is using a 3 node cluster of Enterprise Edition 5.5.0 build 2473 using the Docker images and running on my local machine.

Regarding controlling nodes assignments and replicas

  • If I use {"num_replica": 1}, everything works as intended. I get one main index and one replica, each using all nodes.
  • If I use {"nodes": ["node1:8091", "node2:8091"]}, everything works as intended. I get one index using the defined nodes.
  • If I use {"nodes": ["node1:8091", "node2:8091"], "num_replica": 0}, everything works as intended. I get one index using the defined nodes.
  • If I use {"nodes": ["node1:8091", "node2:8091"], "num_replica": 1} it works as I would expect, one main index and one replica, both spread across node1 and node2.
  • If I use {"nodes": ["node1:8091", "node2:8091", "node3:8091"], "num_replica": 1}, I would expect to get 2 replicas each using those three nodes. Instead I get an error on create Fails to create index. Parameter num_replica should be one less than parameter nodes. like I would with unpartitioned indexes.

Regarding replica builds

Additionally, when creating replicas, I’m seeing problems where the replica sometimes fails to build. I’ve done some experimentation and I believe I’ve narrowed down to the following case:

  • Creating using a script so that steps proceed very quickly (in this case I was using the NodeJS SDK)
  • Creating a partitioned index with {"num_replica": 1, "defer_build": true}
  • Immediately following this with a BUILD INDEX step to build the index

In this case, it seems to sporadically fail to build the replica. I’ve seen it with no build progress, or stick at an even percentage (i.e. 50%). I suspect it’s because the BUILD is being triggered before all nodes are up to date with replica and partition information.

In case it’s applicable, in my testing BUILD INDEX was being done using the NodeJS buildDeferredIndexes call:
http://docs.couchbase.com/sdk-api/couchbase-node-client-2.2.1/BucketManager.html#buildDeferredIndexes in the NodeJS SDK

Internally, this uses a SELECT from system:indexes to get the list of deferred indexes and builds that into a BUILD INDEX statement for a single bucket.

Thanks,
Brant

2

Thanks @btburnett3 for trying it out. The first issue is actually a bug. It will be addressed on MB-29321.

Regarding the build index issue, there is a background process to ensure the indexes will get built eventually in case of race conditions in propagating the create index to respective nodes. Do you see the same behavior even if you wait for a few minutes?

3

@deepkaran.salooja

You are correct, if I wait a few minutes the replica builds. Seems somewhat non-ideal since it has to stream all the data to the nodes a second time, but functional.

Thanks,
Brant

4

Index replicas always have to stream data from the source unlike data service replicas.