Index mutations per Collection

Couchbase Server
,
1

We have a single bucket (profiles) with the following scopes/collections/indexes:

CREATE SCOPE `profiles`.core
CREATE SCOPE `profiles`.configuration

CREATE COLLECTION `profiles`.core.user_profile;

CREATE PRIMARY INDEX `#primary` ON `profiles`.`configuration`.`attribute`;

In our tests, we are mutating on profiles.core.user_profile (insert, replace). However, as shown in the picture, we observe that Couchabse sent these mutations to the profiles.configuration.attribute primary index.

index
index1744Ă—239 22.7 KB

Why is this happening? Indexes are supposed to be created per collection.

2

Which version of Couchbase Server are you using? Also, can you give a code snippet of how you’re doing the insert and replaces, including details on scopes/collections used?

I’m not aware of any problems in this area, so I suspect it may just be something basic with how the code is scoping the inserts/replaces.

1 Like
3

We are using Java SDK(reactive API):

  @Autowired
  private com.couchbase.client.java.Cluster cluster;

  private static final String SCOPE = "core";
  private static final String BUCKET = "profiles";

cluster.reactive().bucket(BUCKET).scope(SCOPE).collection("user_profile")
        .insert(documentId, content, InsertOptions.insertOptions().durability(durability)
            .retryStrategy(retryStrategy))


cluster.reactive().bucket(BUCKET).scope(SCOPE).collection("user_profile")
        .mutateIn(documentId, mutateInSpec, MutateInOptions.mutateInOptions()
            .durability(durability)
            .storeSemantics(StoreSemantics.REPLACE)
            .cas(cas)
            .retryStrategy(retryStrategy))

And Spring repositories for the configuration collections:

@Document
@Collection("attribute")
@Scope("configuration")
@Data
@EqualsAndHashCode(callSuper = true)
@NoArgsConstructor
public class Attribute {

  private static final long serialVersionUID = 4195336417156562036L;

  @Id
  private String externalId;

  @Field("name")
  private String name;

...
}

Couchbase Server Community Edition 7.0.2 build 6703
spring-boot-starter-data-couchbase-reactive:2.6.1

Please also note that the same happens if we use the “_default” scope for all collections:

CREATE COLLECTION `profiles`.`_default`.user_profile;
CREATE COLLECTION `profiles`.`_default`.attribute;
CREATE PRIMARY INDEX `#primary` ON `profiles`.`_default`.`attribute`;

  @Autowired
  private ReactiveCouchbaseTemplate template;

template.getCollection("user_profile").reactive()
        .insert(documentId, content, InsertOptions.insertOptions().durability(durability)
            .retryStrategy(retryStrategy))
4

Hi @anonimos

TL;DR;
This is a stats issue. The mutations should not be shared across collections i.e. the mutations on profiles.core.user_profile will not be sent to indexes on profiles.configuration.attribute. You should check other stats at index level like Indexed items and Index data size.

To explain in detail,

When an index is building it gets the data from INIT (initial) stream which operates at collection level. Once the index is moved to ready state it uses MAINT (maintenance) stream to get data from KV which operates at bucket level. This is done to optimise resources during maintenance phase of indexes.

Mutations remaining shown on the UI is difference of the bucket level sequence number in KV and max sequence number seen by indexer. As the index is using MAINT stream the mutations on a collection (profiles.core.user_profile) are increasing the bucket sequence numbers and the same is reflected on index (primary) of another collection (profiles.configuration.attribute) on same bucket as its using bucket level maintenance stream.

We are looking into this issue and will address this stats issue using https://issues.couchbase.com/browse/MB-51161 in later versions.

Please let us know if you are seeing any further issues in this regard.

Thanks
Sai