Test Container (7.1.6) is giving UNKNOWN_COLLECTION

Java SDK
1

I am creating a testcontainer which is working fine.

But at the time of insert, I am getting unknown collection error. Sharing my code :

class CouchbaseTestContainer {

    def static COUCHBASE_BUCKET = 'stvs-stock-dev'
    def static COUCHBASE_SCOPE = 'stockMovement'
    def static COUCHBASE_SSM_COLLECTION = 'storeStockMovement'
    def static COUCHBASE_APP_COLLECTION = 'config'
    def static COUCHBASE_COUCHMOVE_METADATA = 'couchmoveMetadata'

    static CouchbaseContainer couchbaseContainer

    static Cluster cluster

    static {
        couchbaseContainer = new CouchbaseContainer("couchbase/server:enterprise-7.1.6")
                .withBucket(new BucketDefinition(COUCHBASE_BUCKET).withPrimaryIndex(true))
                .withStartupTimeout(Duration.ofMinutes(2))
                .withStartupAttempts(3)

        couchbaseContainer.start()
        print("Started Couchbase Test Container")

        var seedNodes = new HashSet<>(Collections.singletonList(SeedNode.create(couchbaseContainer.getHost(),
                Optional.of(couchbaseContainer.getBootstrapCarrierDirectPort()),
                Optional.of(couchbaseContainer.getBootstrapHttpDirectPort()))));
        cluster = Cluster.connect(seedNodes, ClusterOptions.clusterOptions(couchbaseContainer.getUsername(), couchbaseContainer.getPassword()))

        cluster.bucket(COUCHBASE_BUCKET).collections().createScope(COUCHBASE_SCOPE)

        // couchmove can't create collections owing to limitation of user over TPC
        cluster.bucket(COUCHBASE_BUCKET).collections().createCollection(CollectionSpec.create(COUCHBASE_COUCHMOVE_METADATA, COUCHBASE_SCOPE))
        cluster.bucket(COUCHBASE_BUCKET).collections().createCollection(CollectionSpec.create(COUCHBASE_SSM_COLLECTION, COUCHBASE_SCOPE))
        cluster.bucket(COUCHBASE_BUCKET).collections().createCollection(CollectionSpec.create(COUCHBASE_APP_COLLECTION, COUCHBASE_SCOPE))
        cluster.bucket(COUCHBASE_BUCKET).collections().createCollection(CollectionSpec.create(COUCHBASE_APP_COLLECTION, COUCHBASE_SCOPE))
        cluster.waitUntilReady(Duration.ofMinutes(2))


    }
}

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ActiveProfiles('test')
@Testcontainers
@EnableAutoConfiguration
 class IntegrationBaseConfig extends Specification {

    @DynamicPropertySource
    static void initializeProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.couchbase.username", () -> CouchbaseTestContainer.couchbaseContainer.getUsername())
        registry.add("spring.couchbase.password", () -> CouchbaseTestContainer.couchbaseContainer.getPassword())
        registry.add("spring.couchbase.bucket", () -> CouchbaseTestContainer.COUCHBASE_BUCKET)
        registry.add("spring.couchbase.connection-string", () -> CouchbaseTestContainer.couchbaseContainer.getConnectionString())
        registry.add("spring.couchbase.scope", () -> CouchbaseTestContainer.COUCHBASE_SCOPE)
    }

    String getDocumentID() {
        var documentId = UUID.randomUUID().toString()
        return documentId
    }

    void insertData(Collection collection, String Id, JsonObject object) {
        collection.insert(Id, object)
    }
}

class JobExecutionSpec extends IntegrationBaseConfig {

    @Autowired
    CouchbaseTemplate couchbaseTemplate

    def static COUCHBASE_BUCKET = 'stvs-stock-dev'
    def static COUCHBASE_SCOPE = 'stockMovement'
    def static COUCHBASE_SSM_COLLECTION = 'storeStockMovement'
    def static COUCHBASE_APP_COLLECTION = 'config'
    def static COUCHBASE_COUCHMOVE_METADATA = 'couchmoveMetadata'

    def setup() {

        JsonArray values = JsonArray.from("STOCK", "NON-STOCK")
        JsonObject object = JsonObject.create()
                .put("countryCode", "GB")
                .put("name", "WASTAGE")
                .put("value", values)
        String documentID = getDocumentID()
        Collection collection = couchbaseTemplate.getCollection(COUCHBASE_APP_COLLECTION)
        insertData(collection, documentID, object)
    }

And the error which I get is

2023-11-18 13:54:58,076 WARN [cb-events] , config - [com.couchbase.config][CollectionMapRefreshFailedEvent][2504ms] Collection Map refresh (CollectionIdentifier{bucket='my-bucket', scope=Optional[_default], collection=Optional[config], isDefault=false}) failed: FAILED {"alternateIdentifier":"external","coreId":"0x450cf55200000002"}
com.couchbase.client.core.error.UnambiguousTimeoutException: GetCollectionIdRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0x450cf55200000002","idempotent":true,"lastChannelId":"450CF55200000002/00000000ECFEA06D","lastDispatchedFrom":"127.0.0.1:61812","lastDispatchedTo":"localhost:61523","reason":"TIMEOUT","requestId":33,"requestType":"GetCollectionIdRequest","retried":14,"retryReasons":["KV_COLLECTION_OUTDATED"],"service":{"bucket":"my-bucket","collection":"config","errorCode":{"description":"Operation specified an unknown collection.","name":"UNKNOWN_COLLECTION"},"opaque":"0x3c","scope":"_default","type":"kv","vbucket":0},"timeoutMs":2500,"timings":{"dispatchMicros":2539,"totalDispatchMicros":21269,"totalServerMicros":0,"serverMicros":0}}
	at com.couchbase.client.core.msg.BaseRequest.cancel(BaseRequest.java:184)
	at com.couchbase.client.core.msg.Request.cancel(Request.java:71)
	at com.couchbase.client.core.Timer.lambda$register$2(Timer.java:172)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
	at com.couchbase.client.core.deps.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
	at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)


2023-11-18 16:09:19,549 WARN [cb-events] , config - [com.couchbase.config][CollectionMapRefreshFailedEvent][2506ms] Collection Map refresh (CollectionIdentifier{bucket='stvs-stock-dev', scope=Optional[_default], collection=Optional[config], isDefault=false}) failed: FAILED {"alternateIdentifier":"external","coreId":"0xe54c397100000002"}
com.couchbase.client.core.error.UnambiguousTimeoutException: GetCollectionIdRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0xe54c397100000002","idempotent":true,"lastChannelId":"E54C397100000002/0000000053F56236","lastDispatchedFrom":"127.0.0.1:57235","lastDispatchedTo":"localhost:56953","reason":"TIMEOUT","requestId":31,"requestType":"GetCollectionIdRequest","retried":14,"retryReasons":["KV_COLLECTION_OUTDATED"],"service":{"bucket":"stvs-stock-dev","collection":"config","errorCode":{"description":"Operation specified an unknown collection.","name":"UNKNOWN_COLLECTION"},"opaque":"0x33","scope":"_default","type":"kv","vbucket":0},"timeoutMs":2500,"timings":{"dispatchMicros":5805,"totalDispatchMicros":53453,"totalServerMicros":1,"serverMicros":0}}
	at com.couchbase.client.core.msg.BaseRequest.cancel(BaseRequest.java:184)
	at com.couchbase.client.core.msg.Request.cancel(Request.java:71)
	at com.couchbase.client.core.Timer.lambda$register$2(Timer.java:172)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
	at com.couchbase.client.core.deps.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
	at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
	at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
2023-11-18 16:09:21,725 WARN [cb-events] , tracing - [com.couchbase.tracing][OverThresholdRequestsRecordedEvent][10s] Requests over Threshold found: {"query":{"top_requests":[{"operation_name":"query","last_dispatch_duration_us":2920300,"last_remote_socket":"localhost:56952","last_local_socket":"127.0.0.1:57242","total_dispatch_duration_us":2920300,"timeout_ms":75000,"total_duration_us":2940546}],"total_count":1},"kv":{"top_requests":[{"operation_name":"insert","last_remote_socket":"localhost:56953","last_local_id":"E54C397100000002/0000000053F56236","last_local_socket":"127.0.0.1:57235","encode_duration_us":715,"operation_id":"0x34","timeout_ms":2500,"total_duration_us":2506654}],"total_count":1}}

InsertRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0xe54c397100000002","idempotent":false,"lastChannelId":"E54C397100000002/0000000053F56236","lastDispatchedFrom":"127.0.0.1:57235","lastDispatchedTo":"localhost:56953","reason":"TIMEOUT","requestId":32,"requestType":"InsertRequest","retried":14,"retryReasons":["COLLECTION_MAP_REFRESH_IN_PROGRESS"],"service":{"bucket":"stvs-stock-dev","collection":"config","documentId":"c7a83a11-75cf-47dc-a8d3-00fa72334ca9","opaque":"0x34","scope":"_default","type":"kv","vbucket":425},"timeoutMs":2500,"timings":{"encodingMicros":715,"totalMicros":2506654}}
com.couchbase.client.core.error.AmbiguousTimeoutException: InsertRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0xe54c397100000002","idempotent":false,"lastChannelId":"E54C397100000002/0000000053F56236","lastDispatchedFrom":"127.0.0.1:57235","lastDispatchedTo":"localhost:56953","reason":"TIMEOUT","requestId":32,"requestType":"InsertRequest","retried":14,"retryReasons":["COLLECTION_MAP_REFRESH_IN_PROGRESS"],"service":{"bucket":"stvs-stock-dev","collection":"config","documentId":"c7a83a11-75cf-47dc-a8d3-00fa72334ca9","opaque":"0x34","scope":"_default","type":"kv","vbucket":425},"timeoutMs":2500,"timings":{"encodingMicros":715,"totalMicros":2506654}}
	at app//com.couchbase.client.core.util.CoreAsyncUtils.block(CoreAsyncUtils.java:54)
	at app//com.couchbase.client.core.api.kv.CoreAsyncResponse.toBlocking(CoreAsyncResponse.java:53)
	at app//com.couchbase.client.core.api.kv.CoreKvOps.insertBlocking(CoreKvOps.java:136)
	at app//com.couchbase.client.java.Collection.insert(Collection.java:518)
	at app//com.couchbase.client.java.Collection.insert(Collection.java:498)
	at app//com.tesco.finance.stvs.config.IntegrationBaseConfig.insertData(IntegrationBaseConfig.groovy:35)
	at app//com.tesco.finance.stvs.JobExecutionSpec.setup(JobExecutionSpec.groovy:44)
	Suppressed: java.lang.Exception: The above exception was originally thrown by another thread at the following location.
		at com.couchbase.client.core.msg.BaseRequest.cancel(BaseRequest.java:184)
		at com.couchbase.client.core.msg.Request.cancel(Request.java:71)
		at com.couchbase.client.core.Timer.lambda$register$2(Timer.java:172)
		at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
		at com.couchbase.client.core.deps.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
		at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
		at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
		at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
		at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
		at java.base/java.lang.Thread.run(Thread.java:833)

Started Couchbase Test Container

I can see that in the log it mentions the bucket name but in the scope it is mentioning “_default” although i have created the cluster with bucket and scope.

What am I doing wrong in the above code

2

Creation of couchbase artifacts is not synchronous. After issuing the command to create the collection use WaitUntilReady before using the collection

3

Hmm, it looks from the CollectionMapRefreshFailedEvent that it’s failing to fetch the collection map. I’m suspecting this bit is related:

        var seedNodes = new HashSet<>(Collections.singletonList(SeedNode.create(couchbaseContainer.getHost(),
                Optional.of(couchbaseContainer.getBootstrapCarrierDirectPort()),
                Optional.of(couchbaseContainer.getBootstrapHttpDirectPort()))));

could you try instead the container.getConnectionString() method from Couchbase Module - Testcontainers for Java?

1 Like
4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.