Every REST API call with oidc Bearer Token increases sequence number since sync gateway 3.1, leading to spamming changes feed

Mobile Sync Gateway
1

Hi,

since upgrading to sync gateway 3.1, we experience issues with API calls authenticated using Bearer token. I am testing with Couchbase Sync Gateway/3.1.0(592;2a9837d) EE.
Any API call with a Bearer Token will increase the sequence number. When using the _changes feed, this will increase the sequence number every second and spams changes.

When calling changes feed with /db/_changes?include_docs=true&since=10147607 we immediately get the result back:

{
    "results": [
        {
            "seq": 10147608,
            "id": "_user/oidc_url_830acde4-ef26-4411-a9b1-150957cb3a83",
            "changes": []
        }
    ],
    "last_seq": "10147608"
}

We will then call the changes feed with since 10147608, which will in turn trigger the next update. This both spams the changes feed and increases the sequence number into infinity.
When using longpoll inside browsers, this leads to user browsers spamming requests as well.

Debug logs look like this:

2023-06-06T21:06:01.217+02:00 [DBG] Auth+: c:#4185 db:db OIDCUsername: <ud>oidc_url_830acde4-ef26-4411-a9b1-150957cb3a83</ud>
2023-06-06T21:06:01.264+02:00 [INF] Auth: c:#4185 db:db Saved principal w/ name:<ud>oidc_url_830acde4-ef26-4411-a9b1-150957cb3a83</ud>, seq: #10147606
2023-06-06T21:06:01.264+02:00 [INF] DCP: db:db Received #10147606 ("<ud>_user/oidc_url_830acde4-ef26-4411-a9b1-150957cb3a83</ud>")
2023-06-06T21:06:01.264+02:00 [DBG] Changes+: Notifying that "bucket" changed (key="<ud>_sync:user:oidc_url_830acde4-ef26-4411-a9b1-150957cb3a83</ud>") count=23

We have “register”: true set for oidc provider, but this is an already existing user and no new user is added.

We also have another, maybe related, issue with Bearer Token and the api since upgrading: Http: panic serving <IP>: runtime error: invalid memory address or nil pointer dereference after sync gateway upgrade to 3.1

This pretty much breaks the changes feed functionality for us.

Any help is greatly appreciated.

2

It shouldn’t be the case that every JWT auth triggers an update to the user document (with the accompanying sequence allocation).

Can you share your Sync Gateway OIDC config, and if possible the claims being included in the JWT being used for auth?

3

This is the server config (this is a fresh installation to test):
Here’s the config:
Output of {db}/_config:

{
    "bucket": "my-bucket",
    "name": "db",
    "import_docs": true,
    "oidc": {
        "providers": {
            "keycloakatu": {
                "issuer": "https://url/auth/realms/master",
                "register": true,
                "client_id": "automated-testing",
                "username_claim": "",
                "roles_claim": "",
                "channels_claim": "",
                "allow_unsigned_provider_tokens": false,
                "IsDefault": false,
                "Name": "",
                "InsecureSkipVerify": false
            },
            "keycloakimplicit": {
                "issuer": "https://url/auth/realms/master",
                "register": true,
                "client_id": "mobile_apps",
                "username_claim": "",
                "roles_claim": "",
                "channels_claim": "",
                "allow_unsigned_provider_tokens": false,
                "IsDefault": false,
                "Name": "",
                "InsecureSkipVerify": false
            }
        }
    },
    "enable_shared_bucket_access": true,
    "num_index_replicas": 0
}

output of /{db}/

{
    "db_name": "db",
    "update_seq": 9,
    "committed_update_seq": 9,
    "instance_start_time": 1686612218699950,
    "compact_running": false,
    "purge_seq": 0,
    "disk_format_version": 0,
    "state": "Online",
    "server_uuid": "66f9b1cc34f7dd149a6928a6b01b3a67"
}

server config, output of /_config:

{
    "bootstrap": {
        "server": "couchbase://couchbase-server",
        "username": "sync-gateway",
        "password": "xxxxx",
        "use_tls_server": false
    },
    "api": {
        "admin_interface": ":4985",
        "admin_interface_authentication": false,
        "https": {},
        "cors": {
            "origin": [
                "http://localhost:8080",
                "http://localhost:4984"
            ],
            "login_origin": [
                "http://localhost:8080",
                "http://localhost:4984"
            ],
            "headers": [
                "Content-Type",
                "Authorization",
                "Set-Cookie",
                "sentry-trace"
            ],
            "max_age": 1728000
        }
    },
    "logging": {
        "console": {
            "rotation": {},
            "log_level": "trace",
            "log_keys": [
                "*"
            ]
        },
        "error": {
            "rotation": {}
        },
        "warn": {
            "rotation": {}
        },
        "info": {
            "rotation": {}
        },
        "debug": {
            "enabled": true,
            "rotation": {}
        },
        "trace": {
            "rotation": {}
        },
        "stats": {
            "rotation": {}
        }
    },
    "auth": {},
    "replicator": {},
    "unsupported": {
        "serverless": {},
        "http2": {}
    }
}

Server information:

{
    "ADMIN": true,
    "couchdb": "Welcome",
    "vendor": {
        "name": "Couchbase Sync Gateway",
        "version": "3.1"
    },
    "version": "Couchbase Sync Gateway/3.1.0(592;2a9837d) CE",
    "persistent_config": true
}

These are the jwt claims of the identity token:

{
    "exp": 1687420504,
    "iat": 1687384504,
    "auth_time": 0,
    "jti": "",
    "iss": "https://oidcprovider/realms/vestigas",
    "aud": "automated-testing",
    "sub": "830acde4-ef26-4411-a9b1-150957cb3a83",
    "typ": "ID",
    "azp": "automated-testing",
    "session_state": "value",
    "at_hash": "vallue",
    "acr": "1",
    "email_verified": true,
    "name": "Name",
    "preferred_username": "username",
    "given_name": "Name",
    "locale": "en",
    "family_name": "name",
    "email": "email@example.com"
}

Thank you for looking into it!

4

Apologies for lack of response earlier.
We identified and fixed an issue in the upcoming 3.1.1 SGW release based on the above information.

https://issues.couchbase.com/browse/CBG-3129

Thanks!

5

Thank you!

Looking forward to the 3.1.1 release :slight_smile:

6

FYI, 3.1.1 was recently released: Release Notes | Couchbase Docs

1 Like