Fluent bit has no access to read log files

I’m following exactly the same thing as a tutorial to add a fluent bit. But I’m getting an error that says “cannot open /opt/couchbase/var/lib/couchbase/logs/XXX.log”

https://docs.couchbase.com/operator/current/tutorial-couchbase-log-forwarding.html#create-a-custom-configuration

And this is the configuration i used
https://github.com/couchbase/couchbase-fluent-bit/blob/main/conf/fluent-bit-simple.conf

# Cut down version with no real work in it to use for testing/debug
@include /fluent-bit/etc/couchbase/service.conf

# Just pull in all logs with no parsing at all or Multiline support
[INPUT]
    Name tail
    Alias simple_tail
    Path ${COUCHBASE_LOGS}/*.log
    Refresh_Interval 10
    Skip_Long_Lines On
    Path_Key filename
    Tag couchbase.log.<logname>
    Tag_Regex ${COUCHBASE_LOGS}/(?<logname>[^.]+).log$

@include /fluent-bit/etc/couchbase/out-stdout.conf

What sort of storage is backing the Persistent Volumes here? Perhaps there is some sort of security context in place with the provisioned storage.

Hi, tommie, Thanks for the reply, I’m just using docker desktop (WSL ubuntu), and I’m following the same thing as the example provided in the documentation. As I understand security context can be a case if I put this on something like openshift. But here I’m in my local docker-desktop
So this is how i mounted volume

apiVersion: v1
kind: Secret
metadata:
  name: cb-example-auth
type: Opaque
data:
  username: QWRtaW5pc3RyYXRvcg== # Administrator
  password: cGFzc3dvcmQ=         # password
---
apiVersion: couchbase.com/v2
kind: CouchbaseBucket
metadata:
  name: default
---
apiVersion: couchbase.com/v2
kind: CouchbaseCluster
metadata:
  name: cb-example
spec:
  logging:
    server:
      enabled: true 
      manageConfiguration: true
      configurationName: "fluent-bit-config"
      sidecar:
        image: "couchbase/fluent-bit:1.0.4"
    audit:
      enabled: true 
      garbageCollection:
        sidecar:
          enabled: true 
  image: couchbase/server:6.6.2
  security:
    adminSecret: cb-example-auth
  buckets:
    managed: true
  servers:
  - size: 3
    name: all_services
    services:
    - data
    - index
    - query
    - search
    - eventing
    - analytics
    volumeMounts:
      default: couchbase
  volumeClaimTemplates: 
  - metadata:
      name: couchbase
    spec:
      resources:
        requests:
          storage: 1Gi

And this a fluent bit config

apiVersion: v1
kind: Secret
metadata:
  name: fluent-bit-config
stringData:
  fluent-bit.conf: |
    [SERVICE]
        flush        1
        daemon       Off
        log_level    Warning
        parsers_file /fluent-bit/etc/parsers-couchbase.conf
    # Include audit log only
    @include /fluent-bit/etc/couchbase/in-audit-log.conf
    # Send to the standard output
    [OUTPUT]
        name  stdout
        match couchbase.log.*

Got it, setup seems fine. It’s possible that the user of the fluent sidecar is running as a different user than the Couchbase user. Could you try explicitly setting a SecurityContext so see if this resolves the problem here?

apiVersion: couchbase.com/v2
kind: CouchbaseCluster
metadata:
  name: cb-example
spec:
  securityContext:
    runAsUser: 1000
   ...

ref: CouchbaseCluster Resource | Couchbase Docs

1 Like

Thank you , it works