Hi,
I’m using Terraform to deploy to k8s. I had it all working with the operator, cluster, + sync gateway.
I see the new operator 1.2 is out, so today I was working on updating to that. I noticed a couple changes:
- There is now an admission controller pod
- The operator no longer uses a
clusterroleand instead uses a standardrole
I have the admission controller working now, but the old operator script used to have a volume / volume mount that would mount a secret with a username/password. The new script removes that:
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
name: couchbase-operator
spec:
replicas: 1
selector:
matchLabels:
app: couchbase-operator
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: couchbase-operator
spec:
containers:
- args:
- --pod-create-timeout=10m
- --create-crd=false
command:
- couchbase-operator
env:
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
image: couchbase/operator:1.2.0
name: couchbase-operator
ports:
- containerPort: 8080
name: http
readinessProbe:
failureThreshold: 20
httpGet:
path: /readyz
port: http
initialDelaySeconds: 3
periodSeconds: 3
resources: {}
serviceAccountName: couchbase-operator
status: {}
Is this correct? When I spawn the operator like this, it fails to start the pod and the logs indicate:
$ kc logs couchbase-operator-admission-7c49f757d-z8dxm
I0508 20:57:50.499645 1 admission.go:300] couchbase-operator-admission 1.2.0 (release)
ml-dford1tb-01623:localhost dford$ kc logs couchbase-operator-676b4d94c4-dh6h9
time="2019-05-08T20:58:31Z" level=info msg="couchbase-operator v1.2.0 (release)" module=main
panic: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
goroutine 1 [running]:
github.com/couchbase/couchbase-operator/pkg/util/k8sutil.MustNewKubeClient(0xc0003862a0, 0x23)
/home/couchbase/jenkins/workspace/couchbase-operator-build/goproj/src/github.com/couchbase/couchbase-operator/pkg/util/k8sutil/k8sutil.go:69 +0x6a
main.main()
/home/couchbase/jenkins/workspace/couchbase-operator-build/goproj/src/github.com/couchbase/couchbase-operator/cmd/operator/main.go:93 +0x5e1
My old script used to mount this secret…but I removed that b/c it is gone from the new YAML files (note this is Terraform HCL script):
resource "kubernetes_deployment" "couchbase-operator" {
metadata {
name = "couchbase-operator"
labels = {
app = "couchbase-operator"
}
}
spec {
# we don't want more than one replica
replicas = 1
selector {
match_labels {
app = "couchbase-operator"
}
}
template {
metadata {
labels {
app = "couchbase-operator"
}
}
spec {
service_account_name = "${var.cb-operator-service-account-name}"
container {
name = "couchbase-operator"
image = "${var.cb-operator-image}"
command = ["couchbase-operator"]
args = "${var.cb-operator-args}"
env {
name = "MY_POD_NAMESPACE"
value_from {
field_ref {
field_path = "metadata.namespace"
}
}
}
env {
name = "MY_POD_NAME"
value_from {
field_ref {
field_path = "metadata.name"
}
}
}
port {
name = "readiness-port"
container_port = 8080
}
# must explicitly mount with terraform/k8s provider
# https://github.com/kubernetes/kubernetes/issues/27973#issuecomment-463903176
volume_mount {
mount_path = "/var/run/secrets/kubernetes.io/serviceaccount"
name = "${kubernetes_service_account.couchbase-operator.default_secret_name}"
read_only = true
}
readiness_probe {
http_get {
path = "/readyz"
port = "readiness-port"
}
initial_delay_seconds = 3
period_seconds = 3
failure_threshold = 19
}
}
volume {
name = "${kubernetes_service_account.couchbase-operator.default_secret_name}"
secret {
secret_name = "${kubernetes_service_account.couchbase-operator.default_secret_name}"
}
}
}
}
}
}