ECR Updater in a separate namespace - Secrets is forbidden

[artyom-p]

Hi, I'm quite new to k8s, so maybe just general guidance needed. I'm trying to run ecr-updater under a separate namespace, but I can't make it work. It works perfectly when I launch it in the default namespace tho.

starting the thing
Running update loop
starting update loop
ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': '028d6d8b-cddf-45e0-aaad-2a3f78c75d5c', 'Cache-Control': 'no-cache, private', 'Content-Length': '283', 'Content-Type': 'application/json', 'Date': 'Thu, 04 Mar 2021 08:32:58 GMT', 'X-Content-Type-Options': 'nosniff'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets is forbidden: User "system:serviceaccount:gcc:ecr-updater" cannot list resource "secrets" in API group "" at the cluster scope","reason":"Forbidden","details":{"kind":"secrets"},"code":403}

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ecr-updater
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  - configmaps
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
  - list

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ecr-updater
  namespace: gcc
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ecr-updater
subjects:
- kind: ServiceAccount
  name: ecr-updater
  namespace: gcc

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ecr-updater
  namespace: gcc
secrets:
- name: ecr-updater-token-fp7zx

apiVersion: v1
data:
  AWS_ACCESS_KEY_ID: BASE64ACCESSKEY
  AWS_SECRET_ACCESS_KEY: BASE64SECRET
kind: Secret
metadata:
  name: ecr-aws-credentials
  namespace: gcc
type: Opaque

data:
  .dockerconfigjson: SOMEVALUE
kind: Secret
metadata:
  name: ecr
  namespace: gcc
type: kubernetes.io/dockerconfigjson

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "3"
  generation: 3
  labels:
    k8s-app: ecr-updater
  name: ecr-updater
  namespace: gcc
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: ecr-updater
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        k8s-app: ecr-updater
        name: ecr-updater
        namespace: gcc
    spec:
      containers:
      - env:
        - name: K8S_PULL_SECRET_NAME
          value: ecr:gcc
        - name: ECR_CREATE_MISSING
          value: "false"
        - name: AWS_DEFAULT_REGION
          value: eu-west-1
        - name: AWS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              key: AWS_ACCESS_KEY_ID
              name: ecr-aws-credentials
        - name: AWS_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              key: AWS_SECRET_ACCESS_KEY
              name: ecr-aws-credentials
        image: trondhindenes/k8s-ecrupdater
        imagePullPolicy: Always
        name: ecrupdater
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - image: palmstonegames/kubectl-proxy
        imagePullPolicy: Always
        name: kubectl-proxy
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: ecr-updater
      serviceAccountName: ecr-updater
      terminationGracePeriodSeconds: 30

[trondhindenes]

hm that looks mostly right. You might need a Clusterrolebinding instead of a rolebinding.

[amjad489]

i created it like this and works.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ecr-updater
rules:
  - apiGroups:
      - ""
    resources:
      - secrets
      - configmaps
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs: ["get", "list", "watch",  "update", "patch"]

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ecr-updater
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ecr-updater
subjects:
- kind: ServiceAccount
  name: ecr-updater
  namespace: ecr-updater