ArgoCD is trying to create a KSOPS Kubernete resource

[dalekurt]

Version

Argo CD v2.1.2
kustomizs-sops {Version:v4.3.0+ksops.v3.0.1 GitCommit:[email protected] BuildDate:2021-09-01T20:13:35Z GoOs:linux GoArch:amd64

What I've done so far

.sops.yaml

---
creation_rules:
  - encrypted_regex: '^(data|stringData)$'
    kms: "redacted"

Using a custom Argo CD container image with the kustomize-sops

Custom Dockerfile

ARG ARGO_CD_VERSION="v2.1.2"
ARG KSOPS_VERSION="v3.0.1"

FROM viaductoss/ksops:$KSOPS_VERSION as ksops-builder

FROM argoproj/argocd:$ARGO_CD_VERSION


# Switch to root for the ability to perform install
USER root

# Set the kustomize home directory
ENV XDG_CONFIG_HOME=$HOME/.config
ENV KUSTOMIZE_PLUGIN_PATH=$XDG_CONFIG_HOME/kustomize/plugin/

ARG PKG_NAME=ksops

# Override the default kustomize executable with the Go built version
COPY --from=ksops-builder /go/bin/kustomize /usr/local/bin/kustomize

# Copy the plugin to kustomize plugin path
COPY --from=ksops-builder /go/src/github.com/viaduct-ai/kustomize-sops/*  $KUSTOMIZE_PLUGIN_PATH/viaduct.ai/v1/${PKG_NAME}/

# Switch back to non-root user
USER argocd

The updated argocd-cm ConfigMap with the following:

data:
  kustomize.buildOptions: "--enable-alpha-plugins"

Additionally, the following files are in the repo (and path)

kustomization.yaml

generators:
  - ./secret-generator.yaml

secret-generator.yaml

apiVersion: viaduct.ai/v1
kind: ksops
metadata:
  name: demo-secret-generator
  namespace: demo
files:
  - ./secrets.yaml

secrets.yaml

redacted

Issue

Is this an expected result from ArgoCD attempting to create the KSOPS resource ?

[devstein]

@dalekurt

1. What does the redacted Argo CD diff look like?
2. What does it look like after you sync the resource? If it fails, can you share the error message?

[dalekurt]

Does this help?

secrets.yaml

apiVersion: v1
data:
  one: ++++++++
kind: Secret
metadata:
  labels:
    app.kubernetes.io/instance: demo
  name: my-secret
  namespace: demo
sops:
  age: []
  azure_kv: []
  encrypted_regex: ^(data|stringData)$
  gcp_kms: []
  hc_vault: []
  kms:
    - arn: 'redacted'
      aws_profile: ''
      created_at: '2021-09-23T15:20:35Z'
      enc: >-
        AQICAHiO4gFkYTvHhAZ5pzQfT6mZ/jBfGjZk/V3LIcBHK+SmmAHvIkFFLvpPZ87aqJPEd8gcAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMUl3ya7FYfUkfXvT9AgEQgDuZxzGOn/VCcPz9NAdxFvLYABS7APYLXoBRoFgWXR/6BNeig25ZC0rvv66yHAWbALvI3eqg38QAXmx/eg==
  lastmodified: '2021-09-23T15:20:35Z'
  mac: >-
    ENC[AES256_GCM,data:tURY4SuISWToV28vU90y77HGc5fpKdfDyIDJ61O8aoghMQ/8dgGPO2TWIzT36t4fSp1p5YY6k853AcwDdo+IcwDS9bLGsYYfm2s/4SG9zugoaRZmzSkujFTmcWDSK1OVibtk9z2NxwdSl+Tu+ssDYvR2Zr1z4EwQVcA7cg9bNQc=,iv:3TuU2O37IJULKJXiXSrq7v0xHS5/QMD6YDAI904bh5M=,tag:ASkRnwIRd9yLLvV4+864+A==,type:str]
  pgp: []
  version: 3.7.1

Application conditions

[devstein]

Thanks that is helpful, something is not correctly configured. My guess would have been that you are using resources instead of generators when referencing the ksops manifest in the kustomization.yaml file, but I see that is not the case in the one you shared.

Can you share the redacted diff shown when you click the App Diff button?

[dalekurt]

[devstein]

Interesting @dalekurt looking at that diff there must be something incorrect in either the kustomization.yaml or Argo CD Application, essentially kustomize is rendering both the Secret and the ksops manifest as resources instead of using the ksops manifest as a generator

[dalekurt]

@devstein Thank you for the help so far. Just to cover all the bases the manifest for the Argo CD Application is as follows:

project: demo
source:
  repoURL: 'https://github.com/redacted.git'
  path: >-
    path/to/demo
  targetRevision: dm/demo
  directory:
    recurse: true
    jsonnet: {}
destination:
  namespace: '*'
  name: sandbox
syncPolicy:
  automated:
    prune: true
    selfHeal: true

I will review the configuration as per your suggestion.

[devstein]

@dalekurt Two suggestions

1. Remove the directory section of the Application manifest

  directory:
    recurse: true
    jsonnet: {}

2. If that doesn't help, can you share the sourceType that Argo CD detects

kubectl -n argocd get apps demo -o yaml | grep -i sourceType

[dalekurt]

@devstein An update, I was able to resolve this issue. Apparently, the cluster had permission to access AWS KMS resources however the deployment(and pod) required a service account and annotation in order to use the KMS decrypt keys. Once that was done, KSOPS worked perfectly.

Thank you for the assist.

[sthomson-wyn]

For future debuggers: removing

  directory:
    recurse: true

fixed this issue for me