fluxcd not installing helm chart and crds

[tcurdt]

I am a little confused. I have this kustomization file

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - namespace.yaml
  - repo.yaml
  - helm.yaml
  - cluster.yaml

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: cloudnative-pg
  namespace: database
spec:
  interval: 60m
  url: https://cloudnative-pg.github.io/charts

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: cloudnative-pg
  namespace: database
spec:
  interval: 60m
  chart:
    spec:
      chart: cloudnative-pg
      version: "0.23.0"
      sourceRef:
        kind: HelmRepository
        name: cloudnative-pg
        namespace: database
      interval: 60m

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: foo
  namespace: database
spec:
  instances: 1
  primaryUpdateStrategy: unsupervised
  enableSuperuserAccess: true
  storage:
    size: 1Gi

but somehow the CRDs are not getting installed

flux check --kubeconfig .configs/a/kubeconfig
► checking prerequisites
✔ Kubernetes 1.32.0 >=1.28.0-0
► checking version in cluster
✔ distribution: flux-v2.4.0
✔ bootstrapped: false
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v1.1.0@sha256:4c75ca6c24ceb1f1bd7e935d9287a93e4f925c512f206763ec5a47de3ef3ff48
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.39.0@sha256:5b6c2e97055cfe69fe8996f48b53db039c136210dbc98c5631864a9e573d0e20
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.33.0@sha256:c6864684f96cfbb3a91c816084032bb019b302af8b63b0e06b12b4018a9a8242
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.4.0@sha256:e3b0cf847e9cdf47b19af0fbcfe22786b80b598e0caeea8b6d2a5f9c26a48a24
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.4.0@sha256:425309a159b15e07f7d97622effc79bc432a37ed55289dd465d37fa217a92a7d
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.4.1@sha256:3c5f0f022f990ffc0daf00e5b199548fc0fa6e7119e972318f0267081a332963
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta3
✔ buckets.source.toolkit.fluxcd.io/v1
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1
✔ helmreleases.helm.toolkit.fluxcd.io/v2
✔ helmrepositories.source.toolkit.fluxcd.io/v1
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta3
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed

kubectl --kubeconfig .configs/a/kubeconfig get kustomizations -A -w
NAMESPACE     NAME          AGE   READY   STATUS
flux-system   flux-system   36m   False   Cluster/database/foo dry-run failed: no matches for kind "Cluster" in version "postgresql.cnpg.io/v1"...

apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata:
  name: demo
  namespace: demo
spec:
  name: demo
  owner: demo
  cluster:
    name: foo

flux get helmreleases -A --kubeconfig .configs/a/kubeconfig
✗ no HelmRelease objects found in any namespace

flux get sources helm -A --kubeconfig .configs/a/kubeconfig
✗ no HelmRepository objects found in any namespace

Any suggestions where to dig deeper?

[matheuscscp]

You are trying to use the same Kustomization to apply both CRDs and CRs. That doesn't work, kustomize-controller gets stuck by the error you pasted here when trying to apply the CR because the CRD doesn't exist yet, and then it doesn't even try to apply the CRD. kustomize-controller even has logic to apply CRDs before CRs, but because in your case the CRDs are behind a helm chart, kustomize-controller can't know that it would need to apply the chart first, wait for helm-controller to reconcile the chart (effectively installing the CRD) and only then apply the CRs.

To solve this you need 2 Kustomizations, one for installing CRDs, and one for CRs. The second must use .spec.dependsOn to specify that it depends on the first one.

See the examples here:

https://github.com/fluxcd/flux2-kustomize-helm-example/tree/main/clusters%2Fproduction

[matheuscscp]

HelmRepository and HelmRelease can be deployed together, if helm-controller tries to reconcile HelmRelease and sees that HelmRepository is not ready yet, it requeues the reconciliation after an interval.

kustomize-controller is smart enough for deploying namespaces before everything else. And CRDs as well! But again, your Kustomization is not directly deploying any CRDs, it is instead deploying a HelmRelease that will deploy CRDs, which is very different.

[tcurdt]

Sorry, for maybe being a little slow here - and thanks for sticking with me.

I think I understand the problem now. And learned something new.
(I didn't know it was rules and not order that matters for the Kustomizations.)

I am just not entirely sure how to follow you advice and split the Kustomizations though.
It's clear I need to introduce a dependsOn relationship between

[namespace, repo, helm] <- [cluster]

And IIUC there is no k8s native way, so I need to use the flux kustomization to express that dependency.

This is what I tried:

https://github.com/tcurdt/kube/blob/main/flux/clusters/production/infrastructure/postgres.yaml

├── postgres-cluster
│   ├── backup.yaml
│   ├── cluster.yaml
│   └── kustomization.yaml
├── postgres-controller
│   ├── helm.yaml
│   ├── kustomization.yaml
│   ├── namespace.yaml
│   └── repo.yaml
├── postgres.yaml

where postgres.yaml has the two kustomize.toolkit.fluxcd.io/v1 with the dependencies defined.

---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: postgres-controller
  namespace: database
spec:
  path: ./postgres-controller
  # prune: true
  wait: true
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: postgres-cluster
  namespace: database
spec:
  dependsOn:
    - name: postgres-controller
  path: ./postgres-cluster
  # prune: true

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - external-ip-controller
  - local-storage
  - postgres.yaml
  - caddy

But I am still not seeing the CRDs getting installed.

Database/demo/demo dry-run failed: no matches for kind "Database" in version "postgresql.cnpg.io/v1"

[tcurdt]

It would be awesome if you had another pointer, @matheuscscp
I don't see where I got it wrong with the dependsOn

[matheuscscp]

What namespace is namespace.yaml creating? If it's creating the namespace database, then you need to change the namespaces of the two Flux Kustomizations inside postgres.yaml, because that namespace doesn't exist yet so their apply will fail. For example, use the namespace flux-system for them.

[tcurdt]

I tried that and changed it accordingly.

That still hasn't fix the problem though 😔

Still no matches for kind "Database"