Flux completely stops syncing with ACR

[LotusMD]

Context

I currently have an AKS cluster and my Images are in an ACR. I also use Weave Cloud for managing Flux CD. I modified the flux agent manifest to connect flux to the ACR (see issue #1694).

This is the volumes section of the flux agent manifest. I applied the necessary changes to the flux manifest.

             ...
             volumeMounts:
                - name: git-key
                  mountPath: /etc/fluxd/ssh
                - name: keygen
                  mountPath: /var/fluxd/ssh
                - name: acr-credentials
                  mountPath: /etc/kubernetes/azure.json
          serviceAccountName: weave-flux
          volumes:
            - name: git-key
              secret:
                defaultMode: 256
                secretName: flux-git-deploy
            - name: keygen
              emptyDir:
                medium: Memory
            - name: acr-credentials
              hostPath:
                path: /etc/kubernetes/azure.json
                type: ""

This works perfectly fine. I can see the images from the Weave Cloud UI as well as from fluxctl list-images.

Issue

The issue is that sometimes it just stops working. At some point the new images never make it fluxctl list-images even though there are new images in my ACR. There doesn't seem to be anything happening it's really just a matter of time before it stops syncing.

Patch

Right now, the way I make it work is by reapplying the flux agent manifest (URL is given in section bellow) modified with the volume mounts showed aboce. I manually run kubectl apply -f flux.yml.

Additional context

This is the URL I use to download the manifest:

https://cloud.weave.works/k8s/flux.yaml?k8s-version=$(kubectl version | base64 | tr -d '\n')&t=5dg84ye3t4tnpbs5euxsdpahxw13bumg&flux-version=%5E1&git-branch=master&git-label=flux-brisk-blossom-23&git-path=azure&git-url=git%40bitbucket.org%3Alotusmd%2Fmanifests.git

Here is the url with the kubectl version | base64 | tr -d '\n' parsed on my system:

https://cloud.weave.works/k8s/flux.yaml?k8s-version=Q2xpZW50IFZlcnNpb246IHZlcnNpb24uSW5mb3tNYWpvcjoiMSIsIE1pbm9yOiIxNisiLCBHaXRWZXJzaW9uOiJ2MS4xNi42LWJldGEuMCIsIEdpdENvbW1pdDoiZTdmOTYyYmE4NmY0Y2U3MDMzODI4MjEwY2EzNTU2MzkzYzM3N2JjYyIsIEdpdFRyZWVTdGF0ZToiY2xlYW4iLCBCdWlsZERhdGU6IjIwMjAtMDEtMTVUMDg6MjY6MjZaIiwgR29WZXJzaW9uOiJnbzEuMTMuNSIsIENvbXBpbGVyOiJnYyIsIFBsYXRmb3JtOiJkYXJ3aW4vYW1kNjQifQpTZXJ2ZXIgVmVyc2lvbjogdmVyc2lvbi5JbmZve01ham9yOiIxIiwgTWlub3I6IjE1IiwgR2l0VmVyc2lvbjoidjEuMTUuMTAiLCBHaXRDb21taXQ6ImJhMWNlNDEyMmQ2N2IwZjczOTlhNDFhZmJiZDhhNzI1OGQyMmVjYjgiLCBHaXRUcmVlU3RhdGU6ImNsZWFuIiwgQnVpbGREYXRlOiIyMDIwLTA0LTIxVDAwOjEzOjI1WiIsIEdvVmVyc2lvbjoiZ28xLjEyLjEyIiwgQ29tcGlsZXI6ImdjIiwgUGxhdGZvcm06ImxpbnV4L2FtZDY0In0K&t=5dg84ye3t4tnpbs5euxsdpahxw13bumg&flux-version=%5E1&git-branch=master&git-label=flux-brisk-blossom-23&git-path=azure&git-url=git%40bitbucket.org%3Alotusmd%2Fmanifests.git

You should change the git-path, git-label, and git-url before downloading the manifest

Versions

• Flux version: 1.19.0
• Kubernetes version: From kubectl version
  • Client Version: Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0"
  • Server Version:Major:"1", Minor:"15", GitVersion:"v1.15.10"
• Git provider: Bitbucket
• Container registry provider: Azure Container Registry

[kingdonb]

It sounds like your registry credentials are expiring.

Apologies about the length of time that has elapsed since your inquiry. As efforts have been focused on Flux v2, I've recently joined the team to focus on Flux v1 issues, and we have developed a support policy page with references to how community works in our new architecture for Flux v2, different ways to meet us for help. If you've been following our development efforts then of course we hope you are able to upgrade, here's more info on how to find support with that: https://fluxcd.io/support/

If this is still affecting you, I will be happy to reopen and/or troubleshoot with you in the #flux slack channel on CNCF, but in the interest of reducing the number of open issues not directly related to supporting Flux v1 in maintenance mode down to something manageable, and respecting you may have moved on already, I will go ahead and close out this issue for now.