Restrict update automation to certain folders in a repository

[PhilippMT]

Hi there,

I can see there was an "path" field in the spec which was discussed in #24 and removed with pull request #64 due to missing use-cases. I might have a possible use-case. The flux2 documentation states how to setup up multiple clusters/stages in different folders of the same repository. Like the folder "dev" is deployed into the dev-cluster, the folder "prod" to the prod-cluster an so on. But this can cause conflicts. If different clusters have ImagePolicies with the same name, but different spec, the automation-controller of the dev-cluster will not only adjust the version-tags in the "dev" folder, but also in the folders of the other stages. Possible solutions would be:

1. to use different branches for each stage, or even dedicated repositories but this would limit the use of a "base-kustomization" folder which is reused in all stages
2. to use different policy names in all stages. But there is still the eventuality that a developer accidentally uses a policy name which is already used in an other stage and breaks something.

To implement the "path" field would be a big improvement in this case.

Best regards
Philipp

[jvelasquez]

Duplicate of #72

[squaremo]

Not a duplicate of #72, just similar subject matter. #72 is specifically about whether image-automation-controller should honour the ignore field in a GitRepository (it's a reasonable expectation; but it raises some tricky questions).

This issue approaches filtering files in the git repository from another direction: what do you do if you want to run two or more different specs for automation on the same git repository? #72 doesn't solve this satisfactorily -- you would have to have GitRepository objects pointing at the same git repository with each ignoring all the bits they don't want to automate (which might change, if you e.g., add another cluster). It's a rather indirect way of serving this use case.

[stevejr]

I agree that this is going to be a common use case if Flux v2 users are encouraged to build fleet/multi tenant repos. Right now the Image automation is not usable when you have more than 1 cluster with differing image policy specs and the app deployment manifest (be that a HelmRelease, Deployment, Pod etc) stored in a single repo and then using Kustomize to patch accordingly.

For example, I have a single fleet repo that contains apps, infrastructure and clusters folders. Flux is bootstrapped against the folder cluster/. Within that folder are details of what flux should be managing in the cluster, one of those details is a Kustomization object that points to the folder apps/ (this is where the details of any non-infrastructure related apps are held). Within apps I have a base/ folder structure for each app that is available for my clusters. Then within apps/ I source in base/ and patch accordingly, including the HelmRelease and setting the Image policy marker, and finally creating the ImagePolicy I want for that app in that specific cluster.

The issue with ImagePolicy and hence this issue being raised, is that when Image policies with differing specs are created in different clusters BUT created with the same image policy name and namespace, the Image Automation logic will scan the whole git repo, find a marker and update it no matter if that marker belongs in a file that the flux for that cluster is meant to monitor.

The Flux V2 documentation provides a guide for Image Automation - https://toolkit.fluxcd.io/guides/image-update/ - which implies that clusters can have differing Image policy specs. However, there are no details in the guide in how to implement this. The guide only shows the implementation for the Production CI/CD workflow.

Right now it seems the only way to implement this is to make sure the ImagePolicy name is unique in every cluster which is doable but just adds another layer of management that shouldn't be needed.

[relu]

The way I currently handle this in a repository where I have multiple overlays corresponding to different environments is by adding a suffix to the ImagePolicy resources via kustomize so that I can then refer to them using a unique name per environment. It is probably not the best approach but I found it working for my particular use case.

I think the best way to tackle this would be through implementing another attribute in the customize fieldmeta marker. Here's a possible design:

# automation.yaml
apiVersion: image.toolkit.fluxcd.io/v1alpha1
kind: ImageUpdateAutomation
metadata:
  name: default
spec:
  ...
  update:
    strategy: Setters
    selector: uniqueid # <-- this bit here

# kustomization.yaml
images:
- name: app
  newName: someorg/someapp
  newTag: v1.0.0 # {"$imagepolicy": "flux-system:default:tag", "selector": "uniqueid"}

I'm not sure it's possible to implement it this way.

[stevejr]

Thanks @relu. I have done the same 0 my suffix is the cluster name.

[darksinge]

I'm in the process of migrating from flux v1 to v2 with a multi-cluster/multi-environment setup and this has caused a giant headache. I had expected the ImageUpdateAutomation in each environment to respect the --path specified when bootstrapping flux. I've spent the last few hours trying to figure out why it seemed like the ImageUpdateAutomation resources in two different clusters seemed to be competing over who got to update my image policy markers across the entire gitops repository. Now I know why :)

I can imagine many other developers running into this same unexpected behavior, and I don't believe this is mentioned in the docs.

[nomeelnoj]

I would think that you could get around this by telling the GitRepository object to ignore and folder paths where you dont want flux to commit back to.

But, that also seems to not be true. I tried updating the main GitRepository resource to exclude any overlays that were not for dev and yet when a dev image is pushed, all the overlays for all envs get updated if the ImagePolicy markers have the same name.

I would think that if flux is only tarring up the folders as specified in the GitRepository ignore field that it would not be able to write to files it does not have access to, but this seems to not work, either.