kapp adds default additional ownership and label scoping rules

[yanniszark]

Hi all!
I'm new to kapp and trying it out for our use-case.
I had resources preapplied to my cluster and was trying to get kapp to adopt them.
The command I used is:
kapp deploy -a myapp -f dump.yaml

The command failed with:

Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"ml-pipeline-viewer-crd", "app.kubernetes.io/component":"pipelines-viewer", "app.kubernetes.io/name":"pipelines-viewer", "kapp.k14s.io/a$
p":"1598878090435084849"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable (reason: Invalid)

This is because kapp also tries to inject the label in things with spec.selector, like Deployment. But that field is immutable.
Is this by design? Why isn't kapp just applying the labels to the top-level object?

[cppforlife]

kapp by default applies additional ownership and label scoping rules to resources for the purpose of better tracking and resource "isolation" (see ownershipLabelRules and labelScopingRules in https://carvel.dev/kapp/docs/latest/config/#ownershiplabelrules). you can disable them by adding annotations kapp.k14s.io/disable-default-ownership-label-rules: "" and kapp.k14s.io/disable-default-label-scoping-rules: "". (typically is used when adopting existing resources vs having kapp create them from scratch).

Why isn't kapp just applying the labels to the top-level object?

you are also right about top level metadata.labels update. steps above happen in addition to this.

[cppforlife]

closing per explanation above. feel free to reopen if more discussion is wanted.

[revolunet]

Also having this "error";

What are the effects of adding kapp.k14s.io/disable-default-ownership-label-rules: "" and kapp.k14s.io/disable-default-label-scoping-rules: "" to the deployments ?

[gabegorelick]

This seems pretty common when importing existing resources. It would be nice if kapp were smarter about not modifying immutable fields and if the docs provided more guidance about how to handle this.

[pivotaljohn]

@gabegorelick pointed out:

This seems pretty common when importing existing resources. It would be nice if kapp were smarter about not modifying immutable fields ...

Yes. #214 is where we're shaping the improved out-of-the-box experience. 👍🏼

... and if the docs provided more guidance about how to handle this.
Agreed. As a stopgap, we added an item to the FAQ (i.e. carvel-dev/carvel#84 yielded https://carvel.dev/kapp/docs/latest/faq/#migrating-from-kubectl-apply-to-kapp) as a contextualized instance of this. Which is helpful....

But there's evidence that the whole configuring rules mechanism is not sufficiently documented in @revolunet's question:

What are the effects of adding kapp.k14s.io/disable-default-ownership-label-rules: "" and kapp.k14s.io/disable-default-label-scoping-rules: "" to the deployments ?

Today, most of the kapp docs are reference materials. The FAQ section is currently serving to attempt to fill the gap on "how to"s. But this issue is just one example of where without a guide, attempting to configure a kapp deploy can have lots of surprises.

While the roadmap for the maintainers does not currently have a slot for writing such guides, we can at least start to capture the needs. I've done so here: kapp How to Guides (epic).

[ringerc]

See also FAQ "... Field is immutable error"

The FAQ doesn't mention but the CLI option --apply-default-update-strategy=fallback-on-replace can be used to force replacement of resources kapp added the unwanted labels to, so you'll want to use

kapp deploy --default-label-scoping-rules=false --apply-default-update-strategy=fallback-on-replace ...