Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[serviceaccount] Annotations as Values Support #109

Closed

Conversation

onelapahead
Copy link
Contributor

@onelapahead onelapahead commented Apr 21, 2023

I'd like to be able to include annotations on my operator's ServiceAccount as this is a fairly common need for a lot of apps using ServiceAccounts.

Felt like because ServiceAccounts might be special in this regard, I copied processer.ProcessObjMeta and modified it to include an annotations template and value.

Feels a bit hacky and open to feedback as to what the right approach should be.

Copy link
Owner

@arttor arttor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add sample annotations to one of service accounts in ./test_data and regenerate chart examples with cat test_data/sample-app.yaml | go run ./cmd/helmify examples/app and cat test_data/k8s-operator-kustomize.output | go run ./cmd/helmify examples/operator and push changes to your branch

@onelapahead
Copy link
Contributor Author

Thanks for the steer @arttor, this PR should be good to go now in that it has the test_data and examples update you asked for.

@onelapahead onelapahead requested a review from arttor April 28, 2023 20:13
@arttor
Copy link
Owner

arttor commented Apr 29, 2023

@hfuss, I apologize for not bringing up this question earlier, but could you please explain the use case for the serviceAccount annotation and how it differs from other objects?

I've tried to do some research but there is not much information:

  1. k8s doc says that you can annotate a Secret to use it as api token for ServiceAccount
  2. azure-specific IAM annotations. AWS also uses annotation to map serviceAccount to IAM role.

There may be other annotation use cases beyond serviceAccount or IAM. Third-party Kubernetes operators rely on annotations for autoscaling and other tasks.

Unfortunately, Helm does not offer a way to add custom annotations to chart objects, and the chart maintainer may not be aware of which operators the chart users will use and which chart objects they will need to annotate. As a result, it is common to add annotations manually after running helm install because all annotations will be retained during the next helm upgrade.

@onelapahead
Copy link
Contributor Author

Hey @arttor no worries, thanks for doing the initial research.

Specifically, my use case is I've written an operator which requires cloud access to either AWS or Azure. As you pointed out, being able to add IAM ServiceAccount annotations for those clouds simplifies how an operator / app needs to authenticate and assume roles.

Another use case for an app or an operator with a UI, might be wanting to create an OAuth client from a ServiceAccount within OpenShift: https://access.redhat.com/documentation/en-us/openshift_container_platform/4.3/html/authentication/using-service-accounts-as-oauth-client

In my use of open-source Helm charts such as External DNS, cert-manager, and External Secrets, it was very common for the chart maintainers to expose their ServiceAccount annotations for the same reason - allowing simpler cloud IAM authentication. Users of helmify writing cloud-aware operators might often want to offer the same IAM authentication methods for their charts

Your workaround to manually patch ServiceAccount annotations ourselves outside of Helm could introduce a lot of complexity if folks are trying to use Helm in a purely declarative / GitOps driven way such as ArgoCD or Flux.

@arttor
Copy link
Owner

arttor commented Apr 30, 2023

Understood. What do you think about the following approach?

  • add a function option to processor.ProcessObjMeta. Something like func WithAnnotations(v helmify.Values) MetaOpt ...
  • in ProcessObjMeta: template annotations to values.yaml if annotations and option are presented.
  • in service account processor: call ProcessObjMeta with WithAnnotations option.

@onelapahead
Copy link
Contributor Author

@arttor great feedback, took a stab at the functional option implementation as you described and ready for review.

Comment on lines +60 to +63
options := &options{
values: nil,
annotations: false,
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
options := &options{
values: nil,
annotations: false,
}
options := &options{}

labels:
{{- include "operator.labels" . | nindent 4 }}
annotations:
k8s.acme.org/some-meta-data: ACME Inc.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should k8s.acme.org/some-meta-data: ACME Inc. be moved to values.yaml?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arttor does helmify do that with labels?

@Starefossen
Copy link
Contributor

Starefossen commented Jun 19, 2023

Waiting for this 🙏🏻 This is need in order for service account to be able to link with Google Cloud Workload Identity.

kubectl annotate serviceaccount KSA_NAME \
    --namespace NAMESPACE \
    iam.gke.io/gcp-service-account=GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com

@@ -44,6 +44,8 @@ controllerManager:
region: east
type: user-node
replicas: 1
serviceaccount:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hfuss where does this come from? The generated serviceaccount.yaml says {{ toYaml .Values.controllerManager.annotations | nindent 4 }} and I don't see any changes to my values.yaml file.

Copy link
Contributor

@Starefossen Starefossen Jun 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arttor @hfuss I believe this this is the correct template nais@e7f1529 and nais@60c03d1

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Starefossen i think that this PR is stale. Can you please create PR from your fork? Please don't forget to regenerate examples (see #109 (review) )

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that @arttor 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Starefossen for picking up my slack and seeing this through! 🚀

@Starefossen
Copy link
Contributor

Superseded by #115

@arttor
Copy link
Owner

arttor commented Jun 24, 2023

closed with #115

@arttor arttor closed this Jun 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants