As a cluster admin, I can configure separate set of credentials for deployment from different AKS namespaces

[hlarubi]

Hi,

I'm new to ASO, so I might have missed something that's already in place. If so, would appreciate the pointer in the right direction or docs that describe the config I'm looking for.

This user story covers the following:
Allowing cluster administrator to create multiple service principals / managed identities in a administrator provided namespace to ensure user deployments from each AKS namespace have specific permissions (ex: users can only deploy Azure services to a specific resource group when creating ASO custom resources).

The use of a single cluster identity for service provisioning across namespaces doesn't cleanly support cluster multi-tenancy, as user permissions in a given namespace can be leveraged to provision services across any Azure resource group, even the user doesn't have direct permission to this resource group.

The goal is to enable permission model that allows AKS namespaces to be used by multiple teams with different resource groups and Azure RBAC.

Acceptance Criteria
User can manage Azure services using Azure Service Operator via namespace provided service principal / managed identity

Different namespaces can be configured with their own security identity and corresponding Azure RBAC

Administrator has a way to manage assignment of access to Azure services on a per namespace basis (ex: via a set of configmaps / secrets in a "managed" namespace)

[frodopwns]

@hlarubi

How important is the part about having the SPs exist in a single admin namespace as opposed to putting them in the namespaces where they would be used?

[hlarubi]

@frodopwns ,

We want to be able to map who can provision services to specific resource groups via RBAC. Unless there is a way to use the identity of the deployer to determine Azure permissions or ASO is using a unique resource type that I can limit RBAC for in the user namespace, I need a way to configure an admin namespace.

If there are other ideas, I'm happy to disucss.

Thanks!

[eadasiak]

I have a similar need for the operator to support multitenancy, but a slight variation on the use case where use of the operator is scoped to service teams that have been provisioned a namespace. If they have an Azure subscription and the credentials to do so, they would be able to opt-in to using it by configuring their namespace without cluster operator intervention. RBAC within the namespace would determine which group members within a team had permissions to configure the namespace for use by the operator (e.g,. the namespace-admins group) and to create Azure resources (e.g., namespace-edit group). I guess the main difference between what @hlarubi has described is that there's an assumption that each service team has their own Azure subscription and can manage it independently.

Ideally, the Azure Operator could be installed in a namespace-only mode by a cluster administrator. There would be no cluster-wide credentials configured in this mode. Instead, when the operator receives a resource provisioning request, it would check for the existence of the secret object azureoperatorsettings within the namespace specified in the resource's manifest. That secret object would contain the requisite tenant-, subscription-, and client-ID's along with the associated client secret. The operator would then use those credentials to complete the provisioning request.

I'm sure I haven't fully described all the details required, but was hoping this might be enough to start the conversation.

[frodopwns]

@hlarubi @eadasiak could you take a look at PR #1206 and comment on whether it would be useful to your use cases?

[hlarubi]

@frodopwns Thanks! Yes. This looks good. I like the strict modes.

Please let me know when the change is merged.

[eadasiak]

@frodopwns - this POC looks great and would definitely work in my use case.

[matthchr]

Adding some additional requirements and questions from customers I have spoken with:

1. A way to specify the credentials to use with a particular resource at the resource level, rather than at the namespace level.
2. Do we support a single identity (SP or MI) with access to multiple subscriptions as the way to multitenancy, or many identities each with access to a single subscription? Or do we support both of those styles and the user can choose which works best for their model? In the single identity with access to multiple subscriptions, there still needs to be a way to specify on a particular resource which subscription it should be created in - is this via a subscription parameter or some other way?

[babbageclunk]

We've implemented a different approach to multitenancy from the one outlined in this bug but it enables the original use case (different Azure credentials used for resources from different kubernetes namespaces). The key features (configurable target-namespaces and operator mode) are available in both ASO v1 and v2.
Details about how it works are in the design document but it needs user-level documentation about how to deploy the operator in this configuration.

[matthchr]

We're planning on investigating this further for beta.1 of ASOv2 as well

[psontag]

I have seen that there were some promising changes merged for beta.4.

I think the mulitenancy docs mention the new approach under option 2 but there is no further documentation how to use it.
Did I miss anything or is the documentation not ready yet?

[super-harsh]

@philipp-sontag-by We do have new documentation on our docs website already. I've shared the design and 'how to use' links below. Thanks

Multitenancy usage documentation
Multitenancy design discussion