Skip to content

Commit

Permalink
feat(eks): deprecate "kubectlEnabled: false" (#9454)
Browse files Browse the repository at this point in the history
When specifying `kubectlEnabled: false`, it _implicitly_ meant that the underlying resource behind the construct would be the stock `AWS::EKS::Cluster` resource instead of the custom resource used by default. This means that many new capabilities of EKS would not be supported (e.g. Fargate profiles).

Clusters backed by the custom-resource have all the capabilities (and more) of clusters backed by `AWS::EKS::Cluster`. Therefore, we decided that going forward we are going to support only the custom-resource backed solution.

To that end, after this change, defining an `eks.Cluster` with `kubectlEnabled: false` will throw an error with the following message:

    The "eks.Cluster" class no longer allows disabling kubectl support.
    As a temporary workaround, you can use the drop-in replacement class `eks.LegacyCluster`
    but bear in mind that this class will soon be removed and will no longer receive additional
    features or bugfixes. See #9332 for more details

Resolves #9332

BREAKING CHANGE: The experimental `eks.Cluster` construct no longer supports setting `kubectlEnabled: false`. A temporary drop-in alternative is `eks.LegacyCluster`, but we have plans to completely remove support for it in an upcoming release since `eks.Cluster` has matured and should provide all the needed capabilities. Please comment on #9332 if there are use cases that are not supported by `eks.Cluster`.


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
Elad Ben-Israel committed Aug 10, 2020
1 parent deffd40 commit dc6dbaa
Show file tree
Hide file tree
Showing 11 changed files with 1,184 additions and 344 deletions.
66 changes: 1 addition & 65 deletions packages/@aws-cdk/aws-eks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ cluster.addResource('mypod', {
```

In order to interact with your cluster through `kubectl`, you can use the `aws
eks update-kubeconfig` [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/eks/update-kubeconfig.html) command
eks update-kubeconfig` [AWS CLI command](https://docs.aws.amazon.com/cli/latest/reference/eks/update-kubeconfig.html)
to configure your local kubeconfig.

The EKS module will define a CloudFormation output in your stack which contains
Expand Down Expand Up @@ -411,8 +411,6 @@ Furthermore, when auto-scaling capacity is added to the cluster (through
of the auto-scaling group will be automatically mapped to RBAC so nodes can
connect to the cluster. No manual mapping is required any longer.

> NOTE: `cluster.awsAuth` will throw an error if your cluster is created with `kubectlEnabled: false`.
For example, let's say you want to grant an IAM user administrative privileges
on your cluster:

Expand Down Expand Up @@ -467,68 +465,6 @@ If you want to SSH into nodes in a private subnet, you should set up a
bastion host in a public subnet. That setup is recommended, but is
unfortunately beyond the scope of this documentation.

### kubectl Support

When you create an Amazon EKS cluster, the IAM entity user or role, such as a
[federated user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers.html)
that creates the cluster, is automatically granted `system:masters` permissions
in the cluster's RBAC configuration.

In order to allow programmatically defining **Kubernetes resources** in your AWS
CDK app and provisioning them through AWS CloudFormation, we will need to assume
this "masters" role every time we want to issue `kubectl` operations against your
cluster.

At the moment, the [AWS::EKS::Cluster](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-eks-cluster.html)
AWS CloudFormation resource does not support this behavior, so in order to
support "programmatic kubectl", such as applying manifests
and mapping IAM roles from within your CDK application, the Amazon EKS
construct library uses a custom resource for provisioning the cluster.
This custom resource is executed with an IAM role that we can then use
to issue `kubectl` commands.

The default behavior of this library is to use this custom resource in order
to retain programmatic control over the cluster. In other words: to allow
you to define Kubernetes resources in your CDK code instead of having to
manage your Kubernetes applications through a separate system.

One of the implications of this design is that, by default, the user who
provisioned the AWS CloudFormation stack (executed `cdk deploy`) will
not have administrative privileges on the EKS cluster.

1. Additional resources will be synthesized into your template (the AWS Lambda
function, the role and policy).
2. As described in [Interacting with Your Cluster](#interacting-with-your-cluster),
if you wish to be able to manually interact with your cluster, you will need
to map an IAM role or user to the `system:masters` group. This can be either
done by specifying a `mastersRole` when the cluster is defined, calling
`cluster.awsAuth.addMastersRole` or explicitly mapping an IAM role or IAM user to the
relevant Kubernetes RBAC groups using `cluster.addRoleMapping` and/or
`cluster.addUserMapping`.

If you wish to disable the programmatic kubectl behavior and use the standard
AWS::EKS::Cluster resource, you can specify `kubectlEnabled: false` when you define
the cluster:

```ts
new eks.Cluster(this, 'cluster', {
kubectlEnabled: false
});
```

**Take care**: a change in this property will cause the cluster to be destroyed
and a new cluster to be created.

When kubectl is disabled, you should be aware of the following:

1. When you log-in to your cluster, you don't need to specify `--role-arn` as
long as you are using the same user that created the cluster.
2. As described in the Amazon EKS User Guide, you will need to manually
edit the [aws-auth ConfigMap](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html)
when you add capacity in order to map the IAM instance role to RBAC to allow nodes to join the cluster.
3. Any `eks.Cluster` APIs that depend on programmatic kubectl support will fail
with an error: `cluster.addResource`, `cluster.addChart`, `cluster.awsAuth`, `props.mastersRole`.

### Helm Charts

The `HelmChart` construct or `cluster.addChart` method can be used
Expand Down
Loading

0 comments on commit dc6dbaa

Please sign in to comment.