Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Promote addon docs to first level menu item #10355

Merged
merged 1 commit into from
Dec 4, 2020
Merged

Promote addon docs to first level menu item #10355

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
241 changes: 241 additions & 0 deletions docs/addons.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
# kOps addons

kOps supports two types of addons:

* Managed addons, which are configurable through the [cluster spec](cluster_spec.md)
* Static addons, which are manifest files that are applied as-is

## Managed addons

The following addons are managed by kOps and will be upgraded following the kOps and kubernetes lifecycle, and configured based on your cluster spec. kOps will consider both the configuration of the addon itself as well as what other settings you may have configured where applicable.

### Available addons

#### Cluster autoscaler
{{ kops_feature_table(kops_added_default='1.19', k8s_min='1.15') }}

Cluster autoscaler can be enabled to automatically adjust the size of the kubernetes cluster.

```yaml
spec:
clusterAutoscaler:
enabled: true
skipNodesWithLocalStorage: true
skipNodesWithSystemPods: true
```

Read more about cluster autoscaler in the [official documentation](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler).



#### Metrics server
{{ kops_feature_table(kops_added_default='1.19') }}

Metrics Server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.

```yaml
spec:
metricsServer:
enabled: true
```

Read more about Metrics Server in the [official documentation](https://github.com/kubernetes-sigs/metrics-server).


#### Node local DNS cache
{{ kops_feature_table(kops_added_default='1.18', k8s_min='1.15') }}

NodeLocal DNSCache can be enabled if you are using CoreDNS. It is used to improve the Cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet.

`memoryRequest` and `cpuRequest` for the `node-local-dns` pods can also be configured. If not set, they will be configured by default to `5Mi` and `25m` respectively.

If `forwardToKubeDNS` is enabled, kubedns will be used as a default upstream

```yaml
spec:
kubeDNS:
provider: CoreDNS
nodeLocalDNS:
enabled: true
memoryRequest: 5Mi
cpuRequest: 25m
```

#### Node termination handler

{{ kops_feature_table(kops_added_default='1.19') }}

Node Termination Handler ensures that the Kubernetes control plane responds appropriately to events that can cause your EC2 instance to become unavailable, such as EC2 maintenance events, EC2 Spot interruptions, ASG Scale-In, ASG AZ Rebalance, and EC2 Instance Termination via the API or Console. If not handled, your application code may not stop gracefully, take longer to recover full availability, or accidentally schedule work to nodes that are going down.

```yaml
spec:
nodeTerminationHandler:
enabled: true
```

## Static addons

The command `kops create cluster` does not support specifying addons to be added to the cluster when it is created. Instead they can be added after cluster creation using kubectl. Alternatively when creating a cluster from a yaml manifest, addons can be specified using `spec.addons`.
```yaml
spec:
addons:
- manifest: kubernetes-dashboard
- manifest: s3://kops-addons/addon.yaml
```

This document describes how to install some common addons and how to create your own custom ones.

### Available addons

#### Ambassador

The [Ambassador API Gateway](https://getambassador.io/) provides all the functionality of a traditional ingress
controller (i.e., path-based routing) while exposing many additional capabilities such as authentication, URL rewriting,
CORS, rate limiting, and automatic metrics collection.

Install using:
```
kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/ambassador/ambassador-operator.yaml
```

Detailed installation instructions in the [addon documentation](https://github.com/kubernetes/kops/blob/master/addons/ambassador/README.md).
See [Ambassador documentation](https://www.getambassador.io/docs/) on configuration and usage.

#### Dashboard

The [dashboard project](https://github.com/kubernetes/dashboard) provides a nice administrative UI:

Install using:
```
kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.10.1.yaml
```

And then follow the instructions in the [dashboard documentation](https://github.com/kubernetes/dashboard/wiki/Accessing-Dashboard---1.7.X-and-above) to access the dashboard.

The login credentials are:

* Username: `admin`
* Password: get by running `kops get secrets kube --type secret -oplaintext` or `kubectl config view --minify`

##### RBAC

It's necessary to add your own RBAC permission to the dashboard. Please read the [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) docs before applying permissions.

Below you see an example giving **cluster-admin access** to the dashboard.

```yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
labels:
k8s-app: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
```

### Monitoring with Heapster - Standalone

**This addons is deprecated. Please use metrics-server instead**

Monitoring supports the horizontal pod autoscaler.

Install using:
```
kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/monitoring-standalone/v1.11.0.yaml
```
Please note that [heapster is retired](https://github.com/kubernetes/heapster/blob/master/docs/deprecation.md). Consider using [metrics-server](https://github.com/kubernetes-incubator/metrics-server) and a third party metrics pipeline to gather Prometheus-format metrics instead.

### Monitoring with Prometheus Operator + kube-prometheus

The [Prometheus Operator](https://github.com/coreos/prometheus-operator/) makes the Prometheus configuration Kubernetes native and manages and operates Prometheus and Alertmanager clusters. It is a piece of the puzzle regarding full end-to-end monitoring.

[kube-prometheus](https://github.com/coreos/prometheus-operator/blob/master/contrib/kube-prometheus) combines the Prometheus Operator with a collection of manifests to help getting started with monitoring Kubernetes itself and applications running on top of it.

```console
kubectl apply -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/prometheus-operator/v0.26.0.yaml
```

### Route53 Mapper

**This addon is deprecated. Please use [external-dns](https://github.com/kubernetes-sigs/external-dns) instead.**

Please note that kOps installs a Route53 DNS controller automatically (it is required for cluster discovery).
The functionality of the route53-mapper overlaps with the dns-controller, but some users will prefer to
use one or the other.
[README for the included dns-controller](https://github.com/kubernetes/kops/blob/master/dns-controller/README.md)

route53-mapper automates creation and updating of entries on Route53 with `A` records pointing
to ELB-backed `LoadBalancer` services created by Kubernetes. Install using:

The project is created by wearemolecule, and maintained at
[wearemolecule/route53-kubernetes](https://github.com/wearemolecule/route53-kubernetes).
[Usage instructions](https://github.com/kubernetes/kops/blob/master/addons/route53-mapper/README.md)

```
kubectl apply -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/route53-mapper/v1.3.0.yml
```

### Custom addons

The docs about the [addon management](development/addons.md#addon-management) describe in more detail how to define a addon resource with regards to versioning.
Here is a minimal example of an addon manifest that would install two different addons.

```yaml
kind: Addons
metadata:
name: example
spec:
addons:
- name: foo.addons.org.io
version: 0.0.1
selector:
k8s-addon: foo.addons.org.io
manifest: foo.addons.org.io/v0.0.1.yaml
- name: bar.addons.org.io
version: 0.0.1
selector:
k8s-addon: bar.addons.org.io
manifest: bar.addons.org.io/v0.0.1.yaml
```

In this example the folder structure should look like this;

```
addon.yaml
foo.addons.org.io
v0.0.1.yaml
bar.addons.org.io
v0.0.1.yaml
Comment on lines +211 to +215
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that this is just copy/pasted but this code block doesn't make any sense to me. Maybe someone can clarify this with a tree output or something in a separate PR.

```

The yaml files in the foo/bar folders can be any kubernetes resource. Typically this file structure would be pushed to S3 or another of the supported backends and then referenced as above in `spec.addons`. In order for master nodes to be able to access the S3 bucket containing the addon manifests, one might have to add additional iam policies to the master nodes using `spec.additionalPolicies`, like so;
```yaml
spec:
additionalPolicies:
master: |
[
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": ["arn:aws:s3:::kops-addons/*"]
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": ["arn:aws:s3:::kops-addons"]
}
]
```
The masters will poll for changes in the bucket and keep the addons up to date.
50 changes: 2 additions & 48 deletions docs/cluster_spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ The complete list of keys can be found at the [Cluster](https://pkg.go.dev/k8s.i

On this page, we will expand on the more important configuration keys.

The documentation for the optional addons can be found on the [addons page](/addons)

## api

This object configures how we expose the API:
Expand Down Expand Up @@ -650,25 +652,6 @@ spec:

**Note:** If you are upgrading to CoreDNS, kube-dns will be left in place and must be removed manually (you can scale the kube-dns and kube-dns-autoscaler deployments in the `kube-system` namespace to 0 as a starting point). The `kube-dns` Service itself should be left in place, as this retains the ClusterIP and eliminates the possibility of DNS outages in your cluster. If you would like to continue autoscaling, update the `kube-dns-autoscaler` Deployment container command for `--target=Deployment/kube-dns` to be `--target=Deployment/coredns`.

## Node local DNS cache
{{ kops_feature_table(kops_added_default='1.18', k8s_min='1.15') }}

NodeLocal DNSCache can be enabled if you are using CoreDNS. It is used to improve the Cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet.

`memoryRequest` and `cpuRequest` for the `node-local-dns` pods can also be configured. If not set, they will be configured by default to `5Mi` and `25m` respectively.

If `forwardToKubeDNS` is enabled, kubedns will be used as a default upstream

```yaml
spec:
kubeDNS:
provider: CoreDNS
nodeLocalDNS:
enabled: true
memoryRequest: 5Mi
cpuRequest: 25m
```

## kubeControllerManager
This block contains configurations for the `controller-manager`.

Expand All @@ -688,35 +671,6 @@ spec:

For more details on `horizontalPodAutoscaler` flags see the [official HPA docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) and the [kOps guides on how to set it up](horizontal_pod_autoscaling.md).

## Metrics server
{{ kops_feature_table(kops_added_default='1.19') }}

Metrics Server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.

```yaml
spec:
metricsServer:
enabled: true
```

Read more about Metrics Server in the [official documentation](https://github.com/kubernetes-sigs/metrics-server).


## Cluster autoscaler
{{ kops_feature_table(kops_added_default='1.19', k8s_min='1.15') }}

Cluster autoscaler can be enabled to automatically adjust the size of the kubernetes cluster.

```yaml
spec:
clusterAutoscaler:
enabled: true
skipNodesWithLocalStorage: true
skipNodesWithSystemPods: true
```

Read more about cluster autoscaler in the [official documentation](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler).

### Feature Gates

Feature gates can be configured on the kubelet.
Expand Down
Loading