elastic · jmlrt · Apr 28, 2020 · Apr 22, 2020 · Apr 22, 2020 · Apr 22, 2020
diff --git a/BREAKING_CHANGES.md b/BREAKING_CHANGES.md
@@ -0,0 +1,96 @@
+# Breaking changes
+
+
+## 7.6.2 - 2020/03/31
+
+### Kibana default resources
+
+Kibana default resources (cpu/memory requests and limits) are increased in 
+[#540][].
+
+This change may impact cpu/memory available resources capacity in your 
+Kubernetes cluster.
+
+To come back to former default values, use the following values:
+
+```yaml
+extraEnvs:
+- name: "NODE_OPTIONS"
+  value: ""
+resources:
+  requests:
+    cpu: "100m"
+    memory: "500Mi"
+  limits:
+    cpu: "1000m"
+    memory: "1Gi"
+```
+
+
+## 7.6.0 - 2020/02/11
+
+### Elasticsearch default resources
+
+Elasticsearch default cpu requests is increased in [#458][] following our 
+recommendation that resources requests and limits should have the same values.
+
+This change may impact available cpu capacity in your Kubernetes cluster.
+
+To come back to former default values, use the following values:
+
+```yaml
+resources:
+  requests:
+    cpu: "100m"
+```
+
+
+## 7.5.0 - 2019/12/02
+
+### Metricbeat kube-state-metrics upgrade
+
+[kube-state-metrics][] chart dependency is upgraded from 1.6.0 to 2.4.1 in 
+[#352][]. This is causing Metricbeat chart upgrade from versions < 7.5.0 failing
+with the following error:
+
+```
+UPGRADE FAILED
+Error: Deployment.apps "metricbeat-kube-state-metrics" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"kube-state-metrics"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && Deployment.apps "metricbeat-metricbeat-metrics" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"metricbeat-metricbeat-metrics", "chart":"metricbeat-7.5.0", "heritage":"Tiller", "release":"metricbeat"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
+Error: UPGRADE FAILED: Deployment.apps "metricbeat-kube-state-metrics" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"kube-state-metrics"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && Deployment.apps "metricbeat-metricbeat-metrics" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"metricbeat-metricbeat-metrics", "chart":"metricbeat-7.5.0", "heritage":"Tiller", "release":"metricbeat"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
+```
+
+The workaround is to use `--force` argument for `helm upgrade` command which
+will force Metricbeat resources update through delete/recreate.
+
+
+## 7.0.0-alpha1 - 2019/04/17
+
+### Elasticsearch upgrade from 6.x
+
+If you were using the default Elasticsearch version from the previous release 
+(6.6.2-alpha1) you will first need to upgrade to Elasticsearch 6.7.1 before 
+being able to upgrade to 7.0.0. You can do this by adding this to your values 
+file:
+
+```yaml
+esMajorVersion: 6
+imageTag: 6.7.1
+```
+
+If you are upgrading an existing cluster that did not override the default 
+`storageClassName` you will now need to specify the `storageClassName`. This 
+only affects existing clusters and was changed in [#94][]. The advantage of this
+is that now the Helm chart will just use the default `storageClassName` rather 
+than needing to override it for any providers where it is not called `standard`.
+
+```
+volumeClaimTemplate:
+  storageClassName: "standard"
+```
+
+
+[#94]: https://github.com/elastic/helm-charts/pull/94
+[#352]: https://github.com/elastic/helm-charts/pull/352
+[#458]: https://github.com/elastic/helm-charts/pull/458
+[#540]: https://github.com/elastic/helm-charts/pull/540
+[kube-state-metrics]: https://github.com/helm/charts/tree/master/stable/kube-state-metrics
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -33,12 +33,6 @@
 
 ### Kibana
 
-**Warning**
-[#540](https://github.com/elastic/helm-charts/pull/540) increase default CPU and memory requests/limits. This may impact the resources (nodes) required in your Kubernetes cluster to deploy Kibana chart.
-
-If you wish to come back to former values, you need to override CPU and Memory requests/limits as well as `NODE_OPTIONS` `extraEnvs` variable when deploying your Helm Chart.
-
-
 | PR                                                     | Author                                   | Title                                                                                  |
 | ------------------------------------------------------ | ---------------------------------------- | -------------------------------------------------------------------------------------- |
 |[#493](https://github.com/elastic/helm-charts/pull/493) | [@jamoflaw](https://github.com/jamoflaw) | Fix Mismatch Between Service Selector and Pod Labels when using Helm Aliases in Kibana |
@@ -239,9 +233,6 @@ If you wish to come back to former values, you need to override CPU and Memory r
 
 ### Metricbeat
 
-**Warning**
-[#352](https://github.com/elastic/helm-charts/pull/352) is introducing a breaking change, please refer to [Metricbeat Breaking Changes](./metricbeat/README.md#breaking-changes) section for users upgrading from a chart version < 7.5.0.
-
 | PR                                                     | Author                                           | Title                                                                                     |
 | ------------------------------------------------------ | ------------------------------------------------ | ----------------------------------------------------------------------------------------- |
 |[#352](https://github.com/elastic/helm-charts/pull/352) | [@masterkain](https://github.com/masterkain)     | Bump kube-state-metrics to latest chart and app version                                   |
@@ -507,19 +498,3 @@ If you wish to come back to former values, you need to override CPU and Memory r
 ### Elasticsearch
 
 * [#94](https://github.com/elastic/helm-charts/pull/94) - @kimxogus - Remove hardcoded storageClassName
-
-### Notes
-
-If you were using the default Elasticsearch version from the previous release (6.6.2-alpha1) you will first need to upgrade to Elasticsearch 6.7.1 before being able to upgrade to 7.0.0. You can do this by adding this to your values file:
-
-```
-esMajorVersion: 6
-imageTag: 6.7.1
-```
-
-If you are upgrading an existing cluster that did not override the default `storageClassName` you will now need to specify the `storageClassName`. This only affects existing clusters and was changed in https://github.com/elastic/helm-charts/pull/94. The advantage of this is that now the helm chart will just use the default storageClassName rather than needing to override it for any providers where it is not called `standard`.
-
-```
-volumeClaimTemplate:
-  storageClassName: "standard"
-```
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,21 +1,121 @@
 # Contributing to the Elastic helm charts
 
+## Requirements for submiting a pull request
+
+Before submitting a pull request make sure you validated the following
+requirements:
+
+* CLA should be signed (see [CLA section][] for more details).
+
+* Charts version shouldn't be bumped (see [Releases section][] for more
+details).
+
+* Charts `README.md` should be updated if required (especially updating default
+values if they have been changed).
+
+* Templating tests should be added/updated (see [Templating tests section][] for
+more details).
+
+* Integration tests should be added/updated (see [Integration tests section][]
+for more details).
+
 ## CLA (Contributor License Agreement)
 
-If you haven't already you will need to sign the [CLA](https://www.elastic.co/contributor-agreement) before your pull request can be reviewed and merged.
+If you haven't already you will need to sign the [CLA][] before your pull
+request can be reviewed and merged.
+
+## Branches workflow
+
+Pull request are merged on `master` branch, then they should be backported to
+version branches (example `7.7` branch).
+
+>TODO: add more details about version branching.
+
+## Releases
+
+Just like with the rest of the stack, all versions in this helm chart repo are
+bumped and released at the same time. There is no need to bump the version in
+your pull request.
+
+Charts are released from version branchs (example `7.7` branch).
+
+[Elastic Helm repository][] is updated only during releases.
+
+>TODO: add more details about releases (see also [release.md][]).
 
-## Version bumps
+## Testing
 
-Just like with the rest of the stack, all versions in this helm chart repo are bumped and released at the same time. There is no need to bump the version in your pull request.
+### Templating tests
 
-## Testing and documentation
+Templating tests which can be found in `${CHART}/tests/*.py`
+([Example][templating test example]).
 
-When making any changes be sure to also update the following:
+These charts use [pytest][] to test the templating logic. The dependencies for
+testing can be installed from the [requirements.txt][] in the parent directory:
 
-* Charts README.md
-* The templating tests which can be found in `${CHART}/tests/*.py`. [Example](/elasticsearch/tests/elasticsearch_test.py)
-* The integration tests which can be found in `${CHART}/examples/*/test/goss.yaml`. [Example](/elasticsearch/examples/default/test/goss.yaml)
+```
+pip install -r ./requirements.txt
+```
+
+Tests can then be run from each chart directory using `make pytest`
+
+You can also use `make template` (equivalent to `helm template` ) to look at the
+YAML being generated:
+
+It is possible to run all of the tests and linting inside of a Docker container
+using `make test`
+
+Note that templating tests are formated using [Black][], you should run
+`make lint-python` (equivalent to `black --diff --check .` ) to validate them or
+`black .` to apply formatting before submitting a pull request which will modify
+them.
+
+### Integration tests
+
+Integration tests which can be found in `${CHART}/examples/*/test/goss.yaml`
+([Example][integration test example]).
+
+Integration tests are run using [goss][] which is a [Serverspec][] like tool
+written in golang. See [integration test example][] for an example of what the
+tests look like.
+
+The different integration tests are present in each chart's `examples`
+directory.
+
+Each charts contains an `examples/default` integration test which validate the
+chart deployment with default values.
+
+`examples` directory contains also integration tests for other use cases (for
+example: using `oss` Docker images, using `6.x` version or using `security` ).
+
+Every directory which contains some `test` subdirectory is an integration test
+(`examples` directory contains also some configuration examples for some
+specific scenarios without tests like configuration for specific k8s providers).
+
+To run the goss tests against the default example:
+
+```
+cd examples/default
+make goss
+```
 
 ## Adding new features
 
-If you aren't 100% sure that this is a feature that makes sense for everyone. Please open an issue first to discuss with the maintainers before investing a lot of time into it. 
+If you aren't 100% sure that this is a feature that makes sense for everyone.
+Please open an issue first to discuss with the maintainers before investing a
+lot of time into it.
+
+[black]: https://black.readthedocs.io/en/stable/
+[cla]: https://www.elastic.co/contributor-agreement
+[cla section]: #cla-contributor-license-agreement
+[elastic helm repository]: https://helm.elastic.co
+[goss]: https://github.com/aelsabbahy/goss/blob/master/docs/manual.md
+[integration test example]: https://github.com/elastic/helm-charts/blob/master/elasticsearch/examples/default/test/goss.yaml
+[integration tests section]: #integration-tests
+[pytest]: https://docs.pytest.org/en/latest/
+[serverspec]: https://serverspec.org
+[templating test example]: https://github.com/elastic/helm-charts/blob/master/elasticsearch/tests/elasticsearch_test.py
+[templating tests section]: #templating-tests
+[release.md]: https://github.com/elastic/helm-charts/blob/master/helpers/release.md
+[releases section]: #releases
+[requirements.txt]: https://github.com/elastic/helm-charts/blob/master/requirements.txt
diff --git a/README.md b/README.md
@@ -2,26 +2,40 @@
 
 [![Build Status](https://img.shields.io/jenkins/s/https/devops-ci.elastic.co/job/elastic+helm-charts+master.svg)](https://devops-ci.elastic.co/job/elastic+helm-charts+master/)
 
-This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
+This functionality is in beta and is subject to change. The design and code is
+less mature than official GA features and is being provided as-is with no
+warranties. Beta features are not subject to the support SLA of official GA
+features.
 
 ## Charts
 
-Please look in the chart directories for the documentation for each chart. These helm charts are designed to be a lightweight way to configure our official docker images. Links to the relevant docker image documentation has also been added below.
+Please look in the chart directories for the documentation for each chart. These
+Helm charts are designed to be a lightweight way to configure our official
+Docker images. Links to the relevant Docker image documentation has also been
+added below.
 
 | Chart                                      | Docker documentation                                                            |
-| ------------------------------------------ | ------------------------------------------------------------------------------- |
+|--------------------------------------------|---------------------------------------------------------------------------------|
+| [APM-Server](./apm-server/README.md)       | https://www.elastic.co/guide/en/apm/server/current/running-on-docker.html       |
 | [Elasticsearch](./elasticsearch/README.md) | https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html     |
+| [Filebeat](./filebeat/README.md)           | https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html   |
 | [Kibana](./kibana/README.md)               | https://www.elastic.co/guide/en/kibana/current/docker.html                      |
 | [Logstash](./logstash/README.md)           | https://www.elastic.co/guide/en/logstash/current/docker.html                    |
-| [Filebeat](./filebeat/README.md)           | https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html   |
 | [Metricbeat](./metricbeat/README.md)       | https://www.elastic.co/guide/en/beats/metricbeat/current/running-on-docker.html |
-| [APM-Server](./apm-server/README.md)       | https://www.elastic.co/guide/en/apm/server/current/running-on-docker.html       |
 
 ## Kubernetes Versions
 
-The charts are [currently tested](https://devops-ci.elastic.co/job/elastic+helm-charts+master/) against all GKE versions available. The exact versions are defined under `KUBERNETES_VERSIONS` in [helpers/matrix.yml](/helpers/matrix.yml)
+The charts are [currently tested][] against all GKE versions available. The
+exact versions are defined under `KUBERNETES_VERSIONS` in
+[helpers/matrix.yml][].
 
 ## Helm versions
 
-While we are checking backward compatibility, the charts are only tested with Helm version mentioned in [helm-tester Dockerfile](helpers/helm-tester/Dockerfile) (currently 2.16.6).
-Note that we don't support [Helm 3](https://v3.helm.sh/) version.
+While we are checking backward compatibility, the charts are only tested with
+Helm version mentioned in [helm-tester Dockerfile][] (currently 2.16.6).
+Note that we don't support [Helm 3][] version.
+
+[currently tested]: https://devops-ci.elastic.co/job/elastic+helm-charts+master/
+[helm 3]: https://v3.helm.sh
+[helm-tester Dockerfile]: https://github.com/elastic/helm-charts/blob/master/helpers/helm-tester/Dockerfile
+[helpers/matrix.yml]: https://github.com/elastic/helm-charts/blob/master/helpers/matrix.yml