Change fleet-server to use changelog tool (elastic#2499)

* Change fleet-server to use changelog tool

Create changelog fragments and log for 8.7.0 release and remove the old
changelog. Use changelog tool for all new changes.

* Apply suggestions from code review

Co-authored-by: Kyle Pollich <[email protected]>

---------

Co-authored-by: Kyle Pollich <[email protected]>

.github/PULL_REQUEST_TEMPLATE.md

@@ -31,7 +31,7 @@ This checklist is a reminder about high level design problems that should be con
 
 - [ ] I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
 - [ ] I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
-- [ ] I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc. 
+- [ ] I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.
 
 ## Checklist
 
@@ -43,7 +43,7 @@ This checklist is to help creators of PRs to find parts which might not be direc
 - [ ] I have made corresponding changes to the documentation
 - [ ] I have made corresponding change to the default configuration files
 - [ ] I have added tests that prove my fix is effective or that my feature works
-- [ ] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.
+- [ ] I have added an entry in `./changelog/fragments` using the [changelog tool](https://github.com/elastic/fleet-server#changelog)
 
 
 ## Related issues

README.md

@@ -29,6 +29,22 @@ The `golang-crossbuild:1.16.X-darwin-debian10` images expects the minimum MacOSX
 
 The following are notes to help developers onboarding to the project to quickly get running. These notes might change at any time.
 
+### Changelog
+
+The changelog for fleet-server is generated and maintained using the [elastic-agent-changelog-tool](https://github.com/elastic/elastic-agent-changelog-tool).
+Read the [installation](https://github.com/elastic/elastic-agent-changelog-tool/blob/main/docs/install.md) and [usage](https://github.com/elastic/elastic-agent-changelog-tool/blob/main/docs/usage.md#im-a-developer) instructions to get started.
+
+The changelog tool produces fragment files that are consolidated to generate a changelog for each release
+Each PR containing a change with user impact (new feature, bug fix, etc.) must contain a changelog fragment describing the change.
+
+A simple example of a changelog fragment is below for reference:
+```yaml
+kind: feature
+summary: Accept raw errors as a fallback to detailed error type
+pr: https://github.com/elastic/fleet-server/pull/2079
+issue: https://github.com/elastic/elastic-agent/issues/931
+```
+
 ### Development build
 
 To compile the fleet-server in development mode set the env var `DEV=true`.
@@ -190,7 +206,7 @@ cp build/binaries/fleet-server-8.7.0-SNAPSHOT-linux-x86_64/fleet-server ./data/e
 
 When developing new features as you write code you would want to make sure your changes are not breaking any pre-existing
 functionality. For this reason as you make changes you might want to run a subset of tests or the full tests before
-you create a pull request. 
+you create a pull request.
 
 #### Running go tests
 
@@ -206,13 +222,13 @@ This make target will execute the go unit tests and should normally pass without
 It's a good practice before you start your changes to establish the current baseline of the benchmarks in your machine.
 To establish the baseline benchmark report you can follow the following workflow
 
-__Establish a baseline__ 
+__Establish a baseline__
 ```bash
 BENCH_BASE=base.out make benchmark
 ```
 
-This will execute all the go benchmark test and write the output into the file build/base.out. If you omit the 
-`BENCH_BASE` variable it will automatically select the name `build/benchmark-{git_head_sha1}.out`. 
+This will execute all the go benchmark test and write the output into the file build/base.out. If you omit the
+`BENCH_BASE` variable it will automatically select the name `build/benchmark-{git_head_sha1}.out`.
 
 __Re-running benchmark after changes__
 
@@ -232,8 +248,8 @@ And this will print the difference between the baseline and next results.
 
 You can read more on the [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat) official site.
 
-There are some additional parameters that you can use with the `benchmark` target. 
+There are some additional parameters that you can use with the `benchmark` target.
 - `BENCHMARK_FILTER`: you can define the test filter so that you only run a subset of tests (Default: Bench, only run
 the test BenchmarkXXXX and not unit tests)
 - `BENCHMARK_COUNT`: you can define the number of iterations go test will run. Having larger number helps
-remove run-to-run variations (Default: 8)
+remove run-to-run variations (Default: 8)

changelog/8.7.0.yaml

@@ -0,0 +1,84 @@
+version: 8.7.0
+entries:
+    - kind: feature
+      summary: Accept raw errors as a fallback to detailed error type
+      description: ""
+      component: ""
+      pr:
+        - "2079"
+      issue:
+        - https://github.com/elastic/elastic-agent/issues/931
+      timestamp: 1668434027
+      file:
+        name: 1668434027-Accept-raw-errors-as-a-fallback-to-detailed-error-type.yaml
+        checksum: 9409686f6ea1d4edc1291531f9e4bfb8b7a0b554
+    - kind: bug-fix
+      summary: filter out UPDATE_TAGS and FORCE_UNENROLL actions on agent checkin
+      description: ""
+      component: ""
+      pr:
+        - "2200"
+      issue:
+        - "2199"
+      timestamp: 1671566400
+      file:
+        name: 1671566400-filter-out-UPDATE_TAGS-and-FORCE_UNENROLL.yaml
+        checksum: d1fb905d44cb56930188a7916cc463363cf1c644
+    - kind: breaking-change
+      summary: Remove automatic agent unenrollment
+      description: ""
+      component: ""
+      pr:
+        - "2096"
+      issue:
+        - "2063"
+      timestamp: 1672689600
+      file:
+        name: 1672689600-remove-automatic-agent-unenrollment.yaml
+        checksum: 39f5d89bc387cca29a2696b3369d7bda02591239
+    - kind: feature
+      summary: Replaced upgrade expiration and minimum_execution_duration with rollout_duration_seconds
+      description: ""
+      component: ""
+      pr:
+        - "2243"
+      issue: []
+      timestamp: 1673467200
+      file:
+        name: 1673467200-replaced-upgrade-expiration-and-minimum_execution_time.yaml
+        checksum: 115fcdac8c8c128bff36489c6e335e872461ad39
+    - kind: feature
+      summary: Document how to run the fleet server locally.
+      description: ""
+      component: ""
+      pr:
+        - "2212"
+      issue:
+        - "1423"
+      timestamp: 1673553600
+      file:
+        name: 1673553600-document-how-to-run-fleet-server-locally.yaml
+        checksum: f702299f22a08d353b2cce789012de5338dc983e
+    - kind: feature
+      summary: Fleet server now supports file uploads for a limited subset of integrations.
+      description: ""
+      component: ""
+      pr:
+        - "1902"
+      issue: []
+      timestamp: 1674244800
+      file:
+        name: 1674244800-fleet-server-file-upload.yaml
+        checksum: e6c3fef52a872e46e8cb0e2a169928a23891f6cf
+    - kind: bug-fix
+      summary: Fix fleet-server discarding duplicate `server` keys input when creating config from policy.
+      description: ""
+      component: ""
+      pr:
+        - "2354"
+      issue:
+        - "2303"
+      timestamp: 1677009600
+      file:
+        name: 1677009600-fix-fleet-server-discard-keys.yaml
+        checksum: 3a6f42e0edf9d9226c2971e6a087792d85f3e7c9

changelog/fragments/1671566400-filter-out-UPDATE_TAGS-and-FORCE_UNENROLL.yaml

@@ -0,0 +1,31 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: bug-fix
+
+# Change summary; a 80ish characters long description of the change.
+summary: filter out UPDATE_TAGS and FORCE_UNENROLL actions on agent checkin
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component:
+
+# PR number; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: 2200
+
+# Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: 2199

changelog/fragments/1672689600-remove-automatic-agent-unenrollment.yaml

@@ -0,0 +1,31 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: breaking-change
+
+# Change summary; a 80ish characters long description of the change.
+summary: Remove automatic agent unenrollment
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component:
+
+# PR number; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: 2096
+
+# Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: 2063

changelog/fragments/1673467200-replaced-upgrade-expiration-and-minimum_execution_time.yaml

@@ -0,0 +1,31 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Replaced upgrade expiration and minimum_execution_duration with rollout_duration_seconds
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component:
+
+# PR number; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: 2243
+
+# Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+#issue:

changelog/fragments/1673553600-document-how-to-run-fleet-server-locally.yaml

@@ -0,0 +1,31 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Document how to run the fleet server locally.
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component:
+
+# PR number; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: 2212
+
+# Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: 1423

changelog/fragments/1674244800-fleet-server-file-upload.yaml

@@ -0,0 +1,31 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Fleet server now supports file uploads for a limited subset of integrations.
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component:
+
+# PR number; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: 1902
+
+# Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+#issue: