Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edit upgrade and breaking changes doc #2767

Merged
merged 1 commit into from
Oct 13, 2016
Merged

Edit upgrade and breaking changes doc #2767

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions libbeat/docs/breaking.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[[breaking-changes]]
== Breaking changes
== Breaking Changes

As a general rule, we strive to keep backwards compatibility between minor
versions (e.g. 5.x to 5.y) so you can upgrade without any configuration file
Expand All @@ -10,14 +10,14 @@ We list the breaking changes at the top of the <<release-notes,release notes>>
for each version.

[[breaking-changes-5.0]]
=== Breaking changes in 5.0
=== Breaking Changes in 5.0

This section discusses the changes that you should be aware of if you upgrade
the Beats from version 1.x to 5.x.

==== Configuration file changes
==== Configuration File Changes

Beats 5.0 come with a more powerful and more robust <<config-file-format,file
Beats 5.0 comes with a more powerful and more robust <<config-file-format,file
configuration format>>. While this necessarily causes a few breaking changes, we
provide a migration script that should make upgrading easy. Please see the
<<upgrading>> section.
Expand All @@ -29,7 +29,7 @@ particular Beat are now prefixed by the name of the Beat. For example, the
migration script.

Similarly, common settings available in all Beats are now top level. As part of
this, the options under the `shipper` section are moved to the top level. For
this effort, the options under the `shipper` section are moved to the top level. For
example, the `shipper.name` settings becomes `name`. This change is made
automatically by the migration script.

Expand All @@ -46,12 +46,12 @@ These changes are made automatically by the migration script.
In Filebeat, we refactored the configuration around when to close and when to
start ignoring files. Options like `close_older` and `force_close_older` were
changed or replaced with options from the `close_*` group. See the
{filebeat}/how-filebeat-works.html[How Filebeat works] docs for details. These
{filebeat}/how-filebeat-works.html[How Filebeat Works] docs for details. These
changes are not made automatically by the migration script.

==== Field renames
==== Field Renames

Beats 5.0 also come with several changes in the exported fields. If you are
Beats 5.0 also comes with several changes in the exported fields. If you are
using these fields in Kibana dashboards, you will likely need to adjust the
dashbaords.

Expand All @@ -73,7 +73,7 @@ Changes:

In addition, the unused `count` field was removed from all Packetbeat modules.

==== Installation paths
==== Installation Paths

For 5.0 we have refactored the {filebeat}/directory-layout.html[directory
layout] of the Beats, so they are more consistent and in-line with the best
Expand Down
53 changes: 26 additions & 27 deletions libbeat/docs/upgrading.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

This section gives general recommendations for upgrading the Beats.

=== Upgrading between minor versions
=== Upgrading Between Minor Versions

As a general rule, upgrading between minor versions (e.g. 5.x to 5.y, where x <
y) can be done by simply installing the new release and restarting the Beat
Expand All @@ -25,24 +25,23 @@ upgrading the Elasticsearch mapping templates.
Before upgrading your Beats, please review the <<breaking-changes, breaking
changes>> doc.

In addition to the subsections here, please read the
<<upgrade-mapping-template>>
In addition to the subsections here, please read <<upgrade-mapping-template>>.

==== Configuration file migration
==== Migrating Configuration Files

Beats 5.0 come with several backwards incompatible configuration changes.
Beats 5.0 comes with several backwards incompatible configuration changes.
However, we provide a script that can automatically fixup the most common
settings in your configuration file.
settings in your configuration files.

This script can be found in the Beats download packages. For the `.tar.gz` and
`.zip` archives, it is found under:
You can find this script in the Beats download packages. For the `.tar.gz` and
`.zip` archives, look under:

[source,shell]
------------------------------------------------------------------------------
./scripts/migrate_beat_config_1_x_to_5_0.py
------------------------------------------------------------------------------

For the `.deb` and `.rpm` packages it is found under the
For the `.deb` and `.rpm` packages, look under the
`/usr/share/<beatname>/scripts/` folder, for example:


Expand All @@ -54,28 +53,28 @@ For the `.deb` and `.rpm` packages it is found under the
Python 2.7 or later is required for executing the migration script.

To upgrade your configuration file automatically, simply pass it as an argument
to the migration script, e.g.:
to the migration script, for example:

[source,shell]
------------------------------------------------------------------------------
> ./scripts/migrate_beat_config_1_x_to_5_0.py packetbeat.yml
Backup file created: packetbeat.yml.bak
------------------------------------------------------------------------------

The script will automatically create a backup file and overwrite it with the
upgraded version. You can also use the `--dry` option, which doesn't modify the
input file, but outputs the upgraded version to stdout instead.
The script will automatically create a backup file and overwrite the input file
with the upgraded version. You can also use the `--dry` option, which doesn't
modify the input file, but outputs the upgraded version to stdout instead.

==== Filebeat registry file location and format
==== Filebeat Registry File Location and Format

Filebeat stores the read offset in a registry file that it persist to disk. The
format of this file has changed in version 5.0, but Filebeat automatically
upgrades the format on first run, so there's normally no action required on
Filebeat stores the read offset in a registry file that it persists to disk.
The format of this file has changed in version 5.0, but Filebeat automatically
upgrades the format on the first run, so there's normally no action required on
upgrading.

If you have installed Filebeat from the DEB or RPM packages, the location of the
file stays the same as well (`/var/lib/filebeat/registry`). Also if you've
installed Filebeat as a service on Windows the location stays the same
installed Filebeat as a service on Windows, the location stays the same
(`C:\ProgramData\filebeat\registry`).

If you are running Filebeat from the `tar.gz` archive, however, the location of
Expand All @@ -88,11 +87,11 @@ recommend copying the `.filebeat` file to `data/registry`.

With the Beats 5.0 release, Topbeat is replaced by Metricbeat, which offers more
functionality. More precisely, the Topbeat functionality is roughly equivalent
with the "system" module in Metricbeat.
to the "system" module in Metricbeat.

While there is no automatic way to convert a Topbeat configuration file in the
equivalent Metricbeat configuration file, both configuration are simple enough
that migrating shouldn't take long.
While there is no automatic way to convert a Topbeat configuration file to the
equivalent of a Metricbeat configuration file, both configurations are simple
enough that migrating shouldn't take long.

For example, this is the input section of the Topbeat configuration file:

Expand Down Expand Up @@ -152,17 +151,17 @@ metricbeat.modules:
processes: ['.*']
------------------------------------------------------------------------------

The rest of the configuration options (outputs, name, tags) stay the same or can
The remaining configuration options (outputs, name, tags) stay the same or can
be upgraded using the migration script.

[[upgrade-mapping-template]]
=== Upgrade the Elasticsearch mapping template
=== Upgrading the Elasticsearch Mapping Template

By default, the Elasticsearch mapping template which comes with the Beats is not
By default, the Elasticsearch mapping template that comes with the Beats is not
re-installed if it already exists. This means that if the new Beats version
adds new fields (e.g. a new module in Metricbeat), they won't be reflected in
the Elasticsearch mapping, which can result in visualization problems or
incorect data.
incorrect data.

You can set the `output.elasticsearch.template.overwrite` option to `true` to
make the Beats install the new mapping template even if a template with the same
Expand Down Expand Up @@ -197,7 +196,7 @@ DELETE /packetbeat-2016.09.20
------------------------------------------------------------------------------

Note that the reindex API command can take a long time, depending on the size of
the index. It is recommended that the Beat is stopped during this time, so the
the index. It is recommended that you stop the Beat during this time, so the
order of operations should be:

1. Stop the old version of the Beat
Expand Down