Skip to content

Commit

Permalink
[docs] Edit docs about field mappings (#17740)
Browse files Browse the repository at this point in the history
  • Loading branch information
dedemorton authored Apr 22, 2020
1 parent 62b1e90 commit 0c1d299
Showing 1 changed file with 33 additions and 14 deletions.
47 changes: 33 additions & 14 deletions docs/devguide/fields-yml.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
[[event-fields-yml]]
=== Defining field mappings

Fields used by your Beat, along with their mapping details, must be defined in `_meta/fields.yml`. After editing this file, you must re-run `make update`.
You must define the fields used by your Beat, along with their mapping details,
in `_meta/fields.yml`. After editing this file, run `make update`.

Define the field mappings in the `fields` array:

Expand Down Expand Up @@ -29,27 +30,36 @@ Define the field mappings in the `fields` array:
----------------------------------------------------------------------

<1> `name`: The field name
<2> `type`: The field type. The value for the `type` key can be any of the datatypes https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html[available in Elasticsearch]. If no value is specified, the field will default to being a `keyword`.
<2> `type`: The field type. The value of `type` can be any datatype {ref}/mapping-types.html[available in {es}]. If no value is specified, the default type is `keyword`.
<3> `required`: Whether or not a field value is required
<4> `description`: Some information about the field contents

==== Mapping parameters
Other mapping parameters can be specified for each field. Consult the https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-params.html[Elasticsearch reference] for more details on each of these parameters.

You can specify other mapping parameters for each field. See the
{ref}/mapping-params.html[{es} Reference] for more details about each
parameter.

[horizontal]
`format`:: Specify a custom date format used by the field.
`multi_fields`:: For `text` or `keyword` fields, `multi_fields` can be used to define multi-field mappings.
`multi_fields`:: For `text` or `keyword` fields, use `multi_fields` to define
multi-field mappings.
`enabled`:: Whether or not the field is enabled.
`analyzer`:: Which analyzer to use when indexing.
`search_analyzer`:: Which analyzer to use when searching.
`norms`:: Applies to `text` and `keyword` fields. Default is `false`.
`dynamic`:: Dynamic field control. Can be one of `true` (default), `false` or `strict`.
`dynamic`:: Dynamic field control. Can be one of `true` (default), `false`, or
`strict`.
`index`:: Whether or not the field should be indexed.
`doc_values`:: Whether or not the field should have doc values generated. See docs.
`doc_values`:: Whether or not the field should have doc values generated.
`copy_to`:: Which field to copy the field value into.
`ignore_above`:: When this property value is missing or is `0`, it receives the `libbeat` default value of `1024`. If the value is `-1`, the Elasticsearch default value will be applied. Any other specified value will be applied as-is.
`ignore_above`:: {es} ignores (does not index) strings that are longer than the
specified value. When this property value is missing or `0`, the `libbeat`
default value of `1024` characters is used. If the value is `-1`, the {es}
default value is used.

For example, we could use the `copy_to` mapping parameter to copy the `last_name` and `first_name` fields into the `full_name` field at index time:
For example, you can use the `copy_to` mapping parameter to copy the
`last_name` and `first_name` fields into the `full_name` field at index time:

[source,yaml]
----------------------------------------------------------------------
Expand Down Expand Up @@ -78,12 +88,20 @@ For example, we could use the `copy_to` mapping parameter to copy the `last_name
<1> Copy the value of `last_name` into `full_name`
<2> Copy the value of `first_name` into `full_name`

There are also some Kibana-specific properties, not detailed here. These are: `analyzed`, `count`, `searchable`, `aggregatable`, `script`. Kibana parameters can also be described using the `pattern`, `input_format`, `output_format`, `output_precision`, `label_template`, `url_template` and `open_link_in_current_tab`.
There are also some {kib}-specific properties, not detailed here. These are:
`analyzed`, `count`, `searchable`, `aggregatable`, and `script`. {kib}
parameters can also be described using `pattern`, `input_format`,
`output_format`, `output_precision`, `label_template`, `url_template`, and
`open_link_in_current_tab`.

==== Defining text multi-fields
There are various options that can be applied when using text fields. A simple text field using the default analyzer can be defined without any other options, as in the example above.

To keep the original keyword value when using `text` mappings, for instance to use in aggregations or ordering, a multi-field mapping can be used:
There are various options that you can apply when using text fields. You can
define a simple text field using the default analyzer without any other options,
as in the example shown earlier.

To keep the original keyword value when using `text` mappings, for instance to
use in aggregations or ordering, you can use a multi-field mapping:

[source,yaml]
----------------------------------------------------------------------
Expand All @@ -97,8 +115,9 @@ To keep the original keyword value when using `text` mappings, for instance to u
- name: keyword <2>
type: keyword <3>
----------------------------------------------------------------------
<1> `multi_fields`: Define the `multi_fields` mapping parameter
<1> `multi_fields`: Define the `multi_fields` mapping parameter.
<2> `name`: This is a conventional name for a multi-field. It can be anything (`raw` is another common option) but the convention is to use `keyword`.
<3> `type`: Specify the `keyword` type so we can use the field in aggregations or to order documents.
<3> `type`: Specify the `keyword` type to use the field in aggregations or to order documents.

Consult the https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html[reference] for more information on multi-fields.
For more information, see the {ref}/multi-fields.html[{es} documentation about
multi-fields].

0 comments on commit 0c1d299

Please sign in to comment.