expected_values in field definitions

[ebeahan]

Adds support for an expected_values attribute in schema field definitions.

Specific ECS fields include lists of expected values, and expected_values provides a consistent location to capture those values.

Example definition

Fields define an expected_values array. Additionally, the --strict flag now compares the value(s) in example against the expected_values.

---
- name: faas
  fields:
    - name: trigger.type
      level: extended
      type: keyword
      short: The trigger for the function execution.
      expected_values:
        - http
        - pubsub
        - data source
        - timer
        - other
      description: >
        The trigger for the function execution.
     example: http

Example field rendering

The Jinja2 template now consistently renders the expected_values. Unordered lists are now also generated with formatting.

Additional field-details.asciidoc formatting change

Asciidoc supports additional formatting options in a table cell using the a| operator. This change opens up a better formatting experience for ECS field descriptions.

The field-details.j2 template now uses a| for each field description.

Addresses #1951

[djptek]

Hi @ebeahan

Have you looked into using the same structure as allowed_values to implement expected_values ?

The fact that allowed_values has a sub-attribute expected_event_types is unfortunate, however, I would expect expected_values to use the same structure as allowed_values to permit re-use both in ECS tooling and in any potential downstream field validation code.

Assuming inviability of renaming **.allowed_values.expected_event_types, I think it might be an option to have e.g. **.expected_values.expected_event_types echoing the same structure. This is turn might suggest a PR to synchronise previously documented expected values from their related descriptions, so for example file.attributes might look like:

- name: attributes
      level: extended
      type: keyword
      short: Array of file attributes.
      description: >
        Array of file attributes.

        Attributes names will vary by platform. Here's a non-exhaustive list of values
        that are expected in this field: archive, compressed, directory, encrypted,
        execute, hidden, read, readonly, system, write.
      example: '["readonly", "system"]'
      normalize:
        - array
      expected_values:
        -expected_event_types:
            - archive
            - compressed
            - directory
            - encrypted
            - execute
            - hidden
            - read
            - readonly
            - system
            - write

[ebeahan]

Have you looked into using the same structure as allowed_values to implement expected_values ?

I haven't thought of using a list of objects over a list.

allowed_values holds a list of objects with name, description, and expected_event_types keys. In this implementation, expected_keys only holds a list of expected values.

Are there other keys/attributes to set underneath expected_values? If not, I prefer the simplicity of expected_values.

[djptek]

The relationship between allowed_values and expected_values may need care. Prior to commenting on this issue, I did a lot of searching and grep-ing and was considering raising a meta-issue to clarify this before opting to comment here.

I prefer the simplicity of expected_values.

I agree the simplicity of having the values in an array would be aesthetically preferable, but I'm uncomfortable with doing expected_values and allowed_values differently, because that would obstruct reuse in our tools & end-users tools, and allowed_values has already been done this way.

I agree that the description may be redundant for expected_values, however, from a usage standpoint, if a user is writing validation code they would first address allowed_values because these could threaten data integrity. To refactor allowed_values may be a breaking change for users, so if that's not going to change my inclination would be to follow the established pattern.

[djptek]

Discussed offline @ebeahan let's go ahead with the list

[djptek]

LGTM

[djptek]

LGTM

[ebeahan]

💚 All backports created successfully

Status	Branch	Result
✅	8.4

Questions ?

Please refer to the Backport tool documentation