Cherry-pick #20994 to 7.x: [Elastic Agent] Add support for EQL based conditions

[blakerouse]

Cherry-pick of PR #20994 to 7.x branch. Original message:

What does this PR do?

This adds the ability for a condition to be defined any where in the inputs configuration to make that dictionary conditional on the resulting EQL evaluation. If the evaluation is false the dictionary is removed from the parent type and if its true the dictionary remains.

This implements EQL with the same variable syntax used in input variable substitution ${ .. }. The following is implemented for EQL.

• Full PEMDAS math support for + - * / %.
• Compares < <= >= > == !=
• Booleans true false
• and and or
• Array functions arrayContains
• Dict functions hasKey (not in EQL)
• Length functions length
• Math functions add, subtract, multiply, divide, modulo.
• String functions concat, endsWith, indexOf, match, number, startsWith, string, stringContains.

Why is it important?

To support condition enablement on inputs or even any part of the input configuration. The same conditions can be applied to processors or streams or anything inside of the inputs configuration.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Add a condition to your inputs and see that they are not rendered when the condition fails.

Condition on input

inputs:
  - type: logfile
    streams:
      - paths:
         - /var/log/syslog
    condition: ${host.platform} == 'linux'

Condition on stream

inputs:
  - type: system/metrics
    streams:
      - metricset: load
        data_stream.dataset: system.cpu
        condition: ${host.platform} != 'windows'

Condition on processor

inputs:
  - type: system/metrics
    streams:
      - metricset: load
        data_stream.dataset: system.cpu
        condition: ${host.platform} != 'windows'
    processors:
      - add_fields:
          fields:
            platform: ${host.platform}
          to: host
        condition: ${host.platform} != 'windows'

Related issues

• Closes [Elastic Agent] Support EQL condition on an input #20784

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[ph]

backport lgmt, waiting for green.

Looking at this PR, we should have a changelog entry.

[blakerouse]

@ph Added changelog.

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #21039 updated]

• Start Time: 2020-09-09T15:10:04.323+0000

• Duration: 78 min 49 sec

Test stats 🧪

Test	Results
Failed	0
Passed	20142
Skipped	1863
Total	22005

[ph]

Waiting on green.