Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'composed_of' support for template creation with setup commands #29549

Open
novaksam opened this issue Dec 20, 2021 · 6 comments
Open

Add 'composed_of' support for template creation with setup commands #29549

novaksam opened this issue Dec 20, 2021 · 6 comments
Labels
Stalled Team:Elastic-Agent Label for the Agent team Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@novaksam
Copy link

Describe the enhancement: Add the ability to attach component templates to the V2 templates create by the various beat setup processes.

Describe a specific use case for the enhancement or feature: I have a lowercasing normalizer I attach to certain fields for winlogbeat, to make searching easier (less case sensitive). I'm working on shifting from legacy to v2 templates, and I'd like to have a way to include certain component templates that I've made to the templates created by the setup process.

I looked at https://github.com/kvch/beats/blob/master/libbeat/template/config.go but there doesn't appear to be support for appending 'composed_of' to templates this way. I've tested a handful of different ways with the existing reference file, but did not have any success, though I apologize if I missed something and this already works.

Ideally it could be something like setup.template.composed_of: ["component1", "component2"]

For reference, this is the normalizer I have configured:

component template

{
  "template": {
    "settings": {
      "index": {
        "analysis": {
          "normalizer": {
            "normalizer_lowercase": {
              "filter": [
                "lowercase",
                "asciifolding"
              ],
              "type": "custom",
              "char_filter": []
            }
          },
          "analyzer": {
            "analyzer_case_insensitive": {
              "filter": "lowercase",
              "tokenizer": "keyword"
            }
          }
        }
      }
    },
    "mappings": {
      "_source": {
        "enabled": true,
        "includes": [],
        "excludes": []
      },
      "_routing": {
        "required": false
      },
      "dynamic": true,
      "numeric_detection": false,
      "date_detection": true,
      "dynamic_date_formats": [
        "strict_date_optional_time",
        "yyyy/MM/dd HH:mm:ss Z||yyyy/MM/dd Z"
      ],
      "dynamic_templates": [],
      "properties": {
        "process": {
          "type": "object",
          "properties": {
            "args": {
              "type": "keyword",
              "normalizer": "normalizer_lowercase",
              "ignore_above": 1024
            },
            "executable": {
              "type": "keyword",
              "normalizer": "normalizer_lowercase",
              "ignore_above": 1024
            },
            "name": {
              "type": "keyword",
              "normalizer": "normalizer_lowercase",
              "ignore_above": 1024
            },
            "parent": {
              "type": "object",
              "properties": {
                "args": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "executable": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "name": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                }
              }
            }
          }
        },
        "winlog": {
          "type": "object",
          "properties": {
            "event_data": {
              "type": "object",
              "properties": {
                "CommandLine": {
                  "type": "keyword",
                  "index": true,
                  "normalizer": "normalizer_lowercase",
                  "eager_global_ordinals": false,
                  "norms": false,
                  "split_queries_on_whitespace": false,
                  "doc_values": true,
                  "store": false,
                  "index_options": "docs"
                },
                "NewProcessName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "ObjectDN": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "ObjectName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "ParentProcessName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "ProcessCommandLine": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 8191
                },
                "ProcessName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "RelativeTargetName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "ScriptBlockText": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "ShareLocalPath": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "SubjectDomainName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "SubjectUserDomainName": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                },
                "TargetObject": {
                  "type": "keyword",
                  "normalizer": "normalizer_lowercase",
                  "ignore_above": 1024
                }
              }
            }
          }
        }
      }
    }
  }
}

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 20, 2021
@ChrsMark ChrsMark added the Team:Elastic-Agent Label for the Agent team label Jan 10, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 10, 2022
@kvch kvch added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jan 10, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic
Copy link

botelastic bot commented Jul 7, 2023

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jul 7, 2023
@botelastic botelastic bot closed this as completed Jan 3, 2024
@lucabelluccini
Copy link
Contributor

Hello @pierrehilbert
Is it normal this issue is marked as completed?

@pierrehilbert
Copy link
Collaborator

No just has been stalled for too long.

@pierrehilbert pierrehilbert reopened this Jan 5, 2024
@botelastic botelastic bot removed the Stalled label Jan 5, 2024
@botelastic
Copy link

botelastic bot commented Jan 4, 2025

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Stalled Team:Elastic-Agent Label for the Agent team Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

No branches or pull requests

7 participants