Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide the ability to blacklist_values against field names using a field naming pattern #23

Open
geekpete opened this issue Feb 21, 2018 · 0 comments

Comments

@geekpete
Copy link

Provide a way to blacklist field values from unknown fields, aka from any/all/pattern matched field names without needing to explicitly list the field names individually, preferably using a regex pattern for the field name.

In this example, prune is used to remove fields with names containing any of a list of unwanted characters.
There's also a prune to remove fields with blacklisted values but blacklist_values requires each field name to be listed.
This doesn't work so well with some logging use cases where fields might be appearing in events that haven't yet been seen before or for ensuring certain items are blacklisted from all fields.

  prune {
    # can also be written as: blacklist_names => ["&","{","}",">",";",","]
    blacklist_names => ["&{}>;,"]
  }
  prune {
    blacklist_values => [ "bad_field_value1","(;|,|&|{|}|>)",
                          "bad_field_value2","(;|,|&|{|}|>)"  ]
  }

but it would be more flexible to be able to do:

  prune {
    blacklist_values => [ "fileprefix1-.*","(;|,|&|{|}|>)",
                          "fieldprefix2-.*",";",
                           ".*", "<"]
  }

which would drop fields:

  • named fileprefix1-* that have a value with any of these chars (;|,|&|{|}|>)
  • named fieldprefix2-* that have a value containing a ; char
  • any field name with a < char

This would be a more complex loop and probably expensive on performance.
But as a starting point a way to blacklist values from any fields might be useful to begin with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants