Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] add multi_fields in aws cloudtrail fileset #19021

Closed

Conversation

leehinman
Copy link
Contributor

A keyword representation of the following fields was stored. Adding
text multi_fields so it can be searched more easily.

  • request_parameters
  • response_elements
  • additional_eventdata
  • service_event_details

What does this PR do?

A string representation of the following fields was stored and indexed
as keyword. Adding a text multi_fields so it can be searched more easily.

  • request_parameters
  • response_elements
  • additional_eventdata
  • service_event_details

Why is it important?

Adding multi_fields makes searching easier and avoids mapping explosion

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=aws TESTING_FILEBEAT_FILESETS=cloudtrail mage -v pythonIntegTest

Related issues

A keyword representation of the following fields was stored.  Adding
text multi_fields so it can be searched more easily.

- request_parameters
- response_elements
- additional_eventdata
- service_event_details

Closes elastic#18866
@leehinman leehinman added enhancement Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM labels Jun 5, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 5, 2020
andrewkroh
andrewkroh previously approved these changes Jun 10, 2020
@@ -42,6 +42,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
* iptables {pull}18756[18756]
* Checkpoint {pull}18754[18754]
- Preserve case of http.request.method. ECS prior to 1.6 specified normalizing to lowercase, which lost information. Affects filesets: apache/access, elasticsearch/audit, iis/access, iis/error, nginx/access, nginx/ingress_controller, aws/elb, suricata/eve, zeek/http. {issue}18154[18154] {pull}18359[18359]
- In aws cloudtrail fileset add multi_fields for request_parameters, response_elements, additional_eventdata & service_event_details. {issue}18866[18866] {pull}XXX[XXX]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- In aws cloudtrail fileset add multi_fields for request_parameters, response_elements, additional_eventdata & service_event_details. {issue}18866[18866] {pull}XXX[XXX]
- In aws cloudtrail fileset add multi_fields for request_parameters, response_elements, additional_eventdata & service_event_details. {issue}18866[18866] {pull}19021[19021]

- name: response_elements
type: keyword
description: >-
The response element for actions that make changes (create,
update, or delete actions).
multi_fields:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with the proposed changes in #19020? It seems like they conflict because response_elements will no longer be string so it cannot be indexed at text.

@andrewkroh andrewkroh self-requested a review June 10, 2020 19:32
@andrewkroh andrewkroh dismissed their stale review June 10, 2020 19:33

Pending discussion of #19020 + this change.

@leehinman leehinman closed this Jun 11, 2020
@leehinman leehinman deleted the 18866_cloudtrail_multifield branch October 5, 2020 19:13
@andrewkroh andrewkroh removed the needs_backport PR is waiting to be backported to other branches. label Dec 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[filebeat][aws][cloudtrail] accessKeyId should be searchable
3 participants