Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry-pick #20308 to 7.x: [Filebeat][Gsuite] Transform all dates to timestamp with processor #20318

Merged
merged 1 commit into from
Jul 29, 2020
Merged

Cherry-pick #20308 to 7.x: [Filebeat][Gsuite] Transform all dates to timestamp with processor #20318

merged 1 commit into from
Jul 29, 2020

Conversation

marc-gr
Copy link
Contributor

@marc-gr marc-gr commented Jul 29, 2020 •
edited by zube bot
Loading

Cherry-pick of PR #20308 to 7.x branch. Original message:

What does this PR do?

Parse properly date time fields in different formats in gsuite module.

Why is it important?

Not all fields come in the same format, and some of them were not accepted as valid dates. This change proactively parses all date fields with the known incoming formats.

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
    - [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Logs

Logs that did show the bug

"caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2020/07/28 04:59:59 UTC] with format [strict_date_optional_time||epoch_millis]","caused_by":{"type":"date_time_parse_exception","reason":"date_time_parse_exception: Failed to parse with all enclosed parsers"

@marc-gr
[Filebeat][Gsuite] Transform all dates to timestamp with processor (e…
76c5aca 
…lastic#20308)

* Transform all dates to timestamp with processor

* Change parse date function to create a chain

(cherry picked from commit 953d067)
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 29, 2020
@marc-gr marc-gr requested a review from a team July 29, 2020 14:36
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20318 opened]

  • Start Time: 2020-07-29T14:31:22.885+0000

  • Duration: 57 min 37 sec

Test stats 🧪

Test Results
Failed 0
Passed 2434
Skipped 385
Total 2819

@marc-gr marc-gr merged commit c8fade2 into elastic:7.x Jul 29, 2020
@marc-gr marc-gr deleted the backport_20308_7.x branch July 29, 2020 15:30
@zube zube bot removed the [zube]: Done label Oct 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adriansr adriansr adriansr approved these changes

Assignees
No one assigned
Labels
backport
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@marc-gr @elasticmachine @adriansr