[Filebeat][New Fileset] Cisco AMP #22768

P1llus · 2020-11-26T05:02:15Z

What does this PR do?

This PR is a WIP for adding Cisco AMP fileset to the cisco module.

Cisco AMP allows collection of endpoint events from their Events API documented here

Why is it important?

Adds support for another very commonly used Cisco product.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Requires [filebeat][httpjson] Initial implementation of httpjson v2 #22320

P1llus · 2020-11-26T05:03:12Z

The initial PR is created as a draft for easy sharing, its current state is that it is waiting for #22320 and is still missing core components like documentation and a finished ingest pipeline. Should finalize this right after the above PR has been merged to master.

elasticmachine · 2020-11-26T05:54:32Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #22768 updated
Start Time: 2021-02-15T08:59:21.499+0000
Duration: 45 min 21 sec
Commit: ee44ea3

Test stats 🧪

Test	Results
Failed	0
Passed	13011
Skipped	2033
Total	15044

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	13011
Skipped	2033
Total	15044

P1llus · 2021-01-04T14:35:20Z

Cisco AMP returns the newest event first instead of last, this would require the possibility to get timestamp of first instead of last event added through the httpjson cursor functionality.

P1llus · 2021-01-15T18:47:08Z

First MVP is finalized and is now ready for review

elasticmachine · 2021-01-15T18:47:12Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

andrewstucki

initial pass

x-pack/filebeat/module/cisco/amp/test/cisco_amp2.ndjson.log-expected.json

x-pack/filebeat/module/cisco/amp/test/cisco_amp.ndjson.log-expected.json

x-pack/filebeat/module/cisco/amp/ingest/pipeline.yml

filebeat/docs/modules/cisco.asciidoc

x-pack/filebeat/module/cisco/amp/_meta/fields.yml

x-pack/filebeat/module/cisco/amp/config/config.yml

x-pack/filebeat/module/cisco/amp/ingest/pipeline.yml

P1llus · 2021-01-29T14:20:33Z

The above commit 5a0e1e4 fixes most of the comments made to the PR. Anything not marked resolved are still being worked on.

P1llus · 2021-02-03T10:41:14Z

@andrewstucki I added in your feedback to the pipeline, a few small things to note:

I can only detect windows based on filepath, its not possible to detect the difference between OSX and Linux, I don't know if you have any ideas or if setting platform to windows is enough to fit most usecases?
I added in uri_parts processor, and added a fallback, so that we can continue the discussion around the backwards compatibility in the other issue.

filebeat/docs/modules/cisco.asciidoc

x-pack/filebeat/module/cisco/amp/config/config.yml

x-pack/filebeat/module/cisco/amp/ingest/pipeline.yml

adriansr

LGTM

* initial stash for cisco amp * stashing more cisco amp changes * stashing httpjson v2 type config and merging with master * stashing amp changes * updating fields.go * first finalized MVP for Cisco AMP * added changelog * This commit resolves most of the comments made to the PR * updating pipeline based on PR feedback and comments * updating field mapping and docs based on PR comments (cherry picked from commit 183fae5)

* [Filebeat][New Fileset] Cisco AMP (#22768) * initial stash for cisco amp * stashing more cisco amp changes * stashing httpjson v2 type config and merging with master * stashing amp changes * updating fields.go * first finalized MVP for Cisco AMP * added changelog * This commit resolves most of the comments made to the PR * updating pipeline based on PR feedback and comments * updating field mapping and docs based on PR comments (cherry picked from commit 183fae5) * Removing changelog entries from other PR's

P1llus added 2 commits November 11, 2020 22:14

initial stash for cisco amp

8859cb7

stashing more cisco amp changes

60fd32a

P1llus added enhancement in progress Pull request is currently in progress. Filebeat Filebeat Team:Security-External Integrations labels Nov 26, 2020

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 26, 2020

P1llus and others added 5 commits December 8, 2020 04:14

Merge branch 'master' into filebeat_cisco_amp

b88ec41

stashing httpjson v2 type config and merging with master

d1d38bf

stashing amp changes

a3ac306

updating fields.go

74e8746

Merging master

157f9e2

jamiehynds added Theme: just_ingest_it 7.12 candidate labels Jan 7, 2021

P1llus added 3 commits January 15, 2021 13:50

Merge branch 'master' into filebeat_cisco_amp

bbb9df1

first finalized MVP for Cisco AMP

d7d96f3

added changelog

77a0062

P1llus marked this pull request as ready for review January 15, 2021 18:47

Merge branch 'master' into filebeat_cisco_amp

413259a

P1llus added review needs_reviewer PR needs to be assigned a reviewer and removed in progress Pull request is currently in progress. review labels Jan 15, 2021

andrewstucki reviewed Jan 15, 2021

View reviewed changes

x-pack/filebeat/module/cisco/amp/ingest/pipeline.yml Outdated Show resolved Hide resolved

marc-gr reviewed Jan 18, 2021

View reviewed changes

This commit resolves most of the comments made to the PR

5a0e1e4

P1llus mentioned this pull request Jan 29, 2021

[Filebeat] Check if processor is supported by ES version #23763

Merged

5 tasks

updating pipeline based on PR feedback and comments

553fa32

andrewkroh reviewed Feb 12, 2021

View reviewed changes

updating field mapping and docs based on PR comments

ee44ea3

adriansr approved these changes Feb 15, 2021

View reviewed changes

P1llus merged commit 183fae5 into elastic:master Feb 15, 2021

P1llus mentioned this pull request Feb 15, 2021

Cherry-pick #22768 to 7.x: [Filebeat][New Fileset] Cisco AMP #24038

Merged

6 tasks

P1llus added the v7.12.0 label Feb 15, 2021

P1llus mentioned this pull request Feb 15, 2021

Cherry-pick #23763 to 7.x: [Filebeat] Check if processor is supported by ES version #24044

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Filebeat][New Fileset] Cisco AMP #22768

[Filebeat][New Fileset] Cisco AMP #22768

P1llus commented Nov 26, 2020 •

edited

Loading

P1llus commented Nov 26, 2020

elasticmachine commented Nov 26, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

P1llus commented Jan 4, 2021

P1llus commented Jan 15, 2021

elasticmachine commented Jan 15, 2021

andrewstucki left a comment

P1llus commented Jan 29, 2021

P1llus commented Feb 3, 2021

adriansr left a comment

[Filebeat][New Fileset] Cisco AMP #22768

[Filebeat][New Fileset] Cisco AMP #22768

Conversation

P1llus commented Nov 26, 2020 • edited Loading

What does this PR do?

Why is it important?

Checklist

Related issues

P1llus commented Nov 26, 2020

elasticmachine commented Nov 26, 2020 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

P1llus commented Jan 4, 2021

P1llus commented Jan 15, 2021

elasticmachine commented Jan 15, 2021

andrewstucki left a comment

Choose a reason for hiding this comment

P1llus commented Jan 29, 2021

P1llus commented Feb 3, 2021

adriansr left a comment

Choose a reason for hiding this comment

P1llus commented Nov 26, 2020 •

edited

Loading

elasticmachine commented Nov 26, 2020 •

edited by jenkins-beats-ci bot

Loading