Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup list fields #153

Merged
merged 7 commits into from
Dec 22, 2021
Merged

Cleanup list fields #153

merged 7 commits into from
Dec 22, 2021

Conversation

mstemm
Copy link
Contributor

@mstemm mstemm commented Dec 8, 2021

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area driver-kmod

/area driver-ebpf

/area libscap

/area libsinsp

/area tests

/area proposals

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Add built-in support for pretty-printing fields that can be used across programs.

@poiana
Copy link
Contributor

poiana commented Dec 8, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mstemm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we have to keep the markdown formatting, otherwise SGTM

@mstemm mstemm force-pushed the cleanup-list-fields branch from 9a085e2 to 345aa4e Compare December 9, 2021 16:58
@poiana poiana added size/XL and removed size/XXL labels Dec 9, 2021
@mstemm
Copy link
Contributor Author

mstemm commented Dec 9, 2021

Sorry I didn't know markdown output was actually used. I added it back. PTAL again!

@LucaGuerra
Copy link
Contributor

Tested this change and terminal+markdown formatting seems to be right. Just a little thing: I noticed that evt (All event types) appears at the bottom of the list, would it be possible to make it appear on top? This would make the documentation easier to consume when piped to a pager or exported to the documentation website.

Copy link
Contributor

@FedeDP FedeDP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just a couple of questions, otherwise this looks fine!

Previously, as plugins were loaded, each plugin would create two
filtercheck objects:

 - A sinsp_filter_check_plugininfo object that defined the fields
   evt.pluginname/evt.plugininfo, with an implementation that only
   returned values for events coming from that specific plugin.
 - A sinsp_filter_check_plugin object that defined the fields exported
   by the plugin.

This made interpreting the output of sysdig -l/falco --list confusing,
as you would get multiple sections in the output for the
almost-duplicate filterchecks.

Fix this by moving evt.plugininfo/evt.pluginname into
sinsp_filter_check_gen_event and looking up the plugin object on the
fly. This avoids the almost-duplicate filtercheck objects, which makes
it easier to print a clean list of fields, as these fields are only
defined in one filtercheck object.

Also, *only* create a sinsp_filter_check_gen_event object for source
plugins. There isn't any need to create one for extractor plugins,
which don't work directly with event meta-data like event number,
timestamp, etc.

Signed-off-by: Mark Stemm <[email protected]>
Add additional info to filter_field_info:

- shortdesc: a small (3-5 word) description of the field class,
  suitable for printing on the same line as the field class without
  line wrapping.
- data_type: the data type the filtercheck field works
  with (e.g. CHARBUF, UINT64, IPADDR, etc).
- tags: a set of free-form tags for the field. Examples include
  "FILTER ONLY", IDX_REQUIRED, etc.

This will make it easier to print rich information about each field.

Also fill in shortdesc for selected filtercheck objects to help
provide context when printing fields.

Signed-off-by: Mark Stemm <[email protected]>
When plugins are loaded for programs that don't use event types, you
might end up with duplicate sinsp_filter_check_gen_event objects being
added to the g_filterlist filter_check_list.

This isn't harmful, but it results in duplicate entries when listing
the full set of filter fields.

Handle this by checking to see if there's already a filter_check
object with the same name (e.g. "evt") and shortdesc (e.g. "All event
types"). If there's a duplicate, simply delete the to-be-added object.

Signed-off-by: Mark Stemm <[email protected]>
Add a convienence method
sinsp_filter_factory::check_infos_to_fieldclass_infos() that converts
from a vector<filter_check_info> (e.g. filterchecks specifically for
sinsp events) to a list<filter_fieldclass_info> (e.g. description of
fields for all event types).

This makes it easier to have a single method to print info about
filtercheck fields.

Signed-off-by: Mark Stemm <[email protected]>
Add methods to filter_fieldclass_info to return a pretty-printed
representation of the field class, as a string. The output looks like
the following:

-------------------------------
Field Class:                  evt (All event types)
Description:                  These fields can be used for all event types
Event Sources:                aws_cloudtrail syscall

evt.num                       (Type: UINT64) event number.
...

Internally, it uses stringstreams to tokenize long strings like the
description, etc, and wrap them as needed. It also cleans up the
output so words aren't broken across lines.

It has options to print verbose information (adding Types for each
field), and if a set of event sources was provided, those are printed
as well.

This will allow the --list outputs of falco, as well as other programs
that use the libs, to have a single representation of fields.

Signed-off-by: Mark Stemm <[email protected]>
list_fields() is used by programs that use the falcosecurity libs but
are not falco, with its strong notion of event types.

Replace the old version, which used printf and character-by-character
iteration over line strings, with using
sinsp_filter_factory::check_infos_to_fieldclass_infos to get a generic
representation of field classes, and
filter_fieldclass_info::as_string to format the field class info as a
string.

markdown output is now in a dedicated (static) function list_fields_markdown().

This allows for a consistent display of field info across programs.

Signed-off-by: Mark Stemm <[email protected]>
@mstemm mstemm force-pushed the cleanup-list-fields branch from 345aa4e to db5b6cf Compare December 20, 2021 18:50
This ensures that they will be displayed next to each other when
printing all fields.

Signed-off-by: Mark Stemm <[email protected]>
@mstemm
Copy link
Contributor Author

mstemm commented Dec 20, 2021

Sorry I didn't know markdown output was actually used. I added it back. PTAL again!

Tested this change and terminal+markdown formatting seems to be right. Just a little thing: I noticed that evt (All event types) appears at the bottom of the list, would it be possible to make it appear on top? This would make the documentation easier to consume when piped to a pager or exported to the documentation website.

There isn't any explicit sorting of fields at the moment, the order is the order in which the filterchecks are added to the global sinsp_filter_check_list: https://github.com/falcosecurity/libs/blob/master/userspace/libsinsp/filter_check_list.cpp#L106. But I did move sinsp_filter_check_gen_event to the top so it will be next to evt (Syscall events only).

Copy link
Contributor

@LucaGuerra LucaGuerra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mstemm yes, I did the same when I wanted to reorder the doc page. LGTM.

@poiana
Copy link
Contributor

poiana commented Dec 20, 2021

@LucaGuerra: changing LGTM is restricted to collaborators

In response to this:

@mstemm yes, I did the same when I wanted to reorder the doc page. LGTM.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@leogr
Copy link
Member

leogr commented Dec 21, 2021

/retest

@leogr
Copy link
Member

leogr commented Dec 21, 2021

/check-dco

Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana poiana added the lgtm label Dec 21, 2021
@poiana
Copy link
Contributor

poiana commented Dec 21, 2021

LGTM label has been added.

Git tree hash: 3beda0f0710961840fac24f3aacb20266d390b9e

@poiana
Copy link
Contributor

poiana commented Dec 21, 2021

@FedeDP: changing LGTM is restricted to collaborators

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ldegio ldegio self-requested a review December 22, 2021 19:10
@poiana poiana merged commit bb9bee8 into master Dec 22, 2021
@poiana poiana deleted the cleanup-list-fields branch December 22, 2021 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants