Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-2021 Improve UX #146

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Conversation

jpinsonneau
Copy link
Contributor

@jpinsonneau jpinsonneau commented Jan 8, 2025

Description

  • if one or more filters option is enabled to automatically enable filtering and disable when we have no filtering options and probably in this case we can remove --enable-filter option
  • if filtering require feature specific we can enable the feature and when the filter is removed disable it
  • support multi filter rules
  • add examples
  • List options in alphabetical order
  • List option common separately to both flows and packets to avoid listing them separately for log-level, max-time etc.
  • Use true as default values when option is set, i.e. only check for value == "false" to avoid setting value== true 
  • Have enable_all option to enable all features. 
  • Address OSDOCS-10730 feedback
  • Force having at least one eBPF filter when running packet capture
  • Add missing UDN and pkt xlat features
  • Add Network Name columns

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
    • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
    • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
    • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
    • Standard QE validation, with pre-merge tests unless stated otherwise.
    • Regression tests only (e.g. refactoring with no user-facing change).
    • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Copy link

openshift-ci bot commented Jan 8, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

function setLastFlowFilter() {
"$YQ_BIN" e --inplace " .spec.template.spec.containers[0].env[] |= select(.name == \"FLOW_FILTER_RULES\").value |=(fromjson | .[-1].$1 = $2 | tostring)" "$3"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do u have sample output with mutli rule, I have standalone script fo ut this logic maybe we bring this in and intg with ut ci as logic start to become complex ?

Copy link
Contributor Author

@jpinsonneau jpinsonneau Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I will add examples and tests !

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the following scenarios for now:

basic examples:
  netobserv flows --drops         # Capture dropped flows on all nodes
  netobserv packets --port=8080   # Capture packets on port 8080
  netobserv metrics --enable_all  # Capture all cluster metrics with pktDrop, dns, rtt and network events features

advanced examples:
  Capture drops in background and copy output locally
    netobserv flows --background \                            # Capture flows using background mode
    --max-time=15m \                                          # for a maximum of 15 minutes
    --protocol=TCP --port=8080 \                              # either on TCP 8080
    or --protocol=UDP                                         # or UDP
    netobserv follow                                          # Display the progression of the background capture
    netobserv stop                                            # Stop the background capture by deleting eBPF agents
    netobserv copy                                            # Copy the background capture output data
    netobserv cleanup                                         # Cleanup netobserv CLI by removing the remaining collector pod

  Capture packets on specific nodes and port
    netobserv packets                                         # Capture packets
    --node-selector=netobserv:true \                          # on nodes labelled with netobserv=true
    --port=80 \                                               # on port 80 only
    --max-bytes=100000000                                     # for a maximum of 100MB

Please let me know if you feel more would be useful

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u pls include example with mutli rules ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have one above in the advanced examples 😸

@jpinsonneau jpinsonneau changed the title NETOBSERV-2021 Improve filtering UX NETOBSERV-2021 Improve UX Jan 10, 2025
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

Attention: Patch coverage is 75.00000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 23.45%. Comparing base (547a146) to head (7ccb096).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
cmd/map_format.go 66.66% 6 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #146      +/-   ##
==========================================
+ Coverage   22.51%   23.45%   +0.93%     
==========================================
  Files          11       11              
  Lines        1310     1326      +16     
==========================================
+ Hits          295      311      +16     
+ Misses        999      998       -1     
- Partials       16       17       +1     
Flag Coverage Δ
unittests 23.45% <75.00%> (+0.93%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
cmd/flow_capture.go 31.78% <100.00%> (+1.38%) ⬆️
cmd/options.go 25.00% <ø> (ø)
cmd/map_format.go 25.05% <66.66%> (+1.73%) ⬆️

@msherif1234
Copy link
Contributor

BTW if u added the rules in reverse order this will break filtering , rules has to place in the json array in the same order they are configured.

@jpinsonneau
Copy link
Contributor Author

BTW if u added the rules in reverse order this will break filtering , rules has to place in the json array in the same order they are configured.

The order is kept as I append the whole json from this file: https://github.com/netobserv/network-observability-cli/blob/994f4b4888d67fc72f1a90cd72d4295a50614507/res/flow-filter.json

However we should document this behavior as it's can break easily !

@jpinsonneau
Copy link
Contributor Author

/retest

1 similar comment
@jpinsonneau
Copy link
Contributor Author

/retest

Copy link

@skrthomas skrthomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few nitties 👼 , and otherwise LGTM! Thanks Julien :)

docs/netobserv_cli.adoc Outdated Show resolved Hide resolved
docs/netobserv_cli.adoc Outdated Show resolved Hide resolved
docs/netobserv_cli.adoc Outdated Show resolved Hide resolved
@memodi
Copy link

memodi commented Jan 21, 2025

/ok-to-test

Copy link

New image:
quay.io/netobserv/network-observability-cli:ac57fb3

It will expire after two weeks.

To use this build, update your commands using:

USER=netobserv VERSION=ac57fb3 make commands

or download the updated commands.

@jpinsonneau
Copy link
Contributor Author

Rebased without changes

@msherif1234
Copy link
Contributor

/LGTM
once agent PR merge lets ee if that will clear e2e run Thanks!!

@openshift-ci openshift-ci bot removed the lgtm label Jan 24, 2025
@jpinsonneau
Copy link
Contributor Author

/retest

Copy link
Member

@jotak jotak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link

New image:
quay.io/netobserv/network-observability-cli:1d1df34

It will expire after two weeks.

To use this build, update your commands using:

USER=netobserv VERSION=1d1df34 make commands

or download the updated commands.

@jpinsonneau
Copy link
Contributor Author

jpinsonneau commented Jan 28, 2025

@jotak is it worth replacing UDN column by network names here in the end ?

I see the UDNs column is still there as before so that's covered in this PR. We can create a followup for NetworkNames if needed as it impacts the pipeline as you mentionned in today's scrum

Copy link

@memodi memodi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpinsonneau thanks for addressing UX issues. It looks great.

quick question: when I enable packet translation, how do I get that view?

@jpinsonneau
Copy link
Contributor Author

@jpinsonneau thanks for addressing UX issues. It looks great.

quick question: when I enable packet translation, how do I get that view?

You can use enable_pkt_translation flag and then cycle to the view Packet translation during the capture to see the Xlat coumns

@openshift-ci openshift-ci bot removed the lgtm label Jan 29, 2025
Copy link

openshift-ci bot commented Jan 29, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jotak. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jpinsonneau
Copy link
Contributor Author

@jotak & @memodi I have added the Network Names columns as discussed yesterday
7ccb096

The secondaryNetworks are also configured in the pipeline config when udn feature is enabled.

@openshift-ci openshift-ci bot added the lgtm label Jan 29, 2025
@memodi
Copy link

memodi commented Jan 29, 2025

You can use enable_pkt_translation flag and then cycle to the view Packet translation during the capture to see the Xlat coumns

@jpinsonneau yes, I did use enable_pkt_translation but could not find Packet translation view in "Display"

@jpinsonneau
Copy link
Contributor Author

jpinsonneau commented Jan 29, 2025

You can use enable_pkt_translation flag and then cycle to the view Packet translation during the capture to see the Xlat coumns

@jpinsonneau yes, I did use enable_pkt_translation but could not find Packet translation view in "Display"

@memodi don't you get that output ?

image

@memodi
Copy link

memodi commented Jan 29, 2025

/ok-to-test

Copy link

New image:
quay.io/netobserv/network-observability-cli:479b4f5

It will expire after two weeks.

To use this build, update your commands using:

USER=netobserv VERSION=479b4f5 make commands

or download the updated commands.

@memodi
Copy link

memodi commented Jan 29, 2025

@jpinsonneau I am testing network names, but I am not able to see the network names that I am expecting, I am using "--enable_all" , am I missing something? I confirmed I am able see it on operator side.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants