Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.14](backport #40081) [debugPrintProcessor]: exit directly when log doesn't have debug level enabled #40126

Closed
wants to merge 1 commit into from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jul 5, 2024

Proposed commit message

This PR is the aftermath of profiling the netflow integration test of this PR (PS: the pcap I used as input is ~25MB thus I won't include it in beats repo)

So this is the cpu profile flamegraph before the commit of this PR

Screenshot 2024-07-02 at 10 38 54 PM

As we can see debugPrintProcessor is invoked and we pay the "performance" price of encoding every event in json even when our log isn't set up with DEBUG level.

The respective cpu profile flamegraph after this PR with debugPrintProcessor still being there but exiting directly when log DEBUG is not enabled

Screenshot 2024-07-02 at 10 42 16 PM

Extracting the average EPS of the aformentioned netflow integration tested on my local macbook with a local ES inside a container

Before this PR With this PR
18490 33760

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

N/A

Author's Checklist

N/A

How to test this PR locally

Help get this PR merged or clone it and apply this small commit there 🙂

cd x-pack/filebeat
mage docker:composeUp

run go profiler against TestNetFlowIntegration

Related issues

Use cases

N/A

Screenshots

Look at description

Logs

N/A


This is an automatic backport of pull request #40081 done by Mergify.

… level enabled (#40081)

(cherry picked from commit 58f4fc8)

# Conflicts:
#	libbeat/publisher/processing/processors.go
@mergify mergify bot requested a review from a team as a code owner July 5, 2024 17:54
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Jul 5, 2024
@mergify mergify bot requested review from AndersonQ and belimawr and removed request for a team July 5, 2024 17:54
Copy link
Contributor Author

mergify bot commented Jul 5, 2024

Cherry-pick of 58f4fc8 has failed:

On branch mergify/bp/8.14/pr-40081
Your branch is up to date with 'origin/8.14'.

You are currently cherry-picking commit 58f4fc8579.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   libbeat/publisher/processing/processors.go

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 5, 2024
@botelastic
Copy link

botelastic bot commented Jul 5, 2024

This pull request doesn't have a Team:<team> label.

Copy link
Contributor Author

mergify bot commented Jul 8, 2024

This pull request has not been merged yet. Could you please review and merge it @pkoutsovasilis? 🙏

@pierrehilbert
Copy link
Collaborator

@pkoutsovasilis we will have our first 8.14.3 today, could you please take care of this one asap?

@pkoutsovasilis
Copy link
Contributor

pkoutsovasilis commented Jul 8, 2024

hi @pierrehilbert, as far as I can tell this PR isn't required for 8.14.x as the change that introduced the debugPrintProcessor isn't here, so we are all good. In other words let's close this PR

PS: I don't remember adding the backport 8.14 on the original PR 😀

@pierrehilbert
Copy link
Collaborator

@cmacknz added it, from what I can understand in this comment #40081 (comment), we need it for standalone Beats prior to 8.15

@pkoutsovasilis
Copy link
Contributor

@cmacknz added it, from what I can understand in this comment #40081 (comment), we need it for standalone Beats prior to 8.15

interesting... The commit that introduced it is this de3318d and I see it only in branches main and 8.15 so I think the backporting to 8.14 was an accident!?

git branch -a --contains de3318d19e66dbcecf0ef5235a61e22035d24a36
remotes/beats/8.15
remotes/beats/main
...

@pierrehilbert
Copy link
Collaborator

I think Craig was talking about 83dfb2f

@pkoutsovasilis
Copy link
Contributor

good morning @pierrehilbert, I still think that this PR is not required. Maybe @cmacknz you can help us here? 🙂

Copy link
Contributor Author

mergify bot commented Jul 15, 2024

This pull request has not been merged yet. Could you please review and merge it @pkoutsovasilis? 🙏

1 similar comment
Copy link
Contributor Author

mergify bot commented Jul 22, 2024

This pull request has not been merged yet. Could you please review and merge it @pkoutsovasilis? 🙏

@mergify mergify bot deleted the mergify/bp/8.14/pr-40081 branch July 22, 2024 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport conflicts There is a conflict in the backported pull request needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants