-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent pipeline from queueing large batches before an output is set #34741
Conversation
Pinging @elastic/elastic-agent (Team:Elastic-Agent) |
This pull request does not have a backport label.
To fixup this pull request, you need to add the backport labels for the needed
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need tests for the new behaviour.
Tests added for both checks. The |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Two small changes to batch assembly in the pipeline:
eventConsumer
, don't start assembling batches until we get a non-nil output channeloutputController
, don't provideeventConsumer
with a non-nil output channel if the output list is empty.The first one prevents batches from being assembled while the output configuration is being updated. The second prevents batches from being assembled when running under Agent if we haven't yet received our output configuration. (When running under Agent, there is typically no output when
pipeline.New
is called, which previously allowed the construction of batches with no size limit before the output was really initialized.)Fixes #34703.
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesI have added an entry inCHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.