-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Promtail: Drop stage #2496
Promtail: Drop stage #2496
Conversation
--- | ||
title: drop | ||
--- | ||
# `drop` stage |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a verb here? Or maybe change this to "Filter or drop logs"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for now I would like to leave this as is, which is consistent with how we title all our other stages.
|
||
Complex `drop` stage configurations specify multiple options in one stage or specify multiple drop stages | ||
|
||
#### Drop logs by regex AND length |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#### Drop logs by regex AND length | |
#### Drop logs by regex and length |
|
||
Would drop all logs that contain the word _debug_ *AND* are longer than 1024 bytes | ||
|
||
#### Drop logs by time OR length OR regex |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#### Drop logs by time OR length OR regex | |
#### Drop logs by time or length or regex |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not use capitalization for emphasis. Per the Documentation style guide, use italics for emphasis.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case the capitalization is less for emphasis and more to indicate that it's a binary operation, where say in a programming language you might write if x > 1 && x < 5
it's common to express this as if x >1 AND x < 5
mostly to disambiguate the logical operation from the actual word, (AND from and)
regex: ".*trace.*" | ||
``` | ||
|
||
Would drop all logs older than 24h OR longer than 8192 bytes OR have a json `msg` field containing the word _trace_ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would drop all logs older than 24h OR longer than 8192 bytes OR have a json `msg` field containing the word _trace_ | |
Would drop all logs older than 24h _or_ longer than 8192 bytes _or_ have a json `msg` field containing the word _trace_. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test 👮 in duty.
Will this work if promtail timestamps the file rather than extracting the timestamp from the log line? I guess that'd defeat the purpose as they're always time.Now()
Added a few nits, but LGTM after.
Codecov Report
@@ Coverage Diff @@
## master #2496 +/- ##
==========================================
+ Coverage 63.29% 63.39% +0.09%
==========================================
Files 162 163 +1
Lines 14114 14246 +132
==========================================
+ Hits 8934 9031 +97
- Misses 4470 4504 +34
- Partials 710 711 +1
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great tests, tyvm.
Comments to requested changes were added, apologies for dismissing the review but we are on the clock to get a release out.
The drop stage can be used to drop logs that meet provided criteria, the initial implementation will support dropping logs based on: