Add process monitor capabilities to flows (#541) #7507
Conversation
adriansr
force-pushed
the
feature/pb/flow_procs
branch
from
1e31928 to
3d621df
Compare
Flows are updated to include information about the local processes that generate them. When one or both ends of a flow is identified as a local process, this feature will add the following fields: "proc": Process name for the server-side of the flow, as set up in packetbeat.procs.monitored. "cmdline": Command-line for the local process on the server-side of the flow. "client_proc": Process name for the client-side of the flow, as set up in packetbeat.procs.monitored. "client_cmdline": Command-line for the local process on the client-side of the flow.
adriansr
force-pushed
the
feature/pb/flow_procs
branch
from
3d621df to
c3cfb67
Compare
|
I wonder if we should use ECS fields here where applicable? |
|
That makes complete sense, @ruflin. In this PR I just mimicked the fields names used in the other packetbeat protocols, we should create an issue to rename them all to conform to ECS |
|
I'm not terribly familiar with the Beats code base yet, so take this with the appropriate grain of salt :-)
These two questions have me a bit worried |
|
Hi @webmat Currently, for every logged event that has a local IP as source or destination, the TCP port is looked up in a process cache. Only if not found in this cache, then all processes are scanned for its open ports. It is an expensive operation, usually ranging from 10 ms under Linux and about 100 ms in Windows. As processes monitoring is disabled by default, I think it makes sense to make it more precise when it's enabled, at the expense of some overhead. Previously it would only update the process list after a fixed timeout, causing some events to not be enriched with process information, especially the short-lived processes and sockets. About the flows, yes, it's handling all of them. Of course, there's room for improvement. Feel free to work on it :) |
|
Got it, works for me if this is disabled by default. 👍 |
| @@ -79,10 +79,18 @@ | |||
| description: > | |||
| The name of the process that served the transaction. | |||
|
|
|||
| - name: cmdline | |||
There was a problem hiding this comment.
So these are not new, correct? They were just missing from fields.yml before?
There was a problem hiding this comment.
Do we have something like the fields_check we have for metricbeat in packetbeat to detect these?
There was a problem hiding this comment.
Yes, those were missing
There was a problem hiding this comment.
@ruflin The packetbeat system tests all appear to be checking that the fields are documented. But there are no tests with live data for the process enrichment feature (the feature is disabled when reading from pcaps since that isn't live).
beats/packetbeat/tests/system/packetbeat.py
Line 119 in f3aa089
Flows are updated to include information about the local processes that generate them.
When one or both ends of a flow is identified as a local process, this feature will add the following fields:
"proc": Process name for the server-side of the flow, as set up in packetbeat.procs.monitored.
"cmdline": Command-line for the local process on the server-side of the flow.
"client_proc": Process name for the client-side of the flow, as set up in packetbeat.procs.monitored.
"client_cmdline": Command-line for the local process on the client-side of the flow.
Partially addresses #541