Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve collection of process memory metrics in Windows #8345

Closed
jsoriano opened this issue Sep 18, 2018 · 5 comments
Closed

Improve collection of process memory metrics in Windows #8345

jsoriano opened this issue Sep 18, 2018 · 5 comments
Labels
discuss Issue needs further discussion. enhancement Metricbeat Metricbeat module Stalled Team:Integrations Label for the Integrations team :Windows

Comments

@jsoriano
Copy link
Member

Currently only Private Bytes and Working Set Size is being collected for Windows processes, it can be also interesting to collect Virtual Bytes. Also system.process.memory.share is not being calculated, remaining always as 0.
For more context about this look at this thread: https://discuss.elastic.co/t/virtual-bytes-wrong-counter-in-metricbeat

Thinking on multi-platform support we might want to check what would be an equivalent of these metrics in Windows and Linux.

@jsoriano jsoriano added enhancement module discuss Issue needs further discussion. :Windows Metricbeat Metricbeat labels Sep 18, 2018
@ruflin ruflin added the Team:Integrations Label for the Integrations team label Nov 21, 2018
@martinscholz83
Copy link
Contributor

Here is a good explanation of the differences and why it's a good idea to collect the virtual bytes.

@lachlann562
Copy link

We are looking to use metric beat and virtual bytes is a critical metric for our applications. When our system becomes unstable its always due to virtual bytes and not working set/private bytes. Is there any timeline to schedule this enhancement?

@narph
Copy link
Contributor

narph commented Mar 26, 2020

@jsoriano , indeed the system.process.memory field descriptions do not reflect the actual memory values we retrieve for windows os's.
A PR is opened to fix this #17268.

Regarding adding virtual bytes in the metricset:
system.process.memory.share is not mapped at the moment, in case we want to add virtual bytes or other memory metrics for Windows we might want to use a more Windows appropriate naming.

We are currently using GetProcessMemoryInfo and GlobalMemoryStatusEx win32 apis to retrieve the mapped values, unfortunately they do not contain the virtual bytes, (GlobalMemoryStatusEx will do but only for the current process running).
VirtualQueryEx api could return this value but to get to the total virtual bytes count we need to iterate over all the addressable pages which is not performant enough.
I am afraid that at the moment the only option is to use the windows perfmon metricset and retrieve the virtual bytes using the \Process(*)\Virtual Bytes counter path.

@jsoriano
Copy link
Member Author

jsoriano commented Apr 1, 2020

Now that @fearful-symmetry has started to work on OS-specific system modules (#17267), it would make more sense to use specific windows metricsets to get these metrics. Maybe we could have a windows.memory metricset that uses perfmon to collect them.

@narph narph removed their assignment Oct 25, 2021
@botelastic
Copy link

botelastic bot commented Oct 25, 2022

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Oct 25, 2022
@botelastic botelastic bot closed this as completed Apr 23, 2023
@zube zube bot removed the [zube]: Done label Jul 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Issue needs further discussion. enhancement Metricbeat Metricbeat module Stalled Team:Integrations Label for the Integrations team :Windows
Projects
None yet
Development

No branches or pull requests

7 participants