Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for huge TLB pages on Linux #97

Merged
merged 1 commit into from
Mar 8, 2018

Conversation

jsoriano
Copy link
Member

Add initial support for huge TLB pages on Linux, as requirement for elastic/beats#6351.

} else {
// If Hugetlb is not present, or huge pages of different sizes
// are used, this cipher can be unaccurate.
// TODO (jsoriano): Extract information from /sys/kernel/mm/hugepages too
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it is so common to use custom sizes for huge pages.

self.Surplus, _ = table["HugePages_Surp"]
self.DefaultSize, _ = table["Hugepagesize"]

if totalSize, found := table["Hugetlb"]; found {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This value is documented, but I haven't seen it in any running Linux I could check now.

@kvch
Copy link

kvch commented Feb 21, 2018

Could you add a stub returning ErrNotImplemented to sigar_stub.go?

@jsoriano
Copy link
Member Author

Added, thanks!

Copy link

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @andrewkroh are you ok with merging this?

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsoriano Welcome to the team! The code you added looks good.

I am slightly concerned about the further expansion of the Sigar interface. This isn't a new issue. And it reminds me of "The bigger the interface, the weaker the abstraction" from Rob Pike's talk.

Do you think we will ever add HugeTLBPages implementations for OSes other than Linux? If not, then maybe gosigar isn't the right place for this. Perhaps adding a MemInfo type to procfs would be a better home for this.

I'm interested to hear what others think about this.

@jsoriano
Copy link
Member Author

There seem to be also superpages in FreeBSD and large pages in Windows, I guess there will also be similar concepts in Mac, OpenBSD... What I haven't checked is their metric interfaces and terminology, maybe we can find a less linux-specific naming.

@jsoriano
Copy link
Member Author

jsoriano commented Feb 23, 2018

I have moved the code from the linux_common to the linux file. The linux_common file is also used by freebsd and freebsd implementation is quite different.
freebsd implementation should be reviewed in general, as it depends quite a lot in /proc and it is being deprecated, in recent implementations it contains very few information. Information can be obtained now with sysctl or with the procstat command, including use of superpages.
Edit: freebsd has a more linux-compatible version of procfs with the linprocfs filesystem.

@jsoriano
Copy link
Member Author

Added a commit renaming this to larger pages, that sounds like a more generic name.

@andrewkroh
Copy link
Member

The hardest problem is always naming. I think using Large, Huge, or Super in the name is better than creating a fourth one of "Larger". I'm going to hedge my bets and say that we will probably never implement it for any other OSes so we should stick with "Huge". Can you undo the last commit with the rename?

@jsoriano
Copy link
Member Author

jsoriano commented Mar 8, 2018

Commit with the rename removed :)

@andrewkroh andrewkroh merged commit e8599e2 into elastic:master Mar 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants