Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rec: Compute size *estimate* for record cache allocated size #15038

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

omoerbeek
Copy link
Member

Short description

This implements a better computation of the metric cache-bytes. The existing one neglected the size of the individual records.
This one might overestimate in some cases (string storage in string object itself) and underestimate because malloc bucket sizes, but at least it takes into account variable sized records.

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)

@omoerbeek omoerbeek added this to the rec-5.3.0 milestone Jan 14, 2025
@rgacogne rgacogne self-requested a review January 14, 2025 10:51
@omoerbeek omoerbeek changed the title rec: Compute size *estimate* for record cache allocted size rec: Compute size *estimate* for record cache allocated size Jan 14, 2025
@coveralls
Copy link

coveralls commented Jan 14, 2025

Pull Request Test Coverage Report for Build 12768425464

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 92 of 231 (39.83%) changed or added relevant lines in 5 files are covered.
  • 67 unchanged lines in 12 files lost coverage.
  • Overall coverage decreased (-9.2%) to 64.887%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pdns/recursordist/recursor_cache.cc 31 35 88.57%
pdns/dnsparser.hh 0 6 0.0%
pdns/dnsrecords.hh 40 169 23.67%
Files with Coverage Reduction New Missed Lines %
pdns/packethandler.cc 1 72.74%
pdns/recursordist/lwres.cc 2 69.66%
pdns/recursordist/sortlist.cc 2 72.94%
modules/lmdbbackend/lmdbbackend.cc 2 72.75%
pdns/rcpgenerator.cc 2 90.55%
pdns/misc.hh 3 87.62%
modules/godbcbackend/sodbc.cc 4 70.8%
pdns/recursordist/pdns_recursor.cc 4 72.62%
pdns/recursordist/syncres.cc 5 80.33%
pdns/signingpipe.cc 5 86.06%
Totals Coverage Status
Change from base Build 12765482410: -9.2%
Covered Lines: 126445
Relevant Lines: 164072

💛 - Coveralls

Copy link
Member

@rgacogne rgacogne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question but it looks very good!

@@ -680,6 +850,11 @@ public:

static constexpr size_t const nbTypes = 65536;

[[nodiscard]] size_t sizeEstimate() const
{
return d_bitset ? nbTypes / 8 : d_set.size() * (2U + sizeof(std::set<uint16_t>)); // XXX
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite get the last part of the calculation here?

Copy link
Member Author

@omoerbeek omoerbeek Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is: a set represented as a tree has size() nodes. Each node is at least 2 bytes (to store the uint16_t value) plus the internal node data, which is estimated as being the roughly same size as the head node of type std::set<uint16_t>. Come to think of it, the 2U is redundant as it is already included in sizeof(std::set<uint16_t>. I'll check if there is a better estimate possible.

@omoerbeek omoerbeek requested a review from rgacogne January 14, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants