Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metrics: activity log #10514

Merged
merged 11 commits into from
Jan 26, 2021
Merged

metrics: activity log #10514

merged 11 commits into from
Jan 26, 2021

Conversation

alexanderbez
Copy link
Contributor

No description provided.

@alexanderbez alexanderbez added the core Issues and Pull-Requests specific to Vault Core label Dec 8, 2020
@alexanderbez
Copy link
Contributor Author

@mgritter I've added vault.identity.entity.active.monthly and vault.identity.entity.active.reporting_period. I haven't dug too much into vault.identity.entity.active.partial_month, but I recall you stating that one might not be possible? I can't recall.

I'm marking this ready-for-review just to get some feedback if these are what you expected.

@alexanderbez alexanderbez marked this pull request as ready for review December 11, 2020 20:44
endTime := timeutil.EndOfMonth(time.Unix(lastMonth, 0).UTC())
activePeriodStart := endTime.AddDate(0, -a.defaultReportMonths, 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unfortunately doesn't work as expected. Suppose endTime is October 31, then subtracting 8 months gives not Feb 28th or 29th but "Feb 31th" which becomes March 3 or 2.

timeutil.MonthsPreviousTo should do what you need here, though it picks the start of the month. (The same problem does not arise if using the 1st of the month.)

for nsID, counts := range byNamespace {
pq.Namespaces = append(pq.Namespaces, &activity.NamespaceRecord{
NamespaceID: nsID,
Entities: uint64(len(counts.Entities)),
NonEntityTokens: counts.Tokens,
})

if startTime.After(activePeriodStart) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could fire for every previous month in our retention period, not just once at defaultReportMonths.

If you use the utility function suggested above, that should allow an equality check instead... except for the potential partial month (if the feature was enabled on Jan 15, then that first startTime will be Jan 15, not Jan 1.)

So, I would suggest using timeutil.InRange( startTime, activePeriodStart, EndOfMonth( activePeriodStart )) or writing a new utility in timeutil that does that.

Comment on lines 1660 to 1663
// emit active entity metrics for the interval (times[0], endTime)
for nsID, counts := range byNamespace {
a.metrics.SetGaugeWithLabels(
[]string{"identity", "entity", "active", "monthly"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment is correct but I don't believe that is what this code does

Because we've finished the loop, byNamespace contains all the data from the interval (times[len(times)-1], endTime). times is in reverse order with 0 being the most recent month.

A check at the same place you're doing the defaultReportingPeriod check would be sufficient.

@mgritter
Copy link
Contributor

@mgritter I've added vault.identity.entity.active.monthly and vault.identity.entity.active.reporting_period. I haven't dug too much into vault.identity.entity.active.partial_month, but I recall you stating that one might not be possible? I can't recall.

partial_month is easy without a namespace breakdown, it is just the size of activeEntities (once fully reloaded, might be multiple storage entries.) To do a breakdown by namespace like you did with these two means we would have to walk the segments on storage where that information lives (whenever we emit the metrics) or keep the per-namespace breakdown in memory all the time. Both are feasible, but maybe we don't want to touch storage every 10 minutes so a longer interval would be good.

The storage size metric is the one I thought might be infeasible.

@vercel
Copy link

vercel bot commented Jan 22, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployments, click below or on the icon next to each commit.

vault – ./website

🔍 Inspect: https://vercel.com/hashicorp/vault/bjy5xmm9w
✅ Preview: Canceled

[Deployment for 16fdccd failed]

vault-storybook – ./ui

🔍 Inspect: https://vercel.com/hashicorp/vault-storybook/1hq7d9y6f
✅ Preview: https://vault-storybook-git-core-vlt-898-activity-log-metrics.hashicorp.vercel.app

[Deployment for 5c632ef canceled]

@mgritter mgritter requested a review from swayne275 January 26, 2021 19:38
Copy link
Contributor

@swayne275 swayne275 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved but those two suggestions look in line with the other documentation

website/pages/docs/internals/telemetry.mdx Outdated Show resolved Hide resolved
website/pages/docs/internals/telemetry.mdx Outdated Show resolved Hide resolved
@vercel vercel bot temporarily deployed to Preview – vault January 26, 2021 22:21 Inactive
@vercel vercel bot temporarily deployed to Preview – vault-storybook January 26, 2021 22:21 Inactive
@mgritter mgritter merged commit c46a966 into master Jan 26, 2021
@mgritter mgritter deleted the core/vlt-898-activity-log-metrics branch January 27, 2021 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core/metric core Issues and Pull-Requests specific to Vault Core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants