Additional derived Redis metrics #1015
Merged
+9
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We run a bunch of Redis DBs, and some of them are meant for ephemeral data, over 99.9% of which should have a TTL set. There are several thousand keys that do not, but these are a tiny portion of the database. Should that portion ever increase, we'd like to alert, because it indicates that something is wrong and we're filling our ephemeral store up with non-ephemeral data.
While the current Redis monitoring does let us track the total number of keys and the number of keys with TTLs, Datadog does not (to our knowledge) currently let us alert on a mathematically derived figure, such as the total number of persistent keys (i.e. total keys minus keys with TTLs) or the percentage of keys that are persistent / will expire.
This PR adds three new figures:
redis.persist
—redis.keys
minusredis.expires
redis.persist.percent
—redis.persist
as a percentage ofredis.keys
redis.expires.percent
—redis.expires
as a percentage ofredis.keys
If there's a way to alert on these derived values, we'd love to hear it; similarly, if this PR is accepted and then alerting on derived values is later added, this can of course be reverted. But for now, this is the code we're using, and we're happy to contribute it back if you think it's useful.