Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure cache #443

Merged
merged 3 commits into from
Jun 10, 2021
Merged

Configure cache #443

merged 3 commits into from
Jun 10, 2021

Conversation

scunningham
Copy link

What does this PR do?

Makes the TTL values on the cache objects configurable. Allows cache to be reconfigured without restarting the process. Cache is dropped on reconfiguration.

Increate the TTL on the api key to 15m with optional jitter. This is to work around latency issues at scale with validating API . In addition, validate that 'enabled' flag on the agent during auth. Between the 'enabled' flag, and the ability to drop the cache by reconfiguration in a break glass scenario, it seems minimize auth round trips to elastic. This takes load off the fleet-server as well as elastic.

Also, cache the case when an api key is not valid. This avoids roundtrip for an already disabled api key.

Why is it important?

The server runs into issues at large scale while trying to validate many ApiKeys at once. Particularly when a policy changes and there are many GET requests for artifacts.

Checklist

  • [x ] My code follows the style guidelines of this project
  • [x ] I have commented my code, particularly in hard-to-understand areas
  • [x ] I have made corresponding changes to the documentation
  • [x ] I have made corresponding change to the default configuration files
  • [x ] I have added tests that prove my fix is effective or that my feature works
  • [x ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 8, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #443 updated

  • Start Time: 2021-06-10T14:39:35.595+0000

  • Duration: 4 min 18 sec

  • Commit: 10d45b5

Test stats 🧪

Test Results
Failed 0
Passed 178
Skipped 0
Total 178

Trends 🧪

Image of Build Times

Image of Tests

@scunningham scunningham added enhancement New feature or request v7.14.0 labels Jun 8, 2021
@scunningham scunningham force-pushed the cfgCache branch 2 times, most recently from e6397a3 to e4798bc Compare June 8, 2021 18:20
@scunningham scunningham requested a review from urso June 9, 2021 10:53
Sean Cunningham added 3 commits June 10, 2021 10:38
Allow object TTL's to be configurable.
Default apikey TTL to 15m to avoid auth bottleneck.
Copy link
Contributor

@aleksmaus aleksmaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

val = ""
}

// If enabled, jitter allows us to randomize the expirtion of the artifact
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo "expirtion"

@scunningham scunningham merged commit aadaa4c into elastic:master Jun 10, 2021
@scunningham scunningham deleted the cfgCache branch June 10, 2021 17:15
mergify bot added a commit that referenced this pull request Jun 10, 2021
* Tweak server timeouts.  Limit body size to defend malicious agent

(cherry picked from commit f335936)

* Refactor bulk init

(cherry picked from commit 0cd883a)

* Drop cache if configuration changes.
Allow object TTL's to be configurable.
Default apikey TTL to 15m to avoid auth bottleneck.

(cherry picked from commit aadaa4c)

Co-authored-by: Sean Cunningham <[email protected]>
@urso urso added needs-test test-plan PRs/issues that require additional manual QA labels Jul 9, 2021
@andresrc andresrc added Team:Elastic-Agent Label for the Agent team test-plan-added labels Jul 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs-test Team:Elastic-Agent Label for the Agent team test-plan PRs/issues that require additional manual QA test-plan-added v7.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants