Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase the monitor throughput: increase the default fetch size, make it configurable. #171

Merged
merged 2 commits into from
Mar 29, 2021

Conversation

aleksmaus
Copy link
Contributor

What does this PR do?

Increases the monitor throughput:

  • Increase the monitor fetch size to 1000 by default, the max number of documents fetched at once.
  • Makes the monitor fetch size configurable.
  • Decreased the pause between fetches from 50ms to 10ms.

Why is it important?

Tuning of the throughput.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas

@aleksmaus aleksmaus added the enhancement New feature or request label Mar 25, 2021
@elasticmachine
Copy link
Contributor

elasticmachine commented Mar 25, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #171 updated

  • Start Time: 2021-03-26T14:28:36.278+0000

  • Duration: 13 min 49 sec

  • Commit: 3ac243b

Test stats 🧪

Test Results
Failed 0
Passed 64
Skipped 0
Total 64

Trends 🧪

Image of Build Times

Image of Tests

Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The location of the configuration options needs to be adjusted.

@@ -27,13 +27,17 @@ type Config struct {
Inputs []Input `config:"inputs"`
Logging Logging `config:"logging"`
HTTP HTTP `config:"http"`
Cache Cache `config:"cache"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cache was moved under the input level of the configuration, where it must be for Elastic Agent to be able to adjust these values.

@@ -27,13 +27,17 @@ type Config struct {
Inputs []Input `config:"inputs"`
Logging Logging `config:"logging"`
HTTP HTTP `config:"http"`
Cache Cache `config:"cache"`
Monitor Monitor `config:"monitor"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also needs to be added under the input and not at this top-level.

@aleksmaus
Copy link
Contributor Author

Updated, moved the monitor config into input config.

Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes.

@aleksmaus aleksmaus merged commit e475479 into elastic:master Mar 29, 2021
aleksmaus added a commit to aleksmaus/fleet-server that referenced this pull request Mar 30, 2021
…e it configurable. (elastic#171)

* Increase the monitor throughput: increase the default fetch size, make it configurable.

* Address the code review feedback

(cherry picked from commit e475479)
aleksmaus added a commit to aleksmaus/fleet-server that referenced this pull request Mar 30, 2021
…e it configurable. (elastic#171)

* Increase the monitor throughput: increase the default fetch size, make it configurable.

* Address the code review feedback

(cherry picked from commit e475479)
mergify bot pushed a commit that referenced this pull request Mar 30, 2021
…e it configurable. (#171)

* Increase the monitor throughput: increase the default fetch size, make it configurable.

* Address the code review feedback

(cherry picked from commit e475479)
mergify bot pushed a commit that referenced this pull request Mar 30, 2021
…e it configurable. (#171) (#188)

* Increase the monitor throughput: increase the default fetch size, make it configurable.

* Address the code review feedback

(cherry picked from commit e475479)
mergify bot added a commit that referenced this pull request Mar 30, 2021
…e it configurable. (#171) (#189)

* Increase the monitor throughput: increase the default fetch size, make it configurable.

* Address the code review feedback

(cherry picked from commit e475479)

Co-authored-by: Aleksandr Maus <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request v7.13.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants