Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(uptime): Use multi-processing on the result consumer #82118

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

wedamija
Copy link
Member

This is a rough try at getting multi proc to work. It's not ideal because it doesn't wait for each batch to finish, so could potentially process out of order

This adds thread pool parallelization to the uptime consumer. We should potentially consider process parallelizatoin as well, so that we can take advantage of all cores.

This is a rough draft of this and isn't tested - based off the issue occurrence consumer and monitor consumer.
This is a rough try at getting multi proc to work. It's not ideal because it doesn't wait for each batch to finish, so could potentially process out of order
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Dec 14, 2024
Copy link

codecov bot commented Dec 14, 2024

Codecov Report

Attention: Patch coverage is 88.00000% with 9 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
.../remote_subscriptions/consumers/result_consumer.py 87.50% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #82118      +/-   ##
==========================================
- Coverage   80.36%   80.36%   -0.01%     
==========================================
  Files        7273     7273              
  Lines      321302   321369      +67     
  Branches    20948    20948              
==========================================
+ Hits       258227   258277      +50     
- Misses      62673    62690      +17     
  Partials      402      402              

Base automatically changed from danf/uptime-parallel-consumer to master December 18, 2024 23:24
@getsantry
Copy link
Contributor

getsantry bot commented Jan 9, 2025

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

@getsantry getsantry bot added Stale and removed Stale labels Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant