Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure that the task is not in the tasks queue when it is rejected from the executor #4901

Merged
merged 1 commit into from
Oct 16, 2023
Merged

Ensure that the task is not in the tasks queue when it is rejected from the executor #4901

merged 1 commit into from
Oct 16, 2023

Conversation

ahus1
Copy link
Contributor

@ahus1 ahus1 commented Oct 11, 2023

Closes #4900

Motivation:

This is a PR to address #4900. It is rather crude, and uses package visibility. While this does the job to make the assertions, I'd be happy to receive advice on how to change it to match the Vert.x's project way.

Conformance:

You should have signed the Eclipse Contributor Agreement as explained in https://github.com/eclipse/vert.x/blob/master/CONTRIBUTING.md

@ahus1 ahus1 mentioned this pull request Oct 11, 2023
Closed
@ahus1 ahus1 marked this pull request as ready for review October 11, 2023 17:38
@ahus1 ahus1 force-pushed the is-4900-remove-rejected-task-from-queue branch 3 times, most recently from 09e8e66 to e625617 Compare October 12, 2023 07:32
@ahus1
Copy link
Contributor Author

ahus1 commented Oct 12, 2023

Updated the PR to ensure proper encapsulation in TaskQueue even for tests by adding an isEmpty() method.

@ahus1 ahus1 force-pushed the is-4900-remove-rejected-task-from-queue branch from e625617 to 1ecd3bf Compare October 12, 2023 07:39
@vietj
Copy link
Member

vietj commented Oct 12, 2023

can you elaborate when/how this happens ? if a task is rejected it usually means that vertx is closed, unless I am wrong ?

@ahus1
Copy link
Contributor Author

ahus1 commented Oct 12, 2023 •
edited
Loading

For me, Vert.x is used as part of Quarkus, which is then the base of Keycloak. At the moment I'm running some load tests around Keycloak, and I want to have a first naive load shedding in place.

In a Keycloak setup, Quarkus replaces the executor pool of Vert.x with its own thread pool. I can ask Quarkus to limit the backlog of the executor thread pool using a configuration QUARKUS_THREAD_POOL_QUEUE_SIZE=100. Once I do this, the thread doesn't allow for an unlimited queue size. If under high load the queue size is exceeded, the requests are rejected.

I'm discussing more elaborate means of load shedding and not relying on the thread pool queue site. Until that is in place, this is all I have.

I hope this adds the context the issue is missing.

@ahus1 ahus1 mentioned this pull request Oct 12, 2023
Merged
@vietj
Copy link
Member

vietj commented Oct 13, 2023

thanks for the context

vietj
vietj reviewed Oct 13, 2023
View reviewed changes
src/main/java/io/vertx/core/impl/TaskQueue.java Outdated
@@ -125,19 +125,30 @@ public Consumer<Runnable> unschedule() {
*/
public void execute(Runnable task, Executor executor) {
synchronized (tasks) {
tasks.add(new ExecuteTask(task, executor));
ExecuteTask executeTask = new ExecuteTask(task, executor);
tasks.add(executeTask);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think instead since we are in a synchronized block, we could move this addition to the bottom of the block after we are sure the executor accepted the task instead of adding/removing it

@ahus1 ahus1 force-pushed the is-4900-remove-rejected-task-from-queue branch from 1ecd3bf to 7b3a47c Compare October 13, 2023 08:38
@ahus1 ahus1 force-pushed the is-4900-remove-rejected-task-from-queue branch from 7b3a47c to 2d74868 Compare October 13, 2023 08:44
@ahus1
Copy link
Contributor Author

ahus1 commented Oct 13, 2023

@vietj - thank you for the review. I've moved the statement, and also added a comment to state why this is the right place. Please give it another review when you have the time.

@ahus1 ahus1 requested a review from vietj October 13, 2023 08:46
@vietj vietj added this to the 5.0.0 milestone Oct 16, 2023
@vietj vietj merged commit f3c4482 into eclipse-vertx:master Oct 16, 2023
@vietj
Copy link
Member

vietj commented Oct 16, 2023

thanks for the contribution, do you mind porting this to the 4.x branch ?

@ahus1
Copy link
Contributor Author

ahus1 commented Oct 16, 2023

@vietj - thank you for reviewing and merging this PR. Here's the backport for this change to 4.x: #4904

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vietj vietj Awaiting requested review from vietj

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
5.0.0
Development

Successfully merging this pull request may close these issues.

TasksQueue doesn't remove entry from tasks on RejectedExecutionException
2 participants
@ahus1 @vietj