CQ shared store write optimisations #8507
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lhoguin
commented
Jun 9, 2023
This comment was marked as outdated.
This comment was marked as outdated.
Before they would only be sent periodically or when rolling over to a new file.
We know the messages are on disk or were acked so there is no need to do sets intersections/subtracts in this scenario.
Instead of having the message store send a message to the queue with the confirms for messages ignored due to the flying optimisation, we have the queue handle the confirms directly when removing the messages. This avoids sending potentially 1 Erlang message per 1 AMQP message to the queue.
Also make use of the opened file for multi-reads instead of opening/reading/closing each time.
The way I initially did this the maybe_gc would be triggered based on candidates from 15s ago, but run against candidates from just now. This is sub-optimal because when messages are consumed rapidly, just-now candidates are likely to be in a file about to be deleted, and we don't want to run compaction on those. Instead, when sending the maybe_gc we also send the candidates we had at the time. Then 15s later we check if the file still exists. If it's gone, great! No compaction to do.
|
This is ready for review/merge. |
|
Full test results: The biggest difference is in confirm latency, when using And when publishing to a long queue (no consumers): (the sudden drop in throughput is when the queues reach their |
mkuratczyk
approved these changes
Jun 20, 2023
|
So, a 40% drop in publisher confirm latency on one workload, and a 60% drop on another (both have publishers and consumers online). Not bad. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Another round of CQ shared message store optimisations and refactors, this time around writes.
The first commit is more a refactor than anything, although no longer calling fsync does help in some scenarios. The second commit should provide a noticeable boost in performance when there are no consumers.
@mkuratczyk I have renamed the branch.