Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance when promoting transaction from next layers #5920

Merged
merged 7 commits into from
Sep 27, 2023

Conversation

fab-10
Copy link
Contributor

@fab-10 fab-10 commented Sep 21, 2023

PR description

During the implementation of #5921, a performance issue surfaced in the way the promotion of transactions, from lower layers, was implemented.
The issue was that, to keep the code simple, the promotion was done for one tx at once, but with the improvement on the prioritized layer done #5921, this approach is no more practical since result in a quadratic complexity (number of confirmed txs per number of senders in the ready layer), so the solution is to do the promotion only once after all the confirmed txs have been processed, so the time is linear.

@github-actions
Copy link

github-actions bot commented Sep 21, 2023

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.
  • I thought about the changelog and included a changelog update if required.
  • If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

@fab-10 fab-10 marked this pull request as ready for review September 22, 2023 14:48
@fab-10 fab-10 force-pushed the txpool-promotion-performance branch from e289dbd to f97d749 Compare September 22, 2023 15:56
@fab-10 fab-10 force-pushed the txpool-promotion-performance branch from f97d749 to 17220d8 Compare September 25, 2023 12:29
final Predicate<PendingTransaction> promotionFilter,
final long freeSpace,
final int freeSlots) {
long accSpace = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usedSpace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allocatedSpace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here acc stays for accumulated space, to keep how much space has been accumulated by the selected txs till now, do you think it is better to rename it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accumulated also works

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renaming to accumulatedSpace

Co-authored-by: Sally MacFarlane <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
for (final var candidateTx : senderTxs.values()) {
if (promotionFilter.test(candidateTx)) {
accSpace += candidateTx.memorySize();
if (promotedTxs.size() < freeSlots && accSpace <= freeSpace) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a comment to explain why < for slots and <= for space

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can I ask what you get from the code about the difference? to understand what needs clarification

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

freeSlots and freeSpace have similar names - how much is free - so why does the if statement check < for one and <= for the other?

why not <= for freeSlots?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slots always increment by 1, while for space it could be any amount, so you can read it like that

if (promotedTxs.size() + 1 <= freeSlots && accumulatedSpace <= freeSpace)

but the +1 is redundant knowing that promotedTxs.size() can only increment by 1 at time, and we can shortcut with <

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right so it's because the accumulatedSpace is already incremented at this point but for the slots it happens after. makes sense. it's ok for me :)

final Predicate<PendingTransaction> promotionFilter,
final long freeSpace,
final int freeSlots) {
long accSpace = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allocatedSpace?

Copy link
Contributor

@macfarla macfarla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense!

@fab-10 fab-10 enabled auto-merge (squash) September 27, 2023 08:55
@fab-10 fab-10 merged commit 5d344ad into hyperledger:main Sep 27, 2023
@fab-10 fab-10 deleted the txpool-promotion-performance branch September 27, 2023 10:41
jflo pushed a commit to jflo/besu that referenced this pull request Nov 10, 2023
…rledger#5920)

Signed-off-by: Fabio Di Fabio <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
Signed-off-by: Justin Florentine <[email protected]>
NickSneo pushed a commit to NickSneo/besu that referenced this pull request Nov 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants