Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2.1: Reclaims more old accounts in clean (backport of #4044) #4090

Merged
merged 2 commits into from
Dec 18, 2024

Conversation

mergify[bot]
Copy link

@mergify mergify bot commented Dec 12, 2024

Problem

In clean_accounts(), if there are old storages and old accounts beyond what is expected, we do not always reclaim them. This is because when scanning the candidates during clean, if the account's latest slot in the slot list is not in the uncleaned_roots list, we don't collect reclaims.

This can happen if there are candidates from old/ancient slots, where their newest entry is in a slot that is not being visited by clean. We would find multiple entries for this candidate in the index, yet determine not to collect reclaims.

Summary of Changes

Collect reclaims on candidates that have slot lists greater than one entry, and if they are all cleanable (i.e. rooted).

Justification to Backport

Without this change, nodes running with --disable-accounts-disk-index will collect excessive storages, because startup index generation does not correctly mark duplicate pubkeys when the disk index is disabled. But now clean does go and inspect those old storages, yet it won't reclaim the old versions of the accounts because the newest version of the account is not in the uncleaned_roots list. And because we don't reclaim these old versions, we cannot purge old storages that are all dead.

(Historically the uncleaned_roots were all the most recent slots, so it made sense to check for the newest versions there. Now, we clean really old slots too, which means the newest version of accounts in these old slots may be elsewhere (i.e. not just in uncleaned_roots))


This is an automatic backport of pull request #4044 done by [Mergify](https://mergify.com).

(cherry picked from commit 3d43824)

# Conflicts:
#	accounts-db/src/accounts_db.rs
#	accounts-db/src/accounts_db/tests.rs
@mergify mergify bot requested a review from a team as a code owner December 12, 2024 19:57
@mergify mergify bot added the conflicts label Dec 12, 2024
Copy link
Author

mergify bot commented Dec 12, 2024

Cherry-pick of 3d43824 has failed:

On branch mergify/bp/v2.1/pr-4044
Your branch is up to date with 'origin/v2.1'.

You are currently cherry-picking commit 3d43824154.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   accounts-db/src/accounts_index.rs

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   accounts-db/src/accounts_db.rs
	deleted by us:   accounts-db/src/accounts_db/tests.rs

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

Copy link

@HaoranYi HaoranYi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.
One question, are those test code necessary for backporting?

@brooksprumo
Copy link

brooksprumo commented Dec 13, 2024

One question, are those test code necessary for backporting?

The tests aren't necessary for the logic changes, but seems like a good idea to also keep the tests for backports in general, right?

@brooksprumo brooksprumo requested a review from HaoranYi December 13, 2024 20:27
@brooksprumo brooksprumo merged commit a3f55f6 into v2.1 Dec 18, 2024
28 checks passed
@brooksprumo brooksprumo deleted the mergify/bp/v2.1/pr-4044 branch December 18, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants