Historic block pruning #4158
Open
Historic block pruning #4158
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michaelsproul
added
work-in-progress
|
Dropping this to v4.2.1, I haven't had a chance to implement the state deletion yet |
michaelsproul
added
v4.5.0
|
What are the blockers on this PR? |
|
@delbonis We need to implement pruning for historic states as well, something that drops all the existing historic states and prevents more from being stored would be the simplest. Are you interested in working on it? Otherwise I'm hoping to get to it soon-ish |
|
Opened a separate issue for the state pruning here: #4481 |
|
Dropping this from 4.6.0 as I don't have time |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Closes:
Builds on:
Proposed Changes
Add a step to the finalization migration which deletes blocks if they are older than the newly computed weak subjectivity checkpoint. To avoid burdening the disk this deletion is scheduled to occur once per week.
The block pruning is potentially I/O intensive, so to prevent it running in parallel with the I/O heavy finalization migration, it happens as part of finalization in the migrator. This required plumbing in the
BeaconChain.Limitations
If a node has been synced from genesis or has reconstructed states then block pruning is disabled. This is because we should also prune the historic states from disk in this case, and currently have no machinery in
unstablefor doing this (I had a branch, once upon a time). In the interests of simplicity this is deferred for future work.Although now that I think about it, maybe having states without blocks is OK... and we will trigger the case where the
oldest_block_slotis greater than thestate_upper_limiteven with this restriction (once a node prunes blocks newer than its oldest stored state). Need to think on this some more. Related: #3211.TODO