Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add commmand for pruning states #4835

Closed
wants to merge 19 commits into from

Conversation

jimmygchen
Copy link
Member

@jimmygchen jimmygchen commented Oct 12, 2023

Issue Addressed

Closes #4481.

(Continuation of #4648)

Proposed Changes

  • Add lighthouse db prune-states
  • Make it work
  • Ensure block roots are handled correctly (to be addressed in 4735)
  • Check perf on mainnet/Goerli/Gnosis (takes a few seconds max)
  • Run block root healing logic ([Merged by Bors] - Add block roots heal logic in v18 schema migration. #4875 ) at the beginning
  • Add some tests
  • Update docs
  • Add --freezer flag and other improvements to lighthouse db inspect

@jimmygchen jimmygchen changed the base branch from stable to unstable October 12, 2023 04:42
@jimmygchen jimmygchen mentioned this pull request Oct 12, 2023
7 tasks
bors bot pushed a commit that referenced this pull request Oct 25, 2023
## Issue Addressed

Fixes #4697. 

This also unblocks the state pruning PR (#4835). Because self healing breaks if state pruning is applied to a database with missing block roots.

## Proposed Changes

- Fill in the missing block roots between last restore point slot and split slot when upgrading to latest database version.
# Conflicts:
#	beacon_node/beacon_chain/tests/store_tests.rs
#	beacon_node/store/src/hot_cold_store.rs
#	beacon_node/store/src/lib.rs
#	database_manager/src/lib.rs
# Conflicts:
#	beacon_node/store/src/lib.rs
#	beacon_node/store/src/memory_store.rs
#	database_manager/src/lib.rs
@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Nov 1, 2023
@michaelsproul michaelsproul added the backwards-incompat Backwards-incompatible API change label Nov 2, 2023
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge!

database_manager/src/lib.rs Outdated Show resolved Hide resolved
@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed backwards-incompat Backwards-incompatible API change labels Nov 3, 2023
@michaelsproul
Copy link
Member

I added and removed a backwards-incompat label because I thought we'd broken the prune_payloads command. But the alias exists, so it's fine.

bors r+

bors bot pushed a commit that referenced this pull request Nov 3, 2023
## Issue Addressed

Closes #4481. 

(Continuation of #4648)

## Proposed Changes

- [x] Add `lighthouse db prune-states`
- [x] Make it work
- [x] Ensure block roots are handled correctly (to be addressed in 4735)
- [x] Check perf on mainnet/Goerli/Gnosis (takes a few seconds max)
- [x] Run block root healing logic (#4875 ) at the beginning
- [x] Add some tests
- [x] Update docs
- [x] Add `--freezer` flag and other improvements to `lighthouse db inspect`

Co-authored-by: Michael Sproul <[email protected]>
Co-authored-by: Jimmy Chen <[email protected]>
Co-authored-by: Michael Sproul <[email protected]>
@jimmygchen jimmygchen removed the ready-for-review The code is ready for review label Nov 3, 2023
Copy link

bors bot commented Nov 3, 2023

Pull request successfully merged into unstable.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot changed the title Add commmand for pruning states [Merged by Bors] - Add commmand for pruning states Nov 3, 2023
@bors bors bot closed this Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
database ready-for-merge This PR is ready to merge. v4.6.0 ETA Q1 2024
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add logic to prune all historic states
2 participants