Save LC Bootstrap DB Space Optimization #14775
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rkapka
reviewed
Jan 9, 2025
| @@ -35,9 +37,35 @@ func (s *Store) SaveLightClientBootstrap(ctx context.Context, blockRoot []byte, | |||
| _, span := trace.StartSpan(ctx, "BeaconDB.SaveLightClientBootstrap") | |||
| defer span.End() | |||
|
|
|||
| bootstrapCopy, err := light_client.NewWrappedBootstrap(proto.Clone(bootstrap.Proto())) | |||
There was a problem hiding this comment.
Since we are saving only the proto object, there is no need for light_client.NewWrappedBootstrap, you can set the sync committee to an empty one on the proto object directly.
rkapka
previously approved these changes
Jan 9, 2025
rkapka
approved these changes
Jan 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Feature
What does this PR do? Why is it needed?
This PR adds an optimization mechanism to the process of saving the
LightClientBootstraps. It does this by storing the uniqueSyncCommitteesin a separate bucket, and encoding the hash of the sync committee objects into the bootstrap objects while writing to DB.This results in saving around 20x space for storing these objects. quick calculation:
previously in one period: 8192 blocks * 25KB per bootstrap = 205MB
now in one period: 8192 * 1.2KB per bootstrap + 24KB per period (syncCommittee) = 10MB
Which issues(s) does this PR fix?
Additionally, part of #12991
Other notes for review
Encoding the hash of the sync committee objects was chosen over the period number to not burden the
kvpackage with the assumptions of the chain. If for any reason the sync committee changes mid period, this still works as expected.