Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worldstate-only resync behavior #4875

Merged
merged 9 commits into from
Jan 12, 2023

Conversation

garyschulte
Copy link
Contributor

@garyschulte garyschulte commented Jan 3, 2023

PR description

Implements a debug rpc endpoint and a behavior to resync just the worldstate.

Fixed Issue(s)

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if
    updates are required.

Changelog

@garyschulte garyschulte force-pushed the feature/bonsai-resync-ws branch 2 times, most recently from 44a7cec to 7fd3377 Compare January 5, 2023 00:22
@garyschulte
Copy link
Contributor Author

@shemnon as far as UX for a resync feature, 'touching' a file in the data dir is rather simple to implement and accessible. A debug rpc endpoint may or may not be a good idea for this kind of functionality. What do you think about file vs rpc, file and rpc, or some other command pattern to trigger this behavior?

@shemnon
Copy link
Contributor

shemnon commented Jan 5, 2023

I'm not a devops maven, I'd think we should get the UX feedback from someone who manages fleets of nodes. Touching a file feels hacky to me but I've discovered lots of DevOps "best practices" feel hacky to me.

@garyschulte garyschulte force-pushed the feature/bonsai-resync-ws branch 4 times, most recently from 0a1ffae to 0346dad Compare January 10, 2023 00:13
@garyschulte garyschulte marked this pull request as ready for review January 10, 2023 04:13
@garyschulte garyschulte force-pushed the feature/bonsai-resync-ws branch from 39aa3a3 to bf76408 Compare January 10, 2023 04:14
@matkt
Copy link
Contributor

matkt commented Jan 10, 2023

I'm not a devops maven, I'd think we should get the UX feedback from someone who manages fleets of nodes. Touching a file feels hacky to me but I've discovered lots of DevOps "best practices" feel hacky to me.

I'm no devops but I personally prefer an RPC call or a flag to a file that feels DIY.
I would just say that the RPC call must be admin because it seems too critical to me to simply be in debug

Copy link
Contributor

@fab-10 fab-10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a method that seems unused

Comment on lines +52 to +57
public void reset() {
this.badBlocks.invalidateAll();
this.badHeaders.invalidateAll();
this.latestValidHashes.invalidateAll();
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this called?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@garyschulte garyschulte force-pushed the feature/bonsai-resync-ws branch 2 times, most recently from bf59581 to ee520db Compare January 11, 2023 21:55
@garyschulte garyschulte dismissed matkt’s stale review January 11, 2023 22:00

As discussed oob, we will use debug namespace

@garyschulte garyschulte force-pushed the feature/bonsai-resync-ws branch from ee520db to 8bb393e Compare January 11, 2023 22:26
@garyschulte garyschulte force-pushed the feature/bonsai-resync-ws branch from 8bb393e to d830f2a Compare January 12, 2023 04:17
@garyschulte garyschulte enabled auto-merge (squash) January 12, 2023 04:19
@garyschulte garyschulte merged commit 96972ff into hyperledger:main Jan 12, 2023
@garyschulte garyschulte deleted the feature/bonsai-resync-ws branch January 12, 2023 04:48
@garyschulte garyschulte restored the feature/bonsai-resync-ws branch January 17, 2023 16:20
siladu pushed a commit to jflo/besu that referenced this pull request Feb 7, 2023
* use debug rpc endpoint to resync worldstate
* Reset transaction pool state every time the initial sync is done

Signed-off-by: garyschulte <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
siladu pushed a commit to jflo/besu that referenced this pull request Feb 7, 2023
* use debug rpc endpoint to resync worldstate
* Reset transaction pool state every time the initial sync is done

Signed-off-by: garyschulte <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
@siladu siladu mentioned this pull request Feb 7, 2023
jflo added a commit that referenced this pull request Feb 8, 2023
* Fix transaction pool issue (#4964)
* Cache empty slots (#4874)
* clear after each block and copy during clone
* fix transaction pool issue
* add block replay
* support in-memory snapshots
* Keep Worldstate Storage open for Bonsai archive latest layer (#5039)
* bonsai layered worldstate subscription
* unsubscribe from worldstatestorage on close of BonsaiLayeredWorldState
* minor txpool logging improvements
* Avoid triggering a calculate root hash when empty slot cache is not empty.
* use the updater cache to get an account during block processing (#4897)
* Worldstate-only resync behavior (#4875)
* use debug rpc endpoint to resync worldstate
* Reset transaction pool state every time the initial sync is done
* init heal code



---------

Signed-off-by: Karim TAAM <[email protected]>
Signed-off-by: garyschulte <[email protected]>
Signed-off-by: Justin Florentine <[email protected]>
Signed-off-by: Ameziane H <[email protected]>
Signed-off-by: ahamlat <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Co-authored-by: matkt <[email protected]>
Co-authored-by: garyschulte <[email protected]>
Co-authored-by: ahamlat <[email protected]>
Co-authored-by: Simon Dudley <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
eum602 pushed a commit to lacchain/besu that referenced this pull request Nov 3, 2023
* use debug rpc endpoint to resync worldstate
* Reset transaction pool state every time the initial sync is done

Signed-off-by: garyschulte <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
@garyschulte garyschulte deleted the feature/bonsai-resync-ws branch January 24, 2024 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants