Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sync): set max block if debug.tip provided #1522

Merged
merged 12 commits into from
Feb 23, 2023

Conversation

rkrasiuk
Copy link
Member

Closes #1510

Set Pipeline::max_block if debug.tip is provided. This leads to pipeline termination upon reaching the tip.

@rkrasiuk rkrasiuk added the A-cli Related to the reth CLI label Feb 23, 2023
@rkrasiuk rkrasiuk requested a review from mattsse February 23, 2023 08:26
@codecov-commenter
Copy link

codecov-commenter commented Feb 23, 2023

Codecov Report

Merging #1522 (ea2efff) into main (3c77dd9) will increase coverage by 0.00%.
The diff coverage is 3.65%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##             main    #1522   +/-   ##
=======================================
  Coverage   75.75%   75.75%           
=======================================
  Files         361      362    +1     
  Lines       42927    42956   +29     
=======================================
+ Hits        32518    32543   +25     
- Misses      10409    10413    +4     
Flag Coverage Δ
integration-tests 21.77% <0.00%> (-0.02%) ⬇️
unit-tests 70.23% <3.65%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
bin/reth/src/lib.rs 100.00% <ø> (ø)
bin/reth/src/node/mod.rs 2.53% <0.00%> (-0.25%) ⬇️
bin/reth/src/p2p/mod.rs 0.00% <0.00%> (ø)
bin/reth/src/utils.rs 0.00% <0.00%> (ø)
crates/stages/src/pipeline/mod.rs 86.37% <12.50%> (-1.11%) ⬇️
crates/stages/src/pipeline/ctrl.rs 100.00% <100.00%> (ø)
crates/stages/src/stages/hashing_storage.rs 95.65% <0.00%> (-0.28%) ⬇️
crates/net/discv4/src/lib.rs 66.08% <0.00%> (-0.15%) ⬇️
crates/net/network/src/peers/manager.rs 82.55% <0.00%> (+0.08%) ⬆️
crates/transaction-pool/src/pool/txpool.rs 61.56% <0.00%> (+0.49%) ⬆️
... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@rkrasiuk
Copy link
Member Author

rkrasiuk commented Feb 23, 2023

@mattsse this PR introduces a code path where we do a redundant download of the sync target. do u think it's worth changing SyncTarget::Tip(H256) to SyncTarget::Tip(H256, Option<BlockNumber>) and set it if possible?

@rkrasiuk rkrasiuk marked this pull request as ready for review February 23, 2023 09:35
@rkrasiuk rkrasiuk requested a review from onbjerg as a code owner February 23, 2023 09:35
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand all the changes included here yet

Comment on lines 220 to 226
let max_block = if let Some(block) = self.max_block {
Some(block)
} else if let Some(tip) = self.tip {
Some(self.fetch_or_lookup_tip(db.clone(), fetch_client, tip).await?)
} else {
None
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the motivation behind doing this here in the command.

this is a bit convoluted

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because this has to do with pipeline instantiation depending on cli args

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, thanks!

bin/reth/src/p2p/mod.rs Outdated Show resolved Hide resolved
bin/reth/src/node/mod.rs Outdated Show resolved Hide resolved
@@ -15,11 +15,13 @@ pub(crate) enum ControlFlow {
/// The progress of the last stage
progress: BlockNumber,
},
NoProgress,
NoProgress {
previous_progress: Option<BlockNumber>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the naming progress a bit confusing here, because this does not really convey what this precisely means.
is this the block number the previous stage is at? or the number of blocks the stage advanced/made progress? not really clear.

needs docs, can revise naming perhaps separately.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the last progress of the current stage. agree on the renaming, added the docs to clarify

bin/reth/src/utils.rs Outdated Show resolved Hide resolved
Comment on lines +369 to +380
debug!(target: "reth::cli", ?tip, "Fetching tip header from the network.");
loop {
match get_single_header(fetch_client.clone(), BlockHashOrNumber::Hash(tip)).await {
Ok(tip_header) => {
debug!(target: "reth::cli", ?tip, number = tip_header.number, "Successfully fetched tip");
return Ok(tip_header.number)
}
Err(error) => {
error!(target: "reth::cli", %error, "Failed to fetch the tip. Retrying...");
}
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to fetch this in the first place?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be able to instantiate the pipeline with the max_block parameter if debug.tip was provided. otherwise, the pipeline doesn't have a terminating condition

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah okay, understood, thanks for clarifying this in the docs.

Does this mean that even if we would continue after restarting in for example execution stage we need to wait for the first peer?

Copy link
Member Author

@rkrasiuk rkrasiuk Feb 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, because the tip was already stored in the db. First, I attempt the db lookup. If the tip is not there, then i fetch

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i should prob change the fn name from fetch_or_lookup_tip to lookup_or_fetch_tip to reflect the order of operations

@rkrasiuk rkrasiuk requested a review from mattsse February 23, 2023 10:25
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm,

pending question regarding restarts

@rkrasiuk rkrasiuk merged commit 22d20d5 into main Feb 23, 2023
@rkrasiuk rkrasiuk deleted the rkrasiuk/cli-tip-max-block branch February 23, 2023 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cli Related to the reth CLI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Infinite loop after reaching --debug.tip
3 participants