Pruner should have time-dependent deletion limits

[shekhirin]

Problem

Pruner has a setting for a number of entries it can delete from the database per block

reth/crates/prune/src/pruner.rs

Lines 35 to 36 in 0ddb753

	/// Maximum total entries to prune (delete from database) per block.
	delete_limit: usize,

We need that limit because only one database write transaction can be opened at the same time, and there's a fight for this transaction between the main Engine loop (insert new data into the database) and the Pruner (delete data from the database).

In theory, if everyone has the same hardware setup, we can find a limit that satisfies this important condition: Pruner should finish before we need to process a Forkchoice Update in the Engine loop. But in practice, it's hard and on lower-end hardware we may go beyond the allowed threshold and miss or delay an FCU.

In most of the cases, it's not a problem because the limits are set higher than the actual number of entries that's produced per block, so the Pruner finishes earlier than 100ms. BUT if we have any I/O slowdown or a huge block, we can easily go over this 100ms limit.

Solution

Current

Currently, we set the deletion limit for pruner manually, expressed in number of entries per block:

reth/crates/primitives/src/chain/spec.rs

Lines 626 to 630 in 0ddb753

	/// The delete limit for pruner, per block. In the actual pruner run it will be multiplied by
	/// the amount of blocks between pruner runs to account for the difference in amount of new
	/// data coming in.
	#[serde(default)]
	pub prune_delete_limit: usize,

For Mainnet, this number is higher because more data is produced on every block

reth/crates/primitives/src/chain/spec.rs

Line 69 in 0ddb753

prune_delete_limit: 3500,

And for testnets, it's lower

reth/crates/primitives/src/chain/spec.rs

Line 113 in 0ddb753

prune_delete_limit: 1700,

Proposal

Make the pruner limit time-dependent, i.e. allow a maximum of 100ms for the Pruner to delete any data, and stop after that and commit everything to the database, returning the control over an exclusive write transaction to the Engine loop.

That way, we will also not need any chain-dependent settings in the ChainSpec.