trie: support proof generation from the iterator #16722
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm in a bit of a salvage mission here, trying to extract any useful standalone code from my bit rotted leaf-sync work. This PR adds support to the trie iterator to generate proofs of the leaf nodes it visits.
In theory this functionality could be fully covered by
trie.Prove(it.LeafKey())
. The reason that's not viable is becausetrie.Prove
traverses the path manually, pulling the data from the database. However, the iterator already has all that data available in-memory in its iteration stack. Supporting this functionality from the iterator allows us to do it almost freely, without the extra database overhead.The second question that begs answering is whether we could then replace
trie.Prove
with the iterator approachtrie.NewIterator(t.NodeIterator(key)).Prove()
? In theory both do the same thing, so a targeted iterator should be adequate enough to replace the simple prover. Unfortunatelytrie.Prove
also supports proving non-existence of nodes, whereas the iterator obviously cannot position itself on top of a non existing item to prove it.Based on the logic above, both ways of proving a leaf is useful, so the PR also expands the proof tests to cross-reference the two implementation between one another.
The original code that used this PR was my leaf-sync extension to fast sync, which retrieves trie leaves directly instead of subtries. If these trie leaves are larger than our packet allowance, we can't send the entire sub-trie, so we send partial leaves + the boundary leaf proofs. I'm not fully sure this logic will survive the new sync ideas, but proving an iterated trie node fast seems useful enough to merge in itself too.