Database cachednode callback walk

[holiman]

Replaces #20527 , this one does not contain all the other experimental fluff.

In the trie database, there are several operations that recursively walk children of nodes, and collect them into lists of hashes. This means that on the leaf level, all the upper levels hold N lists with hashes. So at root level, the list contains all leaf hashes, at level below, a list with 1/16th of the entries, and so on.

This PR changes it to walk the nodes with a callback instead. Needs to be benchmarked and tested IRL, of course.

before

goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/trie
BenchmarkDerefRootFixedSize/10-6                   61821             19229 ns/op           14609 B/op         38 allocs/op
BenchmarkDerefRootFixedSize/100-6                  20331             58074 ns/op           67856 B/op        142 allocs/op
BenchmarkDerefRootFixedSize/1K-6                    1717            636877 ns/op          696097 B/op       1369 allocs/op
BenchmarkDerefRootFixedSize/10K-6                    145           8177885 ns/op         7116049 B/op      13908 allocs/op
BenchmarkDerefRootFixedSize/100K-6                    13          85116617 ns/op        70853392 B/op     138395 allocs/op

after

goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/trie
BenchmarkDerefRootFixedSize/10-6                   76370             16321 ns/op             785 B/op         11 allocs/op
BenchmarkDerefRootFixedSize/100-6                  24958             47222 ns/op             784 B/op         11 allocs/op
BenchmarkDerefRootFixedSize/1K-6                    2838            419798 ns/op             784 B/op         11 allocs/op
BenchmarkDerefRootFixedSize/10K-6                    202           5425507 ns/op             784 B/op         11 allocs/op
BenchmarkDerefRootFixedSize/100K-6                    20          55343687 ns/op             784 B/op         11 allocs/op

[karalabe]

It only gathers the direct children of a node, not recursively the entire subtrie.

[holiman]

It only gathers the direct children of a node, not recursively the entire subtrie.

You are correct. I suspect the problem is that we did a children := make([]common.Hash, 0, 16) at every junction, even though many don't even have children, let alone 16. So it causes a lot of tiny mallocs that were not even used.

https://github.com/ethereum/go-ethereum/pull/20529/files#diff-e49891341ce9d49a244d9ead8df3d072L186

[holiman]

Apparently this code doesn't work, though. It changes the semantics, as it exceutes the callback immediately, instead of collecting the children first, so it breaks things.

A bug was in there, fixed now hopefully

[holiman]

Giving this a shot on the benchmarkers, replacing the current run (https://geth-bench.ethdevops.io/d/Jpk-Be5Wk/dual-geth?orgId=1&var-exp=mon08&var-master=mon09&var-percentile=50&from=1578482122263&to=now ) without restarting them. The experimental (mon08) was at 5.021M when it was replaced.

[holiman]

Ah, immediate crash on the gpo thingy.

[holiman]

This actually does seem to make a difference. After running for a couple of hours, I checked the same section of blocks for both of these.
First is the mon09 (master) between 5.1M - 5.2M:

Although absolute time-comparisons between machines is a bit flakey, we can see that commit averages 13.16ms, above account commit at 11.62ms.

For this pr, mon08, this is the corresponding chart:

In this chart, account commit is at 11.37ms (which is very close, even in absolute numbers, to 11.62ms), whereas commit has dropped to third place at 11.13ms.

TLDR; this PR should (if anything) improve commit, and fwict it does cut down the average commit time from ~13.16ms to ~11.13ms in the section of blocks I checked.

[karalabe]

LGTM