Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use fastest non-master go-verkle version & pull trie/Verkle.go changes to use new api #184

Merged
merged 2 commits into from
Mar 19, 2023
+17 −44
Merged

use fastest non-master go-verkle version & pull trie/Verkle.go changes to use new api #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ef759cd
mod: update to fastest go-verkle version today
jsign Mar 17, 2023
31a8c29
trie/verkle: use new batch serialization api
jsign Mar 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ require (
github.com/fjl/gencodec v0.0.0-20220412091415-8bb9e558978c
github.com/fjl/memsize v0.0.0-20190710130421-bcb5799ab5e5
github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff
github.com/gballet/go-verkle v0.0.0-20230314090848-2b9d2a2f7467
github.com/gballet/go-verkle v0.0.0-20230317174103-141354da6b11
github.com/go-stack/stack v1.8.0
github.com/golang-jwt/jwt/v4 v4.3.0
github.com/golang/protobuf v1.5.2
Expand Down
18 changes: 2 additions & 16 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,6 @@ github.com/consensys/gnark-crypto v0.4.1-0.20210426202927-39ac3d4b3f1f/go.mod h1
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/cpuguy83/go-md2man/v2 v2.0.2 h1:p1EgwI/C7NhT0JmVkwCD2ZBK8j4aeHQX2pMHHBfMQ6w=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/crate-crypto/go-ipa v0.0.0-20221111143132-9aa5d42120bc h1:mtR7MuscVeP/s0/ERWA2uSr5QOrRYy1pdvZqG1USfXI=
github.com/crate-crypto/go-ipa v0.0.0-20221111143132-9aa5d42120bc/go.mod h1:gFnFS95y8HstDP6P9pPwzrxOOC5TRDkwbM+ao15ChAI=
github.com/crate-crypto/go-ipa v0.0.0-20230202201618-2e6f5bfc5401 h1:TSXRL74LZ7R2xWOI1M0mz9E56PiPKGlSw0drgR8g7CE=
github.com/crate-crypto/go-ipa v0.0.0-20230202201618-2e6f5bfc5401/go.mod h1:gFnFS95y8HstDP6P9pPwzrxOOC5TRDkwbM+ao15ChAI=
github.com/crate-crypto/go-ipa v0.0.0-20230315201338-1643fdc2ead8 h1:2EBbIwPDRqlCD2K34Eojyy0x9d3RhOuHAZfbQm508X8=
github.com/crate-crypto/go-ipa v0.0.0-20230315201338-1643fdc2ead8/go.mod h1:gzbVz57IDJgQ9rLQwfSk696JGWof8ftznEL9GoAv3NI=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
Expand Down Expand Up @@ -139,14 +135,8 @@ github.com/garslo/gogen v0.0.0-20170306192744-1d203ffc1f61 h1:IZqZOB2fydHte3kUgx
github.com/garslo/gogen v0.0.0-20170306192744-1d203ffc1f61/go.mod h1:Q0X6pkwTILDlzrGEckF6HKjXe48EgsY/l7K7vhY4MW8=
github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff h1:tY80oXqGNY4FhTFhk+o9oFHGINQ/+vhlm8HFzi6znCI=
github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff/go.mod h1:x7DCsMOv1taUwEWCzT4cmDeAkigA5/QCwUodaVOe8Ww=
github.com/gballet/go-verkle v0.0.0-20221122140954-75ceda26b7db h1:YvtZfE11QEYWPjsQCyZLoZCGMsxJs9mTEbhF3MnM32Q=
github.com/gballet/go-verkle v0.0.0-20221122140954-75ceda26b7db/go.mod h1:DMDd04jjQgdynaAwbEgiRERIGpC8fDjx0+y06an7Psg=
github.com/gballet/go-verkle v0.0.0-20221129125207-513116151b28 h1:UbB7D2R1OQCkNFX+LYoo2pHZ0u5LhwR9ldUsY4ZbZqI=
github.com/gballet/go-verkle v0.0.0-20221129125207-513116151b28/go.mod h1:DMDd04jjQgdynaAwbEgiRERIGpC8fDjx0+y06an7Psg=
github.com/gballet/go-verkle v0.0.0-20230303104313-a4243d1136b3 h1:UfRrJQF6ohVQGdi3MTU/8dPnayaIR2RuulZ55gUYXIE=
github.com/gballet/go-verkle v0.0.0-20230303104313-a4243d1136b3/go.mod h1:NR+n/LUx+m5SyVTRObiuNdJ50q309MGPD0VrIMDZJuc=
github.com/gballet/go-verkle v0.0.0-20230314090848-2b9d2a2f7467 h1:LkJAMgQtzKrWIr8m513poIhG2eBC1OZqIeziw9jkXPI=
github.com/gballet/go-verkle v0.0.0-20230314090848-2b9d2a2f7467/go.mod h1:NR+n/LUx+m5SyVTRObiuNdJ50q309MGPD0VrIMDZJuc=
github.com/gballet/go-verkle v0.0.0-20230317174103-141354da6b11 h1:x4hiQFgr1SlqR4IoAZiXLFZK4L7KbibqkORqa1fwKp8=
github.com/gballet/go-verkle v0.0.0-20230317174103-141354da6b11/go.mod h1:IyOnn1kujMWaT+wet/6Ix1BtvYwateOBy9puuWH/8sw=
github.com/getkin/kin-openapi v0.53.0/go.mod h1:7Yn5whZr5kJi6t+kShccXS8ae1APpYTW6yheSwk8Yi4=
github.com/getkin/kin-openapi v0.61.0/go.mod h1:7Yn5whZr5kJi6t+kShccXS8ae1APpYTW6yheSwk8Yi4=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
Expand Down Expand Up @@ -520,7 +510,6 @@ golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
Expand Down Expand Up @@ -564,10 +553,7 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211020174200-9d6173849985/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220919091848-fb04ddd9f9c8 h1:h+EGohizhe9XlX18rfpa8k8RAc5XyaeamM+0VHRd4lc=
golang.org/x/sys v0.0.0-20220919091848-fb04ddd9f9c8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0 h1:ljd4t30dBnAvMZaQCevtY0xLLD0A+bRZXbgLMLU1F/A=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0 h1:MVltZSvRTcU2ljQOhs94SXPftV6DCNnZViHeQps87pQ=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
Expand Down
41 changes: 14 additions & 27 deletions trie/verkle.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,9 @@ func (t *VerkleTrie) TryUpdateAccount(key []byte, acc *types.StateAccount) error
}

func (trie *VerkleTrie) TryUpdateStem(key []byte, values [][]byte) error {
resolver :=
func(h []byte) ([]byte, error) {
return trie.db.diskdb.Get(h)
}
resolver := func(h []byte) ([]byte, error) {
return trie.db.diskdb.Get(h)
}
switch root := trie.root.(type) {
case *verkle.InternalNode:
return root.InsertStem(key, values, resolver)
Expand Down Expand Up @@ -184,7 +183,6 @@ func (t *VerkleTrie) TryDeleteAccount(key []byte) error {
)

for i := 0; i < verkle.NodeWidth; i++ {

values[i] = zero[:]
}

Expand Down Expand Up @@ -230,33 +228,22 @@ func nodeToDBKey(n verkle.VerkleNode) []byte {
// Commit writes all nodes to the trie's memory database, tracking the internal
// and external (for account tries) references.
func (trie *VerkleTrie) Commit(_ bool) (common.Hash, *NodeSet, error) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed also that in the current replay block branch we had some onLeaf callback here, which isn't present.
I honored this version; but just mentioning if that's relevant.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's only used for stuff we don't do in verkle, so that's why it's not called. But that might change with the new DB schema, we will see very soon.

flush := make(chan verkle.VerkleNode)
resolver := func(n verkle.VerkleNode) {
flush <- n
root, ok := trie.root.(*verkle.InternalNode)
if !ok {
return common.Hash{}, nil, errors.New("unexpected root node type")
}
nodes, err := root.BatchSerialize()
if err != nil {
return common.Hash{}, nil, fmt.Errorf("serializing tree nodes: %s", err)
}
go func() {
switch root := trie.root.(type) {
case *verkle.InternalNode:
root.Flush(resolver)
case *verkle.StatelessNode:
root.Flush(resolver)
}
close(flush)
}()
var commitCount int
for n := range flush {
commitCount += 1
value, err := n.Serialize()
if err != nil {
panic(err)
}

if err := trie.db.diskdb.Put(nodeToDBKey(n), value); err != nil {
return common.Hash{}, NewNodeSet(common.Hash{}), err
for _, node := range nodes {
if err := trie.db.diskdb.Put(node.CommitmentBytes[:], node.SerializedBytes); err != nil {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the DiskDB() API was removed. Not sure why, but I think that was a good change... is an open pandora box to let "anyone" jump on top of the trie layer. (Maybe was done for that reason).

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's part of the preparations for this new database schema, iirc that's more or less what you assumed.

return common.Hash{}, nil, fmt.Errorf("put node to disk: %s", err)
}
}

return trie.Hash(), NewNodeSet(common.Hash{}), nil
return nodes[0].CommitmentBytes, NewNodeSet(common.Hash{}), nil
}

// NodeIterator returns an iterator that returns nodes of the trie. Iteration
Expand Down