Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/evm: Add --bench flag for benchmarking #20330

Merged
merged 1 commit into from
Dec 18, 2019

Conversation

chfast
Copy link
Member

@chfast chfast commented Nov 19, 2019

The --bench flag uses the testing.B to execute the EVM bytecode many times and get the average exeuction time out of it.

@karalabe
Copy link
Member

A suggestion that would make this a lot better. Instead of simply running once and measuring it (unstable), lets use Go's built in benchmark runner (testing.B). You can specify how much time a run should take and it will iteratively increase the loop counts to your benchmark takes that long. The idea being, that if you try to benchmark something taking milliseconds, a single measurement is useless, but Go's benchmarker will hone in on 1000 iterations (1 second), making outlier execution times irrelevant.

@chfast
Copy link
Member Author

chfast commented Nov 20, 2019

Yes, I was going this direction. I was considering two options:

  1. Add --iterations N option and then run some many times.
  2. Using benchmarking from testing. As you suggested this one, I will pick it up. Should I add --bench flag for it?

@chfast chfast changed the title Add benchmarking capabilities to evm tool cmd/evm: Add --bench flag for benchmarking Nov 21, 2019
@chfast chfast marked this pull request as ready for review November 21, 2019 15:07
@chfast
Copy link
Member Author

chfast commented Nov 21, 2019

@karalabe, this is ready now for review.

The --bench flag uses the testing.B to execute the EVM bytecode many times and get the average exeuction time out of it.
Copy link
Contributor

@holiman holiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This generally LGTM, but could you provide (on the PR description) some sample/example output from running it -- so we get a feel for how to use it and how the output looks?

@chfast
Copy link
Member Author

chfast commented Nov 26, 2019

The output is unchanged, just the evm execution time is more accurate, and other stats are disturbed because the EVM execution is done multiple times.

With --bench:

build/bin/evm --codefile ~/Projects/ethereum/evmone/test/benchmarks/sha1_shifts.evm --statdump --input 1605782b00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000 --bench run
evm execution time: 473.312µs
heap objects:       9490
allocations:        3158024
total allocations:  145871912
GC calls:           63
Gas used:           35234

Without:

build/bin/evm --codefile ~/Projects/ethereum/evmone/test/benchmarks/sha1_shifts.evm --statdump --input 1605782b00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000 run        
evm execution time: 667.701µs
heap objects:       5664
allocations:        1525080
total allocations:  3207648
GC calls:           1
Gas used:           35234

@holiman holiman self-assigned this Dec 17, 2019
@holiman holiman merged commit 49cf000 into ethereum:master Dec 18, 2019
@chfast
Copy link
Member Author

chfast commented Dec 18, 2019

Oh, thanks :)

@axic axic deleted the evm_exec_time branch December 18, 2019 11:31
@axic axic restored the evm_exec_time branch December 18, 2019 11:32
@chfast chfast deleted the evm_exec_time branch April 13, 2020 13:29
enriquefynn pushed a commit to enriquefynn/go-ethereum that referenced this pull request Mar 10, 2021
The --bench flag uses the testing.B to execute the EVM bytecode many times and get the average exeuction time out of it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants