Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fuzzgen: Add fcvt_* ops #4958

Merged
merged 1 commit into from
Sep 27, 2022
Merged

Conversation

afonso360
Copy link
Contributor

@afonso360 afonso360 commented Sep 25, 2022

👋 Hey,

Following up from #4884 (Thanks!) this PR introduces fcvt_* ops to the fuzzer. Additionally it also adds a pass to minimize the number of traps generated by these ops.

Like the int_divz pass, the number of traps generated doesn't seem to correspond with the amount requested in the config. I think I've actually figured out something here, if I limit the amount of bytes the fuzzer uses, the amount of traps is reduced by a lot. (I got 0.4% int_ovf with -max_len=2048)

That leads me to believe that we get such a high number of traps due to executing a lot of inputs. This is something that I want to look further into since I think there could actually be a performance benefit from limiting this.

As a sanity check I also tried to benchmark with 100% insertion rate, and as expected we don't get the other traps.

Benchmarks

Baseline

#92127  REDUCE cov: 30535 ft: 173903 corp: 4510/10465Kb lim: 393617 exec/s: 24 rss: 2448Mb L: 124/393581 MS: 1 EraseBytes-
== FuzzGen Statistics  ====================
Valid Inputs: 50000
Total Runs: 1469777
Successful Runs: 1415404 (96.3% of Total Runs)
Timed out Runs: 425 (0.0% of Total Runs)
Traps:
        user code: int_divz: 53948 (3.7% of Total Runs)
#92200  NEW    cov: 30535 ft: 173904 corp: 4511/10470Kb lim: 393617 exec/s: 23 rss: 2448Mb L: 4614/393581 MS: 3 ChangeASCIIInt-ChangeByte-CMP- DE: "\x1e\x00\x00\x00"-

This PR

I've seen the int_ovf trap count vary a lot while testing, from 0.4% up to almost 7%.

#86636  NEW    cov: 31377 ft: 178040 corp: 4651/8643Kb lim: 119720 exec/s: 25 rss: 2375Mb L: 4548/65539 MS: 2 ChangeASCIIInt-CopyPart-
== FuzzGen Statistics  ====================
Valid Inputs: 50000
Total Runs: 1184585
Successful Runs: 1099371 (92.8% of Total Runs)
Timed out Runs: 1066 (0.1% of Total Runs)
Traps:
        user code: int_ovf: 40522 (3.4% of Total Runs)
        user code: int_divz: 33112 (2.8% of Total Runs)
        user code: bad_toint: 10514 (0.9% of Total Runs)
#86713  REDUCE cov: 31377 ft: 178042 corp: 4652/8645Kb lim: 119720 exec/s: 25 rss: 2375Mb L: 1761/65539 MS: 2 InsertRepeatedBytes-InsertRepeatedBytes-

This PR while always applying the trap pass

#89715  REDUCE cov: 31208 ft: 177781 corp: 4618/8793Kb lim: 136654 exec/s: 25 rss: 2342Mb L: 2397/136620 MS: 1 PersAutoDict- DE: "\x00\x00\x00Y"-
== FuzzGen Statistics  ====================
Valid Inputs: 50000
Total Runs: 1315468
Successful Runs: 1273898 (96.8% of Total Runs)
Timed out Runs: 1016 (0.1% of Total Runs)
Traps:
        user code: int_divz: 40554 (3.1% of Total Runs)
#89889  NEW    cov: 31208 ft: 177782 corp: 4619/8808Kb lim: 136654 exec/s: 25 rss: 2342Mb L: 15765/136620 MS: 4 CopyPart-CopyPart-InsertByte-CrossOver-

cc: @jameysharp

@github-actions github-actions bot added the cranelift Issues related to the Cranelift code generator label Sep 25, 2022
@afonso360 afonso360 force-pushed the fuzzgen-fcvt branch 2 times, most recently from 2ee7cd7 to 7e271f6 Compare September 25, 2022 15:22
Copy link
Contributor

@jameysharp jameysharp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hooray, more opcodes!

@jameysharp jameysharp merged commit 10deb9b into bytecodealliance:main Sep 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift Issues related to the Cranelift code generator
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants