Unicode-preserving mutators #1542
Conversation
|
|
|
perhaps next step could be adding token's category info when add it to the dictionary? |
|
I will try this tomorrow. |
|
when I add a "real" utf8 chars to test I got lots of Utf8Error. |
|
Now I fixed the test so they return proper utf8error. but with the current state it already panick. I'll look tomorrow too |
|
Right -- I think it is not strictly possible to sanely select UTF-8 data from an input, because there's not a clear indicator for the start of a code point. This makes some mutations lead to non-UTF-8 data, which is less than optimal. Not sure how to get around this without a "this region is UTF-8" pass of some kind. That should be fairly cheap to implement, so maybe we can try this. |
addisoncrump
force-pushed
the
unicode-mutator
branch
from
1ab2620 to
01245c5
Compare
|
@tokatoka more CI failures... 😢 |
addisoncrump
force-pushed
the
unicode-mutator
branch
from
8575e8b to
080aef5
Compare
addisoncrump
force-pushed
the
unicode-mutator
branch
from
080aef5 to
ca2d26a
Compare
|
status? |
addisoncrump
force-pushed
the
unicode-mutator
branch
2 times, most recently
from
a41121d to
1e4b04c
Compare
addisoncrump
force-pushed
the
unicode-mutator
branch
from
1e4b04c to
93e2e2d
Compare
|
This is cool! |
* create the string classification stage * modify API to pre-group * preserving mutator * more meaningful test * subproperty mutators + some fixes * document, finalise, integrate with libafl_libfuzzer * add example, fix for weird range select * fix for introspection * fix fuzzer build * speed optimisation: allow, but do not require, stacking * property => category * token replacement * fixup: rare case where rust does not agree on valid character * fix CI again * again again * take two: dynamic unicode discovery * oops * fix: last byte is never selected * opt: bias to smaller unicode categories * fix test * opt: precompute regions and fix tests * cache and allow stacking * document and update libafl_libfuzzer * oops, use reverse * fix bolts clippy error * fixup part 2 * clippy * part 2 * clippy warning allow * clippy complaint * use alloc not std --------- Co-authored-by: toka <[email protected]>
This PR adds mutators which preserve the unicode categories of mutated regions.