You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Related to #94, which is about classifying generated values and showing stats about those classes.
It might be nice to show stats unrelated to classifications too: for example, here's a doc on how Hypothesis does things: there, if you run with cmdline argument --hypothesis-show-statistics, you'll see:
- during generate phase (0.06 seconds):
- Typical runtimes: < 1ms, ~ 47% in data generation
- 100 passing examples, 0 failing examples, 0 invalid examples
- Stopped because settings.max_examples=100
or, if you use something that throws out a value with invalid (eg. filter, andThen, ...),
test_even_integers:
- during generate phase (0.08 seconds):
- Typical runtimes: < 1ms, ~ 57% in data generation
- 100 passing examples, 0 failing examples, 12 invalid examples
- Events:
* 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
* 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
- Stopped because settings.max_examples=100
or, if you use an event function to classify a generated value,
test_even_integers:
- during generate phase (0.09 seconds):
- Typical runtimes: < 1ms, ~ 59% in data generation
- 100 passing examples, 0 failing examples, 32 invalid examples
- Events:
* 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
* 31.06%, i mod 3 = 2
* 28.79%, i mod 3 = 0
* 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
* 15.91%, i mod 3 = 1
- Stopped because settings.max_examples=100
The text was updated successfully, but these errors were encountered:
Janiczek
changed the title
Optionally report stats about fuzz tests (generation, shrinking)
Optionally report stats about fuzz tests (generation, shrinking, rejections)
Jun 23, 2022
Related to #94, which is about classifying generated values and showing stats about those classes.
It might be nice to show stats unrelated to classifications too: for example, here's a doc on how Hypothesis does things: there, if you run with cmdline argument
--hypothesis-show-statistics
, you'll see:or, if you use something that throws out a value with
invalid
(eg.filter
,andThen
, ...),or, if you use an
event
function to classify a generated value,The text was updated successfully, but these errors were encountered: