Testing under Miri

[kpreid]

I saw a mention of a fuzzing tool generating code that was then executed under Miri, in order to avoid the cost of compilation of small pieces of code. It occurred to me that cargo-mutants might benefit from the same technique, at least for programs where the total amount of code to compile is large, but the tests don't execute very many operations.

So, I tried a simple patch removing cargo build and replacing cargo test with cargo miri test and — it's slower in all the test cases I tried. The timings feel like miri test has a significant startup cost in loading the program and running each test, but I don't know if that's what is actually going on.

Just thought I'd report this failed idea in case anyone else was thinking of trying it, or if there's a way to make it better.

[sourcefrog]

Interesting idea!

I wonder if Miri might also allow useful mutations at the MIR level...? Although, you would still have to run the code, which might still be slower...

[kpreid]

I got the idea from Semantic fuzzing of the Rust compiler and interpreter — Master’s Thesis — Qian (Andy) Wang which is in fact mutating MIR, but its goal is to find compiler bugs. I haven't read it in detail (the interesting part was quoted elsewhere) so I don't know how feasible it is to set up as a convenient tool.

[sourcefrog]

I was reading https://notgull.net/cautionary-unsafe-tale/ and I thought it might be good to have an option to run Miri tests in support of mutating unsafe code...