Add Intel PT tracing support #2471

Marcondiro · 2024-08-02T15:25:21Z

WIP, any feedback is welcome

libafl_qemu/build_linux.rs

addisoncrump · 2024-08-04T22:24:14Z

I'll rebase on this before making libafl_qemu changes in #2438.

domenukk · 2024-08-05T16:17:07Z

What's the benefit of the perf-event-open-sys crate thingy over bindgen? Just double checking / asking stupid questions :)

Marcondiro · 2024-08-05T17:00:08Z

No worries :) It has a couple of syscall/ioctl wrappers already available and it addresses an issue I had with Bindgen and `#define`s. They solved it already in their `wrapper.h` adding an enum. Let me know if you want me to go deeper than this answer :D

domenukk · 2024-08-05T17:05:08Z

Nah sounds good, it's just always good to double-check if new crates are necessary. If it solves issues let's gooo

rmalmain

i propose to start building the Module that would go on top of it once the inte_pt low-level functions look good enough to you.
even if some hooks are missing atm, it may be useful to start building the skeleton to help familiarize with the module system.

libafl_derive/src/lib.rs

libafl_qemu/src/qemu/systemmode/intel_pt.rs

rmalmain · 2024-08-05T21:09:59Z

libafl_qemu/src/qemu/systemmode/intel_pt.rs

+}
+
+#[inline]
+const fn next_page_aligned_addr(address: usize) -> usize {


we should avoid writing this kind of function here i think, we already have libafl_page_from_addr in the qemu repo in C. we should maybe start gathering these utils functions at the same place either in a c (so that it's usable in QEMU as well) or a rs file.

you're linking them together, anyway, right? So you can write it in rust and still use them from C

Do you guys know if we lose inlining this way?

from rust to rust? No. From C to rust, probably not? maybe?

libafl_qemu/Cargo.toml

…ocks map to 0

.github/workflows/build_and_test.yml

domenukk · 2024-11-08T18:43:32Z

libafl_intelpt/Cargo.toml

+
+[dependencies]
+arbitrary-int = { version = "1.2.7" }
+bitbybit = { version = "1.3.2" }


This comment still stands, we should try to keep the amount of dependencies as low as possible

domenukk · 2024-11-09T18:21:34Z

libafl_intelpt/src/lib.rs

+    }?;
+    let config = PtConfig::builder()
+        .with_noretcomp(true)
+        .with_psb_period(u4::new(0))


Also here there's the question if this can't be done with that bitfield trait instead of grabbing yet another dep? Just asking out loud :)

I'm not sure I've understood :D
If you mean: do we really need arbitrary-int?
Then yes, the bitbybit crate is based on it and has it as a dependency anyway...

Yeah that was the question. Thanks

So the question remains:: what's better about bitbybit? And can you change the other bit fields to it if you really want it (I don't care either way but don't want multiple crates for the same thing)

I see, for the moment I changed the code using it and commented out the dependency.
It was used to just build a constant so it's not a big deal. I will introduce it in the future with a new PR changing also the c2rust-bitfields.
(In the future the PtConfig bitfield struct will potentially support more options)

domenukk · 2024-11-11T18:46:29Z

Also, any chance we can merge this today or tomorrow for 0.14?

Marcondiro · 2024-11-12T10:01:03Z

@domenukk I think I've addressed every comment

rmalmain · 2024-11-12T10:17:48Z

ok on my side. let's merge @domenukk ?

domenukk · 2024-11-13T01:33:47Z

libafl_intelpt/Cargo.toml

+
+[dependencies]
+#arbitrary-int = { version = "1.2.7" }
+#bitbybit = { version = "1.3.2" }


Still survived here somehow? :)
We can remove it later, or bring it back.
Again: you're very welcome to use bitbybit instead of the other bitfield thingy! Just please change the current occurances (I think it's only one or two) with the new crate then, so we don't have redundant crates

I might be wrong but, maybe you missed my previous comment? :)

I see, for the moment I changed the code using it and commented out the dependency.
It was used to just build a constant so it's not a big deal. I will introduce it in the future with a new PR changing also the c2rust-bitfields.
(In the future the PtConfig bitfield struct will potentially support more options)

domenukk · 2024-11-13T01:34:39Z

Again to the bitbybit, I am not against it at all, but we shoudn't have two dependencies for the same use case - so we need to change the other usage to bitbybit

domenukk · 2024-11-13T01:35:28Z

Great work btw! :)

* WIP: IntelPT qemu systemmode * use perf-event-open-sys instead of bindgen * intelPT Add enable and disable tracing, add test * Use static_assertions crate * Fix volatiles, finish test * Add Intel PT availability check * Use LibAFL errors in Result * Improve filtering * Add KVM pt_mode check * move static_assertions use * Check for perf_event_open support * Add (empty) IntelPT module * Add IntelPTModule POC * partial ideas to implement intel pt * forgot smth * trace decoding draft * add libipt decoder * use cpuid instead of reading /proc/cpuinfo * investigating nondeterministic behaviour * intel_pt module add thread creation hook * Fully identify deps versions Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates * Move mem image to module, output to file for debug * fixup! Use static_assertions crate * Exclude host kernel from traces * Bump libipt-rs * Callback to get memory as an alterantive to image * WIP Add bootloader fuzzer example * Split availability check: add availability_with_qemu * Move IntelPT to observer * Improve test docs * Clippy happy now * Taplo happy now * Add IntelPTObserver boilerplate * Hook instead of Observer * Clippy & Taplo * Add psb_freq setting * Extremely bad and dirty babyfuzzer stealing * Use thread local cell instead of mutex * Try a trace diff based naive feedback * fix perf aux buffer wrap handling * Use f64 for feedback score * Fix clippy for cargo test * Add config format tests * WIP intelpt babyfuzzer with fork * Fix not wrapped tail offset in split buffer * Baby PT with raw traces diff working * Cache nr_filters * Use Lazy_lock for perf_type * Add baby_fuzzer_intel_pt * restore baby fuzzer * baby_fuzzer with block decoder * instruction decoder instead of block * Fix after upstream merge * OwnedRefMut instead of Cow * Read mem directly instead of going through files * Fix cache lifetime and tail update * clippy * Taplo * Compile caps only on linux * clippy * Fail compilation on unsupported OSes * Add baby_fuzzer_intel_pt to CI * Cleanup * Move intel pt + linux check * fix baby pt * rollback forkexecutor * Remove unused dep * Cleanup * Lints * Compute an edge id instead of using only block ip * Binary only intelPT POC * put linux specific code behind target_os=linux * Clippy & Taplo * fix CI * Disable relocation * No unwrap in decode * No expect in decode * Better logging, smaller aux buffer * add IntelPTBuilder * some lints * Add exclude_hv config * Per CPU tracing and inheritance * Parametrize buffer size * Try not to break commandExecutor API pt.1 * Try not to break commandExecutor API pt.2 * Try not to break commandExecutor API pt.3 * fix baby PT * Support on_crash & on_timeout callbacks for libafl_qemu modules (AFLplusplus#2620) * support (unsafe) on_crash / on_timeout callbacks for modules * use libc types in bindgen * Move common code to bolts * Cleanup * Revert changes to backtrace_baby_fuzzers/command_executor * Move intel_pt in one file * Use workspace deps * add nr_addr_filter fallback * Cleaning * Improve decode * Clippy * Improve errors and docs * Impl from<PtError> for libafl::Error * Merge hooks * Docs * Clean command executor * fix baby PT * fix baby PT warnings * decoder fills the map with no vec alloc * WIP command executor intel PT * filter_map() instead of filter().map() * fix docs * fix windows? * Baby lints * Small cleanings * Use personality to disable ASLR at runtime * Fix nix dep * Use prc-maps in babyfuzzer * working ET_DYN elf * Cleanup Cargo.toml * Clean command executor * introduce PtraceCommandConfigurator * Fix clippy & taplo * input via stdin * libipt as workspace dep * Check kernel version * support Arg input location * Reorder stuff * File input * timeout support for PtraceExec * Lints * Move out method not needing self form IntelPT * unimplemented * Lints * Move intel_pt_baby_fuzzer * Move intel_pt_command_executor * Document the need for smp_rmb * Better comment * Readme and Makefile.toml instead of build.rs * Move out from libafl_bolts to libafl_intelpt * Fix hooks * (Almost) fix intel_pt command exec * fix intel_pt command exec debug * Fix baby_fuzzer * &raw over addr_of! * cfg(target_os = "linux") * bolts Cargo.toml leftover * minimum wage README.md * extract join_split_trace from decode * extract decode_block from decode * add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0 * More generic hook * fix windows * Update CI, fmt * No bitbybit * Fix docker? * Fix Apple silicon? * Use old libipt from crates.io --------- Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Dominik Maier <[email protected]>

* launcher now uses client_id instead of core_id * adding overcommit to an example fuzzer * Replace addr_of with &raw across the codebase (#2669) * Replace addr_of with &raw across the codebase * fix fixes * more fix * undo clang fmt? * oops * fix? * allocator fix * more fix * more more * more docs * more fix * mas mas mas * hm * more * fix Frida * needed * more error * qemu * Introduce workspace (again) (#2673) * Trying to redo workspace deps again after #2672 * unused * clippy * Replace addr_of with &raw across the codebase (#2669) * Replace addr_of with &raw across the codebase * fix fixes * more fix * undo clang fmt? * oops * fix? * allocator fix * more fix * more more * more docs * more fix * mas mas mas * hm * more * fix Frida * needed * more error * qemu * Introduce workspace (again) (#2673) * Trying to redo workspace deps again after #2672 * unused * clippy * fixing formatting issues * cloning values to make borrow checker happy * simplifying cfg constraints, removing excessive clippy allows * printing clang version that is used to find inconsistencies between CI and local formatting * some fixes according to the CI * Specifying types * improved logging for formatter * more attempts at logging for the CI formatting * fixing setting LLVM version in formatting in CI * fixing cippy allows * renaming launcher's ClientID to ClientDescription * Lower capped RAND generators (#2671) * Lower capped rand generators * Updated all references to RAND generators * Formatting updates * New RAND bytes generator constructor * Revert "Updated all references to RAND generators" This reverts commit 9daad89. * Revert "Formatting updates" This reverts commit ff2a61a. * cargo nightly format * Added must_use to with_min_size * fix error '#' is not followed by a macro parameter (#2678) * Use version.workspace (#2682) * LibAFL_QEMU: Don't return a generic Address from Register reads (#2681) * LibAFL_QEMU: Make ReadReg always return GuestReg type * Don't return a generic address * fix fuzzers * fix mips * Add DrCovReader to read DrCov files and DrCov dumper and merge utils (#2680) * Add DrCov Reader * Removed libafl_jumper deps * Fix DrCovWriter, add dump_drcov_addrs * Taplo * Move frida from usize to u64 * DrCov usize=>u64 * Better error print * More u64 * ? * debug * clippy * clippy * Add Merge option to DrCovReader * Add drcov_merge tool * Move folder around * DrCov * More assert * fmt * Move around * Fix print * Add option to read multiple files/full folders * Fix build_all_fuzzers.sh for local runs (#2686) * Add Intel PT tracing support (#2471) * WIP: IntelPT qemu systemmode * use perf-event-open-sys instead of bindgen * intelPT Add enable and disable tracing, add test * Use static_assertions crate * Fix volatiles, finish test * Add Intel PT availability check * Use LibAFL errors in Result * Improve filtering * Add KVM pt_mode check * move static_assertions use * Check for perf_event_open support * Add (empty) IntelPT module * Add IntelPTModule POC * partial ideas to implement intel pt * forgot smth * trace decoding draft * add libipt decoder * use cpuid instead of reading /proc/cpuinfo * investigating nondeterministic behaviour * intel_pt module add thread creation hook * Fully identify deps versions Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates * Move mem image to module, output to file for debug * fixup! Use static_assertions crate * Exclude host kernel from traces * Bump libipt-rs * Callback to get memory as an alterantive to image * WIP Add bootloader fuzzer example * Split availability check: add availability_with_qemu * Move IntelPT to observer * Improve test docs * Clippy happy now * Taplo happy now * Add IntelPTObserver boilerplate * Hook instead of Observer * Clippy & Taplo * Add psb_freq setting * Extremely bad and dirty babyfuzzer stealing * Use thread local cell instead of mutex * Try a trace diff based naive feedback * fix perf aux buffer wrap handling * Use f64 for feedback score * Fix clippy for cargo test * Add config format tests * WIP intelpt babyfuzzer with fork * Fix not wrapped tail offset in split buffer * Baby PT with raw traces diff working * Cache nr_filters * Use Lazy_lock for perf_type * Add baby_fuzzer_intel_pt * restore baby fuzzer * baby_fuzzer with block decoder * instruction decoder instead of block * Fix after upstream merge * OwnedRefMut instead of Cow * Read mem directly instead of going through files * Fix cache lifetime and tail update * clippy * Taplo * Compile caps only on linux * clippy * Fail compilation on unsupported OSes * Add baby_fuzzer_intel_pt to CI * Cleanup * Move intel pt + linux check * fix baby pt * rollback forkexecutor * Remove unused dep * Cleanup * Lints * Compute an edge id instead of using only block ip * Binary only intelPT POC * put linux specific code behind target_os=linux * Clippy & Taplo * fix CI * Disable relocation * No unwrap in decode * No expect in decode * Better logging, smaller aux buffer * add IntelPTBuilder * some lints * Add exclude_hv config * Per CPU tracing and inheritance * Parametrize buffer size * Try not to break commandExecutor API pt.1 * Try not to break commandExecutor API pt.2 * Try not to break commandExecutor API pt.3 * fix baby PT * Support on_crash & on_timeout callbacks for libafl_qemu modules (#2620) * support (unsafe) on_crash / on_timeout callbacks for modules * use libc types in bindgen * Move common code to bolts * Cleanup * Revert changes to backtrace_baby_fuzzers/command_executor * Move intel_pt in one file * Use workspace deps * add nr_addr_filter fallback * Cleaning * Improve decode * Clippy * Improve errors and docs * Impl from<PtError> for libafl::Error * Merge hooks * Docs * Clean command executor * fix baby PT * fix baby PT warnings * decoder fills the map with no vec alloc * WIP command executor intel PT * filter_map() instead of filter().map() * fix docs * fix windows? * Baby lints * Small cleanings * Use personality to disable ASLR at runtime * Fix nix dep * Use prc-maps in babyfuzzer * working ET_DYN elf * Cleanup Cargo.toml * Clean command executor * introduce PtraceCommandConfigurator * Fix clippy & taplo * input via stdin * libipt as workspace dep * Check kernel version * support Arg input location * Reorder stuff * File input * timeout support for PtraceExec * Lints * Move out method not needing self form IntelPT * unimplemented * Lints * Move intel_pt_baby_fuzzer * Move intel_pt_command_executor * Document the need for smp_rmb * Better comment * Readme and Makefile.toml instead of build.rs * Move out from libafl_bolts to libafl_intelpt * Fix hooks * (Almost) fix intel_pt command exec * fix intel_pt command exec debug * Fix baby_fuzzer * &raw over addr_of! * cfg(target_os = "linux") * bolts Cargo.toml leftover * minimum wage README.md * extract join_split_trace from decode * extract decode_block from decode * add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0 * More generic hook * fix windows * Update CI, fmt * No bitbybit * Fix docker? * Fix Apple silicon? * Use old libipt from crates.io --------- Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Dominik Maier <[email protected]> * libafl-fuzz: introduce nyx_mode (#2503) * add nyx_mode * fix frida ci? * damn clippy * clippy * LibAFL: Remove `tui_monitor` from default features (#2685) * No Usermode default * no tui * gg * try fix CI * fmt --------- Co-authored-by: Dominik Maier <[email protected]> * Actually make ConstMapObserver work, introduce `nonnull_raw_mut` macro (#2687) * Actually make ConstMapObserver work * fixes * does that work? * mas * Feature: libafl-fuzzfuzzbench (#2689) * fuzzbench * clippy * fmt * fix unicorn CI? * Move bitfields to bitbybit (#2688) * move to bitbybit * Restore bitbybit dependent code * Clippy * Fix NautilusContext::from_file for python files (#2690) * Bump to 0.14.0 (#2692) * Fix versions in libafl and libafl_intelpt for crates.io (#2693) * Fix versions in libafl and libafl_intelpt for crates * Add libafl_intelpt to publish * StdMOptMutator::new: remove unused type parameter (#2695) `I` is unused in `::new` and thus requires callers to explicitly specify any type as it can't be determined by type inference. Clippy's `extra_unused_type_parameters` should pick this up, but is tuned a bit too conservative in order to avoid false positives AFAICT. * Move test_harness from source directory to OUT_DIR (#2694) * remove test_harness from source directory * fmt * Add package.metadata.docs.rs for libafl_intelpt (#2696) * libafl-fuzz: fix cmplog running on inputs more than once (#2697) * libafl-fuzz: fix cmplog running on inputs more than once * fmt * fix afl++ cmplog header * update to latest afl stable commit * Libafl workspace internal deps in workspace Cargo.toml (#2691) * Add internal deps to workspace * libafl: use workspace internal deps * libafl_bolts: use workspace internal deps * 0.14.0 * use workspace internal deps * Fix tui monitor for example fuzzers (#2699) * Fix tui monitor for example fuzzers * New clippy lint * fix * Update pyo3-build-config requirement from 0.22.3 to 0.23.1 (#2701) Updates the requirements on [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version. - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](PyO3/pyo3@v0.22.3...v0.23.1) --- updated-dependencies: - dependency-name: pyo3-build-config dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * bolts: fix build for tiers 3 platforms. (#2700) cater to platforms knowingly support this feature instead. * Pre init module hooks (#2704) * differenciate pre qemu init and post qemu init hooks * api breakage: Emulator::new_with_qemu is not public anymore. * Fix edge module generators (#2702) * fix generators * fix metadata removal for ExecutionCountRestartHelper (#2705) * Ignore pyo3 update (#2709) * libafl-fuzz: feature-flag nyx mode (#2712) * Bump ctor dependency to make nightly compile again (#2713) * Batched timeout doc (#2716) * timeout doc * clp * FMT * More batched timeout doc (#2717) * timeout doc * clp * FMT * more * fixing an overexited cast * renaming variables * removing unnecessary brackets * fixing imports * fixing imports * renaming more variables * even more variable renaming * removing duplicate clap short options * reverting mistaken variable renaming * comparing the actual cores instead of an enumeration index --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Dominik Maier <[email protected]> Co-authored-by: Subhojeet Mukherjee, PhD <[email protected]> Co-authored-by: jejuisland87654 <[email protected]> Co-authored-by: Marco C. <[email protected]> Co-authored-by: Dongjia "toka" Zhang <[email protected]> Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Aarnav <[email protected]> Co-authored-by: Dominik Maier <[email protected]> Co-authored-by: Andrea Fioraldi <[email protected]> Co-authored-by: Mrmaxmeier <[email protected]> Co-authored-by: Sharad Khanna <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: David CARLIER <[email protected]> Co-authored-by: Henry Chu <[email protected]>

* WIP: IntelPT qemu systemmode * use perf-event-open-sys instead of bindgen * intelPT Add enable and disable tracing, add test * Use static_assertions crate * Fix volatiles, finish test * Add Intel PT availability check * Use LibAFL errors in Result * Improve filtering * Add KVM pt_mode check * move static_assertions use * Check for perf_event_open support * Add (empty) IntelPT module * Add IntelPTModule POC * partial ideas to implement intel pt * forgot smth * trace decoding draft * add libipt decoder * use cpuid instead of reading /proc/cpuinfo * investigating nondeterministic behaviour * intel_pt module add thread creation hook * Fully identify deps versions Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates * Move mem image to module, output to file for debug * fixup! Use static_assertions crate * Exclude host kernel from traces * Bump libipt-rs * Callback to get memory as an alterantive to image * WIP Add bootloader fuzzer example * Split availability check: add availability_with_qemu * Move IntelPT to observer * Improve test docs * Clippy happy now * Taplo happy now * Add IntelPTObserver boilerplate * Hook instead of Observer * Clippy & Taplo * Add psb_freq setting * Extremely bad and dirty babyfuzzer stealing * Use thread local cell instead of mutex * Try a trace diff based naive feedback * fix perf aux buffer wrap handling * Use f64 for feedback score * Fix clippy for cargo test * Add config format tests * WIP intelpt babyfuzzer with fork * Fix not wrapped tail offset in split buffer * Baby PT with raw traces diff working * Cache nr_filters * Use Lazy_lock for perf_type * Add baby_fuzzer_intel_pt * restore baby fuzzer * baby_fuzzer with block decoder * instruction decoder instead of block * Fix after upstream merge * OwnedRefMut instead of Cow * Read mem directly instead of going through files * Fix cache lifetime and tail update * clippy * Taplo * Compile caps only on linux * clippy * Fail compilation on unsupported OSes * Add baby_fuzzer_intel_pt to CI * Cleanup * Move intel pt + linux check * fix baby pt * rollback forkexecutor * Remove unused dep * Cleanup * Lints * Compute an edge id instead of using only block ip * Binary only intelPT POC * put linux specific code behind target_os=linux * Clippy & Taplo * fix CI * Disable relocation * No unwrap in decode * No expect in decode * Better logging, smaller aux buffer * add IntelPTBuilder * some lints * Add exclude_hv config * Per CPU tracing and inheritance * Parametrize buffer size * Try not to break commandExecutor API pt.1 * Try not to break commandExecutor API pt.2 * Try not to break commandExecutor API pt.3 * fix baby PT * Support on_crash & on_timeout callbacks for libafl_qemu modules (AFLplusplus#2620) * support (unsafe) on_crash / on_timeout callbacks for modules * use libc types in bindgen * Move common code to bolts * Cleanup * Revert changes to backtrace_baby_fuzzers/command_executor * Move intel_pt in one file * Use workspace deps * add nr_addr_filter fallback * Cleaning * Improve decode * Clippy * Improve errors and docs * Impl from<PtError> for libafl::Error * Merge hooks * Docs * Clean command executor * fix baby PT * fix baby PT warnings * decoder fills the map with no vec alloc * WIP command executor intel PT * filter_map() instead of filter().map() * fix docs * fix windows? * Baby lints * Small cleanings * Use personality to disable ASLR at runtime * Fix nix dep * Use prc-maps in babyfuzzer * working ET_DYN elf * Cleanup Cargo.toml * Clean command executor * introduce PtraceCommandConfigurator * Fix clippy & taplo * input via stdin * libipt as workspace dep * Check kernel version * support Arg input location * Reorder stuff * File input * timeout support for PtraceExec * Lints * Move out method not needing self form IntelPT * unimplemented * Lints * Move intel_pt_baby_fuzzer * Move intel_pt_command_executor * Document the need for smp_rmb * Better comment * Readme and Makefile.toml instead of build.rs * Move out from libafl_bolts to libafl_intelpt * Fix hooks * (Almost) fix intel_pt command exec * fix intel_pt command exec debug * Fix baby_fuzzer * &raw over addr_of! * cfg(target_os = "linux") * bolts Cargo.toml leftover * minimum wage README.md * extract join_split_trace from decode * extract decode_block from decode * add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0 * More generic hook * fix windows * Update CI, fmt * No bitbybit * Fix docker? * Fix Apple silicon? * Use old libipt from crates.io --------- Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Dominik Maier <[email protected]>

* launcher now uses client_id instead of core_id * adding overcommit to an example fuzzer * Replace addr_of with &raw across the codebase (AFLplusplus#2669) * Replace addr_of with &raw across the codebase * fix fixes * more fix * undo clang fmt? * oops * fix? * allocator fix * more fix * more more * more docs * more fix * mas mas mas * hm * more * fix Frida * needed * more error * qemu * Introduce workspace (again) (AFLplusplus#2673) * Trying to redo workspace deps again after AFLplusplus#2672 * unused * clippy * Replace addr_of with &raw across the codebase (AFLplusplus#2669) * Replace addr_of with &raw across the codebase * fix fixes * more fix * undo clang fmt? * oops * fix? * allocator fix * more fix * more more * more docs * more fix * mas mas mas * hm * more * fix Frida * needed * more error * qemu * Introduce workspace (again) (AFLplusplus#2673) * Trying to redo workspace deps again after AFLplusplus#2672 * unused * clippy * fixing formatting issues * cloning values to make borrow checker happy * simplifying cfg constraints, removing excessive clippy allows * printing clang version that is used to find inconsistencies between CI and local formatting * some fixes according to the CI * Specifying types * improved logging for formatter * more attempts at logging for the CI formatting * fixing setting LLVM version in formatting in CI * fixing cippy allows * renaming launcher's ClientID to ClientDescription * Lower capped RAND generators (AFLplusplus#2671) * Lower capped rand generators * Updated all references to RAND generators * Formatting updates * New RAND bytes generator constructor * Revert "Updated all references to RAND generators" This reverts commit 9daad89. * Revert "Formatting updates" This reverts commit ff2a61a. * cargo nightly format * Added must_use to with_min_size * fix error '#' is not followed by a macro parameter (AFLplusplus#2678) * Use version.workspace (AFLplusplus#2682) * LibAFL_QEMU: Don't return a generic Address from Register reads (AFLplusplus#2681) * LibAFL_QEMU: Make ReadReg always return GuestReg type * Don't return a generic address * fix fuzzers * fix mips * Add DrCovReader to read DrCov files and DrCov dumper and merge utils (AFLplusplus#2680) * Add DrCov Reader * Removed libafl_jumper deps * Fix DrCovWriter, add dump_drcov_addrs * Taplo * Move frida from usize to u64 * DrCov usize=>u64 * Better error print * More u64 * ? * debug * clippy * clippy * Add Merge option to DrCovReader * Add drcov_merge tool * Move folder around * DrCov * More assert * fmt * Move around * Fix print * Add option to read multiple files/full folders * Fix build_all_fuzzers.sh for local runs (AFLplusplus#2686) * Add Intel PT tracing support (AFLplusplus#2471) * WIP: IntelPT qemu systemmode * use perf-event-open-sys instead of bindgen * intelPT Add enable and disable tracing, add test * Use static_assertions crate * Fix volatiles, finish test * Add Intel PT availability check * Use LibAFL errors in Result * Improve filtering * Add KVM pt_mode check * move static_assertions use * Check for perf_event_open support * Add (empty) IntelPT module * Add IntelPTModule POC * partial ideas to implement intel pt * forgot smth * trace decoding draft * add libipt decoder * use cpuid instead of reading /proc/cpuinfo * investigating nondeterministic behaviour * intel_pt module add thread creation hook * Fully identify deps versions Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates * Move mem image to module, output to file for debug * fixup! Use static_assertions crate * Exclude host kernel from traces * Bump libipt-rs * Callback to get memory as an alterantive to image * WIP Add bootloader fuzzer example * Split availability check: add availability_with_qemu * Move IntelPT to observer * Improve test docs * Clippy happy now * Taplo happy now * Add IntelPTObserver boilerplate * Hook instead of Observer * Clippy & Taplo * Add psb_freq setting * Extremely bad and dirty babyfuzzer stealing * Use thread local cell instead of mutex * Try a trace diff based naive feedback * fix perf aux buffer wrap handling * Use f64 for feedback score * Fix clippy for cargo test * Add config format tests * WIP intelpt babyfuzzer with fork * Fix not wrapped tail offset in split buffer * Baby PT with raw traces diff working * Cache nr_filters * Use Lazy_lock for perf_type * Add baby_fuzzer_intel_pt * restore baby fuzzer * baby_fuzzer with block decoder * instruction decoder instead of block * Fix after upstream merge * OwnedRefMut instead of Cow * Read mem directly instead of going through files * Fix cache lifetime and tail update * clippy * Taplo * Compile caps only on linux * clippy * Fail compilation on unsupported OSes * Add baby_fuzzer_intel_pt to CI * Cleanup * Move intel pt + linux check * fix baby pt * rollback forkexecutor * Remove unused dep * Cleanup * Lints * Compute an edge id instead of using only block ip * Binary only intelPT POC * put linux specific code behind target_os=linux * Clippy & Taplo * fix CI * Disable relocation * No unwrap in decode * No expect in decode * Better logging, smaller aux buffer * add IntelPTBuilder * some lints * Add exclude_hv config * Per CPU tracing and inheritance * Parametrize buffer size * Try not to break commandExecutor API pt.1 * Try not to break commandExecutor API pt.2 * Try not to break commandExecutor API pt.3 * fix baby PT * Support on_crash & on_timeout callbacks for libafl_qemu modules (AFLplusplus#2620) * support (unsafe) on_crash / on_timeout callbacks for modules * use libc types in bindgen * Move common code to bolts * Cleanup * Revert changes to backtrace_baby_fuzzers/command_executor * Move intel_pt in one file * Use workspace deps * add nr_addr_filter fallback * Cleaning * Improve decode * Clippy * Improve errors and docs * Impl from<PtError> for libafl::Error * Merge hooks * Docs * Clean command executor * fix baby PT * fix baby PT warnings * decoder fills the map with no vec alloc * WIP command executor intel PT * filter_map() instead of filter().map() * fix docs * fix windows? * Baby lints * Small cleanings * Use personality to disable ASLR at runtime * Fix nix dep * Use prc-maps in babyfuzzer * working ET_DYN elf * Cleanup Cargo.toml * Clean command executor * introduce PtraceCommandConfigurator * Fix clippy & taplo * input via stdin * libipt as workspace dep * Check kernel version * support Arg input location * Reorder stuff * File input * timeout support for PtraceExec * Lints * Move out method not needing self form IntelPT * unimplemented * Lints * Move intel_pt_baby_fuzzer * Move intel_pt_command_executor * Document the need for smp_rmb * Better comment * Readme and Makefile.toml instead of build.rs * Move out from libafl_bolts to libafl_intelpt * Fix hooks * (Almost) fix intel_pt command exec * fix intel_pt command exec debug * Fix baby_fuzzer * &raw over addr_of! * cfg(target_os = "linux") * bolts Cargo.toml leftover * minimum wage README.md * extract join_split_trace from decode * extract decode_block from decode * add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0 * More generic hook * fix windows * Update CI, fmt * No bitbybit * Fix docker? * Fix Apple silicon? * Use old libipt from crates.io --------- Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Dominik Maier <[email protected]> * libafl-fuzz: introduce nyx_mode (AFLplusplus#2503) * add nyx_mode * fix frida ci? * damn clippy * clippy * LibAFL: Remove `tui_monitor` from default features (AFLplusplus#2685) * No Usermode default * no tui * gg * try fix CI * fmt --------- Co-authored-by: Dominik Maier <[email protected]> * Actually make ConstMapObserver work, introduce `nonnull_raw_mut` macro (AFLplusplus#2687) * Actually make ConstMapObserver work * fixes * does that work? * mas * Feature: libafl-fuzzfuzzbench (AFLplusplus#2689) * fuzzbench * clippy * fmt * fix unicorn CI? * Move bitfields to bitbybit (AFLplusplus#2688) * move to bitbybit * Restore bitbybit dependent code * Clippy * Fix NautilusContext::from_file for python files (AFLplusplus#2690) * Bump to 0.14.0 (AFLplusplus#2692) * Fix versions in libafl and libafl_intelpt for crates.io (AFLplusplus#2693) * Fix versions in libafl and libafl_intelpt for crates * Add libafl_intelpt to publish * StdMOptMutator::new: remove unused type parameter (AFLplusplus#2695) `I` is unused in `::new` and thus requires callers to explicitly specify any type as it can't be determined by type inference. Clippy's `extra_unused_type_parameters` should pick this up, but is tuned a bit too conservative in order to avoid false positives AFAICT. * Move test_harness from source directory to OUT_DIR (AFLplusplus#2694) * remove test_harness from source directory * fmt * Add package.metadata.docs.rs for libafl_intelpt (AFLplusplus#2696) * libafl-fuzz: fix cmplog running on inputs more than once (AFLplusplus#2697) * libafl-fuzz: fix cmplog running on inputs more than once * fmt * fix afl++ cmplog header * update to latest afl stable commit * Libafl workspace internal deps in workspace Cargo.toml (AFLplusplus#2691) * Add internal deps to workspace * libafl: use workspace internal deps * libafl_bolts: use workspace internal deps * 0.14.0 * use workspace internal deps * Fix tui monitor for example fuzzers (AFLplusplus#2699) * Fix tui monitor for example fuzzers * New clippy lint * fix * Update pyo3-build-config requirement from 0.22.3 to 0.23.1 (AFLplusplus#2701) Updates the requirements on [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version. - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](PyO3/pyo3@v0.22.3...v0.23.1) --- updated-dependencies: - dependency-name: pyo3-build-config dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * bolts: fix build for tiers 3 platforms. (AFLplusplus#2700) cater to platforms knowingly support this feature instead. * Pre init module hooks (AFLplusplus#2704) * differenciate pre qemu init and post qemu init hooks * api breakage: Emulator::new_with_qemu is not public anymore. * Fix edge module generators (AFLplusplus#2702) * fix generators * fix metadata removal for ExecutionCountRestartHelper (AFLplusplus#2705) * Ignore pyo3 update (AFLplusplus#2709) * libafl-fuzz: feature-flag nyx mode (AFLplusplus#2712) * Bump ctor dependency to make nightly compile again (AFLplusplus#2713) * Batched timeout doc (AFLplusplus#2716) * timeout doc * clp * FMT * More batched timeout doc (AFLplusplus#2717) * timeout doc * clp * FMT * more * fixing an overexited cast * renaming variables * removing unnecessary brackets * fixing imports * fixing imports * renaming more variables * even more variable renaming * removing duplicate clap short options * reverting mistaken variable renaming * comparing the actual cores instead of an enumeration index --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Dominik Maier <[email protected]> Co-authored-by: Subhojeet Mukherjee, PhD <[email protected]> Co-authored-by: jejuisland87654 <[email protected]> Co-authored-by: Marco C. <[email protected]> Co-authored-by: Dongjia "toka" Zhang <[email protected]> Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Aarnav <[email protected]> Co-authored-by: Dominik Maier <[email protected]> Co-authored-by: Andrea Fioraldi <[email protected]> Co-authored-by: Mrmaxmeier <[email protected]> Co-authored-by: Sharad Khanna <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: David CARLIER <[email protected]> Co-authored-by: Henry Chu <[email protected]>

Marcondiro force-pushed the intel_pt branch 2 times, most recently from 615b87b to b4ae309 Compare August 2, 2024 16:28

rmalmain self-requested a review August 2, 2024 17:26

domenukk reviewed Aug 3, 2024

View reviewed changes

libafl_qemu/build_linux.rs Outdated Show resolved Hide resolved

Marcondiro force-pushed the intel_pt branch 3 times, most recently from 138002d to 121d6ca Compare August 5, 2024 08:40

rmalmain reviewed Aug 5, 2024

View reviewed changes

Marcondiro force-pushed the intel_pt branch 4 times, most recently from e65ba68 to 0a8d8ba Compare August 13, 2024 07:21

Marcondiro force-pushed the intel_pt branch 7 times, most recently from 30489e9 to 354ad86 Compare August 20, 2024 16:15

Marcondiro force-pushed the intel_pt branch 2 times, most recently from a2990ad to 8d72014 Compare August 22, 2024 10:27

domenukk reviewed Aug 22, 2024

View reviewed changes

libafl_qemu/Cargo.toml Outdated Show resolved Hide resolved

domenukk reviewed Aug 24, 2024

View reviewed changes

libafl_qemu/Cargo.toml Outdated Show resolved Hide resolved

Marcondiro force-pushed the intel_pt branch from ae1cfc0 to 17c3ccf Compare September 4, 2024 08:42

Marcondiro force-pushed the intel_pt branch from d26447d to 736ac50 Compare September 11, 2024 12:08

Marcondiro force-pushed the intel_pt branch from 1d1104f to 40d434d Compare September 20, 2024 12:00

Marcondiro added 4 commits November 8, 2024 13:49

extract join_split_trace from decode

d31133f

extract decode_block from decode

9c17590

add 1 to previous_block_ip to avoid that all the recursive basic bl…

8cad188

…ocks map to 0

More generic hook

8972400

domenukk reviewed Nov 8, 2024

View reviewed changes

.github/workflows/build_and_test.yml Outdated Show resolved Hide resolved

domenukk reviewed Nov 8, 2024

View reviewed changes

Marcondiro and others added 2 commits November 9, 2024 16:31

fix windows

08eb14d

Merge branch 'main' into intel_pt

88a960f

domenukk reviewed Nov 9, 2024

View reviewed changes

Merge branch 'main' into intel_pt

21ed10c

Marcondiro added 6 commits November 11, 2024 21:52

Merge branch 'main' into intel_pt

dc0a2ca

Update CI, fmt

ccc793e

No bitbybit

b38b9d5

Fix docker?

89cf7a3

Fix Apple silicon?

6499c03

Merge branch 'main' into intel_pt

14add06

Marcondiro added 2 commits November 12, 2024 11:29

Use old libipt from crates.io

59971d4

Merge branch 'main' into intel_pt

6600cb5

domenukk reviewed Nov 13, 2024

View reviewed changes

domenukk merged commit f7f8dff into AFLplusplus:main Nov 13, 2024
89 of 102 checks passed

Marcondiro mentioned this pull request Nov 13, 2024

move to bitbybit #2688

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Intel PT tracing support #2471

Add Intel PT tracing support #2471

Marcondiro commented Aug 2, 2024

addisoncrump commented Aug 4, 2024

domenukk commented Aug 5, 2024

Marcondiro commented Aug 5, 2024 via email •

edited

Loading

domenukk commented Aug 5, 2024

rmalmain left a comment

rmalmain Aug 5, 2024

domenukk Aug 7, 2024

Marcondiro Aug 7, 2024

domenukk Aug 7, 2024

domenukk Nov 8, 2024

domenukk Nov 9, 2024

Marcondiro Nov 10, 2024

domenukk Nov 11, 2024

Marcondiro Nov 12, 2024

domenukk commented Nov 11, 2024

Marcondiro commented Nov 12, 2024

rmalmain commented Nov 12, 2024

domenukk Nov 13, 2024

Marcondiro Nov 13, 2024

domenukk commented Nov 13, 2024

domenukk commented Nov 13, 2024

Add Intel PT tracing support #2471

Add Intel PT tracing support #2471

Conversation

Marcondiro commented Aug 2, 2024

addisoncrump commented Aug 4, 2024

domenukk commented Aug 5, 2024

Marcondiro commented Aug 5, 2024 via email • edited Loading

domenukk commented Aug 5, 2024

rmalmain left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenukk commented Nov 11, 2024

Marcondiro commented Nov 12, 2024

rmalmain commented Nov 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenukk commented Nov 13, 2024

domenukk commented Nov 13, 2024

Marcondiro commented Aug 5, 2024 via email •

edited

Loading