Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an observer for COUNTERS_MAPS for 8-bit SanCov #1283

Merged
merged 11 commits into from
May 23, 2023
Merged
2 changes: 2 additions & 0 deletions libafl_targets/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,6 @@ log = "0.4.17"

rangemap = "1.0"
serde = { version = "1.0", default-features = false, features = ["alloc"] } # serialization lib
intervaltree = "0.2.7"
ahash = "0.8.3"
# serde-big-array = "0.3.2"
290 changes: 288 additions & 2 deletions libafl_targets/src/sancov_8bit.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,26 @@
//! [`LLVM` `8-bi-counters`](https://clang.llvm.org/docs/SanitizerCoverage.html#tracing-pcs-with-guards) runtime for `LibAFL`.
use alloc::vec::Vec;
use alloc::{
string::{String, ToString},
vec::Vec,
};
use core::{
fmt::Debug,
hash::{BuildHasher, Hasher},
iter::Flatten,
slice::{from_raw_parts, Iter, IterMut},
};

use libafl::bolts::ownedref::OwnedMutSlice;
use ahash::RandomState;
use intervaltree::IntervalTree;
use libafl::{
bolts::{
ownedref::OwnedMutSlice, tuples::Named, AsIter, AsIterMut, AsMutSlice, AsSlice, HasLen,
},
inputs::UsesInput,
observers::{DifferentialObserver, MapObserver, Observer, ObserversTuple},
Error,
};
use serde::{Deserialize, Serialize};

/// A [`Vec`] of `8-bit-counters` maps for multiple modules.
/// They are initialized by calling [`__sanitizer_cov_8bit_counters_init`](
Expand All @@ -19,3 +38,270 @@ pub extern "C" fn __sanitizer_cov_8bit_counters_init(start: *mut u8, stop: *mut
));
}
}

#[must_use]
/// Create a new [`CountersMultiMapObserver`] of the [`COUNTERS_MAP`].
///
/// This is a special [`MultiMapObserver`] for the [`COUNTERS_MAP`] and may be used when
/// 8-bit counters are used for `SanitizerCoverage`. You can utilize this observer in a
/// [`HitcountsIterableMapObserver`] like so:
///
/// ```rust,ignore
/// use libafl::{
/// observers::HitcountsIterableMapObserver,
/// feedbacks::MaxMapFeedback,
/// };
/// use libafl_targets::sancov_8bit::counters_maps_observer;
///
/// let counters_maps_observer = unsafe { counters_maps_observer("counters-maps") };
/// let counters_maps_hitcounts_observer = HitcountsIterableMapObserver::new(counters_maps_observer);
/// let counters_maps_feedback = MaxMapFeedback::new(&counters_maps_hitcounts_observer);
/// ```
///
/// # Safety
///
/// This function instantiates an observer of a `static mut` map whose contents are mutated by
/// `SanitizerCoverage` instrumentation. This is unsafe, and data in the map may be mutated from
/// under us at any time. It should never be assumed constant.
pub unsafe fn counters_maps_observer(name: &'static str) -> CountersMultiMapObserver<false> {
CountersMultiMapObserver::new(name)
}

/// The [`CountersMultiMapObserver`] observes all the counters that may be set by
/// `SanitizerCoverage` in [`COUNTERS_MAPS`]
#[derive(Serialize, Deserialize, Debug)]
#[allow(clippy::unsafe_derive_deserialize)]
pub struct CountersMultiMapObserver<const DIFFERENTIAL: bool> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The observers should live in the main lib, not in libafl_targets (I think)

Copy link
Contributor Author

@novafacing novafacing May 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having libafl depend on libafl_targets to make the static visible would introduce a circular dependency, so I think it does have to go in here. I could be missing another option though.

(it can also go in your binary of course, but then I can't reuse this observer 😋)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, I thought the dependency went the other way.
But in this case, maybe we should hide additional observers behind a observers flag? Dragging in intervaltree and ahash for this seems excessive for most usecases, what do you think?

Copy link
Contributor Author

@novafacing novafacing May 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah libafl_targets is already full of feature flags, no biggie to add another :)

I pushed that change up adding an observers flag and moved the observer into there.

Checking over the source, I notice that cmplog.rs has CmpLogObserver and coverage.rs has DifferentialAFLMapSwapObserver (although it is already behind the "pointer_maps" flag). Probably not be worth gating the CmpLogObserver since it doesn't add any dependencies, but it might be worth gating all three the same way to trim down static library size and make things consistent.

One more note is that DifferentialAFLMapSwapObserver uses serde, but it's the only thing other than CountersMapsObserver that does. We should probably make serde an optional dependency as well (we could gate it behind "observers" if we add it as a feature gate for all the observers in the crate), because it has the "alloc" feature enabled which makes it not possible to use libafl_targets in no_std currently.

(This is probably another PR, happy to open it if these suggestions sound good)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. FWIW using alloc in no_std is totally possible. You just need to define an allocator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, today I learned!

intervals: IntervalTree<usize, usize>,
len: usize,
initial: u8,
name: String,
iter_idx: usize,
}

impl<S> Observer<S> for CountersMultiMapObserver<false>
where
S: UsesInput,
Self: MapObserver,
{
#[inline]
fn pre_exec(&mut self, _state: &mut S, _input: &S::Input) -> Result<(), Error> {
self.reset_map()
}
}

impl<S> Observer<S> for CountersMultiMapObserver<true>
where
S: UsesInput,
Self: MapObserver,
{
// in differential mode, we are *not* responsible for resetting the map!
}

impl<const DIFFERENTIAL: bool> Named for CountersMultiMapObserver<DIFFERENTIAL> {
#[inline]
fn name(&self) -> &str {
self.name.as_str()
}
}

impl<const DIFFERENTIAL: bool> HasLen for CountersMultiMapObserver<DIFFERENTIAL> {
#[inline]
fn len(&self) -> usize {
self.len
}
}

impl<const DIFFERENTIAL: bool> MapObserver for CountersMultiMapObserver<DIFFERENTIAL> {
type Entry = u8;

#[inline]
fn get(&self, idx: usize) -> &u8 {
let elem = self.intervals.query_point(idx).next().unwrap();
let i = elem.value;
let j = idx - elem.range.start;
unsafe { &COUNTERS_MAPS[i].as_slice()[j] }
}

#[inline]
fn get_mut(&mut self, idx: usize) -> &mut u8 {
let elem = self.intervals.query_point(idx).next().unwrap();
let i = elem.value;
let j = idx - elem.range.start;
unsafe { &mut COUNTERS_MAPS[i].as_mut_slice()[j] }
}

#[inline]
fn initial(&self) -> u8 {
self.initial
}

fn count_bytes(&self) -> u64 {
let initial = self.initial();
let mut res = 0;
for map in unsafe { &COUNTERS_MAPS } {
for x in map.as_slice() {
if *x != initial {
res += 1;
}
}
}
res
}

fn hash(&self) -> u64 {
let mut hasher = RandomState::with_seeds(0, 0, 0, 0).build_hasher();
for map in unsafe { &COUNTERS_MAPS } {
let slice = map.as_slice();
let ptr = slice.as_ptr() as *const u8;
let map_size = slice.len() / core::mem::size_of::<u8>();
unsafe {
hasher.write(from_raw_parts(ptr, map_size));
}
}
hasher.finish()
}

fn reset_map(&mut self) -> Result<(), Error> {
let initial = self.initial();
for map in unsafe { &mut COUNTERS_MAPS } {
for x in map.as_mut_slice() {
*x = initial;
}
}
Ok(())
}

fn usable_count(&self) -> usize {
self.len()
}

fn to_vec(&self) -> Vec<Self::Entry> {
let cnt = self.usable_count();
let mut res = Vec::with_capacity(cnt);
for i in 0..cnt {
res.push(*self.get(i));
}
res
}

/// Get the number of set entries with the specified indexes
fn how_many_set(&self, indexes: &[usize]) -> usize {
let initial = self.initial();
let cnt = self.usable_count();
let mut res = 0;
for i in indexes {
if *i < cnt && *self.get(*i) != initial {
res += 1;
}
}
res
}
}

impl<const DIFFERENTIAL: bool> CountersMultiMapObserver<DIFFERENTIAL> {
/// Creates a new [`CountersMultiMapObserver`], maybe in differential mode
#[must_use]
fn maybe_differential(name: &'static str) -> Self {
let mut idx = 0;
let mut builder = vec![];
for (v, x) in unsafe { &COUNTERS_MAPS }.iter().enumerate() {
let l = x.as_slice().len();
let r = (idx..(idx + l), v);
idx += l;
builder.push(r);
}
Self {
intervals: builder.into_iter().collect::<IntervalTree<usize, usize>>(),
len: idx,
name: name.to_string(),
initial: u8::default(),
iter_idx: 0,
}
}
}

impl CountersMultiMapObserver<true> {
/// Creates a new [`CountersMultiMapObserver`] in differential mode
#[must_use]
pub fn differential(name: &'static str) -> Self {
Self::maybe_differential(name)
}
}

impl CountersMultiMapObserver<false> {
/// Creates a new [`CountersMultiMapObserver`]
#[must_use]
pub fn new(name: &'static str) -> Self {
Self::maybe_differential(name)
}

/// Creates a new [`CountersMultiMapObserver`] with an owned map
#[must_use]
pub fn owned(name: &'static str) -> Self {
let mut idx = 0;
let mut v = 0;
let mut builder = vec![];
unsafe { &mut COUNTERS_MAPS }.iter_mut().for_each(|m| {
let l = m.as_mut_slice().len();
let r = (idx..(idx + l), v);
idx += l;
builder.push(r);
v += 1;
});
Self {
intervals: builder.into_iter().collect::<IntervalTree<usize, usize>>(),
len: idx,
name: name.to_string(),
initial: u8::default(),
iter_idx: 0,
}
}
}

impl<'it, const DIFFERENTIAL: bool> AsIter<'it> for CountersMultiMapObserver<DIFFERENTIAL> {
type Item = u8;
type IntoIter = Flatten<Iter<'it, OwnedMutSlice<'static, u8>>>;

fn as_iter(&'it self) -> Self::IntoIter {
unsafe { COUNTERS_MAPS.iter().flatten() }
}
}

impl<'it, const DIFFERENTIAL: bool> AsIterMut<'it> for CountersMultiMapObserver<DIFFERENTIAL> {
type Item = u8;
type IntoIter = Flatten<IterMut<'it, OwnedMutSlice<'static, u8>>>;

fn as_iter_mut(&'it mut self) -> Self::IntoIter {
unsafe { COUNTERS_MAPS.iter_mut().flatten() }
}
}

impl<'it, const DIFFERENTIAL: bool> IntoIterator for &'it CountersMultiMapObserver<DIFFERENTIAL> {
type Item = <Iter<'it, u8> as Iterator>::Item;
type IntoIter = Flatten<Iter<'it, OwnedMutSlice<'static, u8>>>;

fn into_iter(self) -> Self::IntoIter {
unsafe { &COUNTERS_MAPS }.iter().flatten()
}
}

impl<'it, const DIFFERENTIAL: bool> IntoIterator
for &'it mut CountersMultiMapObserver<DIFFERENTIAL>
{
type Item = <IterMut<'it, u8> as Iterator>::Item;
type IntoIter = Flatten<IterMut<'it, OwnedMutSlice<'static, u8>>>;

fn into_iter(self) -> Self::IntoIter {
unsafe { &mut COUNTERS_MAPS }.iter_mut().flatten()
}
}

impl<OTA, OTB, S> DifferentialObserver<OTA, OTB, S> for CountersMultiMapObserver<true>
where
Self: MapObserver,
OTA: ObserversTuple<S>,
OTB: ObserversTuple<S>,
S: UsesInput,
{
}