Print a more meaningful message if WithinULP fails

[cschwan]

First of all: Thanks for making Catch2! Its been a really useful library to me for a few years now and still is. I am especially happy about the WithinULP check, which no other testing library seems to have and which is an essential check for me.

However, I think one could improve the message when the check fails. In particular, what I am seeing is a message similar to the following one:

../tests/test_vbs_wpz_real_a6as1_qcd.cpp:96: FAILED:
  CHECK_THAT( results.front().value(), Catch::WithinULP(T(7.7623532726228692e-03), 128) )
with expansion:
  0.0077623533 is within 128 ULPs of 0.0077623533

This message doesn't help me a lot. Are the 128 ULPs to tight? What is the actual difference? How do both numbers look with their full precision?

What I would like to see is, at the very least, a message with the two numbers printed with their full precision (for example scientific notation + precision depending on the numeric type). The full precision is needed if I decide to update my reference value. One could also think about (additionally?) printing the numbers in hexfloat, which is an exact representation, but that might not be useful enough.

A very helpful information would be the actual difference in ULPs. If I decide that 128 digits are too small, by how many ULP do I need to increase the 128? What I am doing right now is to increase the number by two and rerun the test to see if it works, but that's very inefficient.

[horenmar]

I think there are two parts to this issue

1. The stringification done by the ULP matcher could be better, e.g. it could show the acceptable range or format the doubles in a better format
2. The stringification done by the ULP matcher is independent of the input data.

The first can definitely be fixed, I'll admit that when I made the ULP matcher I optimized towards correctness and having it done, and didn't give much thought to useful textual output.

The second is not currently possible to fix, because the Matcher framework assumes that the describe function can be called independently on the result of match (or match even ever being called). At first glance, this is an overly restrictive requirement, as there is no reason for describe to be called if the matcher hasn't been used (given something to match against), but consider matcher chaining via && and ||:

REQUIRE_THAT("Foo", StartsWith("B") && EndsWith("r"));

In this case, the matching will fail for the first matcher and the second matcher thus will not get to see the argument at all, but the combined expression's description should still describe the second matcher as well.