Catch2 does not report unmatched parts of a partially-matched test spec

[fgamador]

Description

Catch2's runTests function calls the reporter's noMatchingTestCases if no tests match the spec, but if some test matches some part of the spec, nothing reports that the rest of the spec matched nothing. In other words, if I pass "Test1 Test2" as command-line args, and "Test1" matches some test, nothing complains about the fact that "Test2" matched nothing. This can lead the user to believe that they're running all the tests in the spec when they are not.

Steps to reproduce

Create a test file with one test. Run it with command-line args that specify two tests: the existing test and
some other, non-existent test. Everything will be green and happy, with no indication that the second test was not found.

Extra information

• Catch version: v2.4.1
• Operating System: Windows 10
• Compiler+version: MS VS 2015

[AlexisWilke]

This is actually true even if you run with just "test2". That is, you are told "No tests ran", but the test calls exit(0). So to my script it looks like everything ran as expected.

I think that:

1. It should report errors for any spec. which does not have a match; and
2. If any such error is found, the test should return 1 (i.e. exit(1)) so my scripts break as expected on errors.

[sfranzen]

You're right. The main issue can be fixed with some changes to the test case filtering mechanism. As for the exit code issue, going by the documentation, the command-line option -w NoTests should do what @AlexisWilke wants, but the code does not seem to respect it. This is also easy to fix. I'll have a go at it soon.